SUNTORY TOKUCHA MUSIC

OVERVIEW

At Dentsu Craft Tokyo, we created a new web service “TOKUCHA MUSIC” for Suntory Iyemon Tokucha, one of the pioneers of green tea functional beverages with specific medicinal benefits. “TOKUCHA MUSIC” is that uses AI to generate music based on user specified daily meals. The tempo, melody, and tone of the music change depending on the nutritional balance and type of food. The goal was to give people a new perspective on the balance of their daily diet through music. Qosmo has provided our original AI music generation system for this project.

BACKGROUND

A survey by Suntory in the fall of 2020 showed that in the new lifestyle triggered by the COVID-19, health behaviors related to diet, such as “taking breakfast” and “eating vegetables first,” have increased compared to 2019, indicating that people are becoming more conscious of their diet. Therefore, Suntory Iyemon Tokucha brand decided to develop “TOKUCHA MUSIC”, a service that easily converts your daily meals into music. Behind this project, we try to give users awareness of the opportunity to be healthy in a fun way through music.

TECHNOLOGY

This system is based on our original deep learning model (Variational Autoencoder using Transformer as encoder and decoder) and can generate multi-track MIDI files with drums, melody, bass line, and harmony based on conditions such as pitch height, note density, chord progression, and balance of consonance/dissonance

In this system, the nutritional balance of the diet entered by the user is calculated in terms of point ranges of food groups, called the four-gun point method. It is then used as an input for the deep learning model that creates music whose atmosphere reflects this nutritional balance.

If the user has a nutritional imbalance or too much fat or carbohydrates, the system will generate dissonant music with minor chords. On the contrary, if the user has a healthy meal with appropriate fruits and vegetables it will have light, up-tempo music. In addition, by using musical instruments that match the genre of the food, the atmosphere of the meal is represented in the music (Koto or Shakuhachi for Japanese food, flamenco guitar for Spanish food, etc.). Since the users can choose from about 2,000 different meals, they can repeatedly enjoy the countless variations of music generated by the AI. We used Web Audio technology to synthesize the instrument sounds and built the system to run on a web browser, including mobile devices.

In this way, this project has succeeded in creating an opportunity for people to review their daily dietary habits in a fun way through music by using AI music generation technology.

This project is also an interesting example of AI-based sonification and musicification of data. Moreover, it is impossible for human composers to generate countless variations of music based on specific data input, which can be seen as a unique form of music made possible through AI. Qosmo will continue to explore new applications of music and new types of music by using AI.

---

LINKS

---

CLIENT

---

CREDITS

• Creative Direction

  Takashi Kawashima（Studio Kawashima）

• Technical Direction/Machine Learning

  Nao Tokui（Qosmo）

• Art Director

  Yuki Sakurai（Metamosphere inc.）

• Planning

  Ryosuke Sone（Dentsu Craft Tokyo）

• Produce

  Maho Tada（Dentsu Craft Tokyo）,Jun Kato（Dentsu Craft Tokyo）,Kikuko Ando（Dentsu Craft Tokyo）

• Project Management

  Tatsuya Maniwa（Dentsu Craft Tokyo）,Sakiko Yasue（Qosmo）, Yumi Takahashi（Qosmo）

• Audio Development

  Robin Jungers（Qosmo）

• Machine Learning

  Christopher Mitcheltree（Qosmo）

• Technical Direction

  Hiroyoshi Murata（Dentsu Craft Tokyo）

• Front Engineering

  Eiki Kurokawa（Dentsu Craft Tokyo）,Atsushi Asakura（Dentsu Craft Tokyo）,Yuki Tanabe（Dentsu Craft Tokyo）

• Direction Support

  Kei Sato（Dentsu Craft Tokyo）

• Backend Engineering

  Hajime Sasaki（MountPosition Inc. ）

• Sound Design/Sound Produce

  Masayuki Noda（invisible design lab）

• Design

  Ikuko Komamoto（Dentsu Creative X）

• Copy Writing

  Keiko Taguchi（Dentsu Creative X）,Shiho Sasaki（Dentsu Creative X）,Yui Mamiya（Dentsu Creative X）