BGM Selection System Proof of Concept Using AI
In the morning, refreshing music fills the air. Toward noon, the mood shifts, picking up tempo. By evening, the music has mellowed. Across genres and time-periods, the stream of music continues throughout the day. The playlist is being generated by AI, providing listeners with a soundtrack attuned to that place and time.
AI BGM is a project aimed at customizing playlists, precisely tailored to individual retail spaces. Selecting from songs distributed by USEN, a member of the USEN-Next Group, the AI selects background music that changes along with the seasons, the weather, the time of day, and other factors.
The first phase consists of a prototype at The Core Kitchen/Space in Toranomon Hills, which began on July 1, 2019 and is scheduled to run for one year.
In this joint project with USEN and Mori Building Co., Ltd., Qosmo has been tasked with the AI development.
One of Qosmo’s previous projects is the “AI DJ Project—A Dialogue Between AI and a Human.” The objective of the project is to explore the relationship that develops between a human being and an AI system in an expressive activity, with AI playing the role of DJ.
Performances using this system are ongoing, both in Japan and overseas. At Google I/O 2019, a developer conference held in May 2019 in Mountain View, California, USA, the AI DJ Project was presented as the opening performance, preceding CEO Sundar Pichai’s keynote address.
The AI DJ Project’s music selection technology has proven its worth in more than just performances. It launched the AI BGM Project, in which AI selects the music for commercial spaces.
In the initial prototype, 150 background songs were prepared for The Core Kitchen/Space. The AI analyzed the songs and selected 3,500 similar songs. During the operation, the AI factored in real-time data on environmental variables such as weather, time of day and season. It also took into account the previous and following songs to create a playlist that felt natural and flowed from one song to another.
USEN provided unique image word tags such as active or relaxing. Qosmo’s AI technology was trained on approximately 500,000 songs that had been tagged with image words.
Song characteristics and USEN’s image words were combined with the predictions generated by the AI, resulting in nuanced metadata tags such as light and sentimental. In addition to image words, the AI can recognize characteristics that indicate the ambiance of a song and assign tags as morning/evening, summer/winter, and sunny/rainy.
For example, the AI recognizes song A’s characteristics as sentimental and mellow, placing it in the morning, winter, snow categories. The AI recognizes song B as a refreshing, summer, evening, and song C as one for a melancholic, rainy, spring. (See the below figure.)
Next, the AI creates the playlist in real-time as songs are chosen based on real-time data, such as the time of day, season, and weather. The AI selects songs for a day by moving steadily through a group of songs distributed in higher-dimensional space, taking into account the order of song selection.
In this way, the AI is able to create dynamic musical encounters. It generates a diversified, flowing playlist that is custom-made for listeners in a specific space, at a specific time.
Nao Tokui (Qosmo, Inc.)
Hiroshi Yamato (Qosmo, Inc.), Yumi Takahashi (Qosmo, Inc.), Wataru Abe (Tape ,Inc.)