Qosmo releases “Video2Music”, an AI which selects music that feels right for the video

Tokyo, Japan, Dec 22, 2022 – Qosmo, Inc., a company advancing the creativity of mankind with Art and Technology, releases Video2Music, a new lineup for the “Qosmo Music and Sound AI” product line as a result of years of R&D efforts. This product uses Qosmo’s proprietary deep-learning AI technology, which selects song candidates that match the content of the input video, allowing the user to significantly shorten the time required to select background music, which many video creators have spent a long time doing in the past. Please note that this product is intended as a B2B service.

■ Drastically reduces song selection

A wide range of knowledge about music is required to select background music that fits the atmosphere of the video and expresses the emotions of the scene well. In addition, there is a large difference in how individuals perceive the music, and it is often necessary to select music from among many candidate songs based on the opinions of different stakeholders. Video directors and creators have thus spent much time in the past on this problem, and our user interviews indicated that hours of time can be spent on the selection of music that only represents a few seconds in the presented work.

Video2Music instantly finds multiple candidate songs simply by uploading a video. The number of candidate songs can be selected, and each candidate song is ranked by the degree of match. While there is a certain degree of similarity among these candidate songs, there is also a certain degree of variation, allowing the producer to make the final selection based on taste and intentions.

The main features of this product are as follows

1. Qosmo’s proprietary AI model lists songs from the target music library (within a few seconds) that “matches” the given video. There are no restrictions on the genre or form of the target music.
2. In addition to “video→music” searches, “music→video” searches can also be performed, thus improving the user experience by implementing cross-media contents search.
3. Together with our music search algorithm, it is possible to build a wide range of search services, such as suggesting similar songs to selected candidates.
4. This product offering can be executed via API or on-premise environment. This product is not intended to be provided directly to end users, but rather on a B2B basis.

■ Video2Music can add value to a variety of products and services

Video2Music can be used in a variety of situations, but here are a few examples

1. Improved purchase rates for stock audio and video services

Stock music/video services conventionally use tags and keywords to aid search but it requires that users know what they are looking for. Video2Music solves this problem directly by allowing them to ask “find music that fits with this video.”

2. Utilization for music selection requests from clients

Song selection has depended on expert librarians at record companies and others who frequently receive requests for matching music. Video2Music allows those less familiar with the library to effectively narrow down candidates to a few selections.

■ How to watch demo

You can watch our product demo from the product website:

• Video2Music product website : https://qosmo.jp/en/products/video2music

■ Proprietary AI model developed by Qosmo

Video2Music uses the deep-learning algorithm called Transformer to convert video and music input into mutually comparable latent vector features. By training a model using a large number of movie contents online*, and using the Contrastive Learning technique, we successfully calculate quantitatively the fitness between a video and a song, two distinct forms of media. The pre-trained model provided with the product license already supports a wide range of input videos and music styles but can be re-trained with additional data to improve accuracy for specific applications.

*Training machine learning models from copyrighted materials is permitted by the copyright law of Japan

■ Prices and specifications

The Video2Music pricing consists of the following three components

• Initial indexing costs based on the size of the target music library
• Initial development support related to client’s system integration
• Monthly usage fee based on the number of API calls (flat rate up to a cap)

Please contact our representative at contact@qosmo.jp for more information.

■ Qosmo Music and Sound AIについて

Qosmo Music and Sound AI currently consists of the following products

• Music Generation AI 
• Music Selection AI 
• Timbre Transfer AI 
• Img2Sound 
• Video2Music