We, as human beings, can unconsciously imagine the sounds of waves when we see a picture of a beach, or the sounds of car horns and street advertising when we see a picture of a crossing.
”Imaginary Soundscape” was launched in 2017 as a project to make AI imagine such soundscapes that people would unconsciously imagine. In this project, an AI that has learned the relationship between images and sounds from a large number of videos, continuously generates imaginary soundscapes by searching for the most suitable sound for the scenery from a vast number of preselected environmental sounds.
In this system, based on the image uploaded by the user, the AI selects the most appropriate sound from a sound library containing over 60,000 sound clips. In the Google Street View mode, the user can walk around a place of their choice and experience the soundscape “imagined” by the AI.
In 2022, we aimed to improve the user experience by incorporating the latest multimodal research into “Imaginary Soundscape”. The improved image-sound matching accuracy allowed more appropriate sounds to be selected, and improved the quality of soundscape imagined by AI. So far nearly 500,000 users worldwide have visited our website.
By externalizing our synesthetic thoughts into a system, we tried to shed lights on the power of imagination we all have and to inspire human imagination through the sounds imagined by AI.
Imaginary Soundscape demo
Imaginary Soundscape + Imaginary Soundwalk
Exhibitions
- 2018/02/09 – 2018/02/25 | Media Ambition Tokyo
Links
- Imaginary Soundscape Official Site - Imaginary Soundscape - How It Works - Qosmo, Inc. | Experiments with Google — AI Experimentsに「Imaginary Soundscape」が掲載されました - Qosmo, Inc. | 「Imaginary Soundscape」がFavorite Website Award (FWA) Site of the dayを受賞 - Experiments with Google — AI Experiments | Imaginary Soundscape - prosthetic knowledge | Imaginary Soundscape - engadget | AI adds background noise to Google Street View scenes - CO.DESIGN | Google Street View Now Has A Soundtrack, Thanks To AI
Articles
- Imaginary Soundscape — Take a walk in soundscapes “imagined” by AI
Credits
Project Direction: Akira Shibata (Qosmo, Inc.) Technical Direction: Nao Tokui (Qosmo, Inc.) Front-end: Robin Jungers (Qosmo, Inc.) Back-end: Bogdan Teleaga (Qosmo, Inc.) Machine Learning: Ryosuke Nakajima (Qosmo, Inc.) Web Design: Tomoyuki Yanagawa
Product
This project uses Qosmo Music & Sound AI. Image2Sound