Imaginary Soundscape

Website to Experience Soundscapes "Imagined" by AI



We, human beings, can imagine sounds by taking a glance at a photo: The scenery of a beach may bring the sound of crashing waves to mind. You may hear sounds of horns and street advertising when you look at a picture of a busy crossing. “Imaginary Soundscape” is a web-based sound installation, focusing on this unconscious behavior, where viewers can freely walk around Google Street View and immerse themselves into imaginary soundscapes generated with deep learning models.

This work is based on the recent development of the cross-modal information retrieval technique, such as image-to-audio, text-to-image, using deep learning. Given video inputs, the system was trained with two models: one well-established, pre-trained image recognition model, processes the frames, while another convolutional neural network reads the audio as spectrogram images, evolving so that the distribution of its output gets as close as possible to that of the first one. Once trained, the two networks allow us to retrieve the best-matched sound file for a scene, out of our massive environmental sound dataset.

The soundscapes generated by the AI sometimes amaze us by meeting our expectation, but occasionally ignore the cultural and geographical context (the sound of waves on an icy field of Greenland for instance). These differences and mistakes lead us to contemplate how the imagination works and how fertile the sound environments surrounding us are. By externalizing our synesthetic thinkings, we tried to shed lights on the power of imagination we all share.

10.2018 featured on Google AI Experiments.
12.2018 featured on Favorite Website Award(FWA) Site of the day.

We have exhibited a sound installation “Imaginary Soundwalk” based on this system at Media Ambition Tokyo 2018.




  • Concept / Machine Learning

    Nao Tokui (Qosmo, Inc.)

  • UI Design / Programming

    Shoya Dozono (Qosmo, Inc.)

  • Machine Learning

    Yuma Kajihara (Qosmo, Inc.)

  • Web Development

    Robin Jungers (Qosmo, Inc.)

  • Logo Design

    Shoko Mitsuma


This project uses Qosmo Music & Sound AI.

Get in touch with us here!