A Series of Audiovisual Experiments on the Web Platform
“Unsettled Music”, is a series of web experiments dedicated to showcasing some of our creative research on music, graphics, and machine learning. We have released a new iteration every few weeks. With this project, we’re hoping to collaborate with researchers, artists and designers from outside of the team, open a dialog, and give them a framework to try out their ideas. Like a gallery, each work can have its own space, and explore themes around music and AI with a certain freedom.
As the team behind Qosmo works on various projects, some focused on music making, others closer to data visualization, many small ideas and pieces of software are often left unused. Works shared as part of the series aren’t meant to explore their subject thoroughly — they are more like snapshots of an idea, with humble ambitions. This is because we focus on more the development of a method and process than the outcome itself. Exploring the outskirts of AI techniques often leads to unusual outputs, and that weirdness is a rich source of curiosity for us.
In the past few years, the web browser has become a platform of choice for media art : not only does it provide a number of features that make modern technologies easy to access, it also, by nature, makes even the earliest prototypes available for everyone to try online. The first technical component of this project is the visual rendering engine. Web browsers have been supporting hardware-accelerated graphics for many years now with WebGL, Three.js being the most famous library built on top of it, and the one we’re using for this project.
The second component is the audio framework — here again, on top of the well-stocked web audio API, libraries like Tone.js allow developers to design sound performances using natural musical approaches.Other libraries like Essentia also make use of WebAssembly to implement advanced analysis algorithms with great efficiency.
environment — and thus conveniently keep everything in one place.
As we work on future experiments, we will try to explore aspects of machine learning in different directions. Some of those topics may barely touch AI in fact, but are meant to question some marginal aspects of technology and software, and give them an audiovisual transcription. We are hoping to collaborate with researchers, artists and designers from outside of the team, open a dialog, and give them a framework to try out their ideas.
While some aspects of the experiments were optimized to avoid performance issues, the audio and visuals were arranged so that they can be dynamically explored in the browser. The interactivity happens through a single parameter, available on all experiments, which we called Unsettledness. In the context of AI, this translates to how far from the norm we choose to sample our new audio. Staying close to the familiar range leads to more expected output, but as we drift away from it, the model starts to produce unexpected results. Practically speaking, this unique Unsettledness slider allows you to scan a wide range of variations, from familiar and clean, to experimental and noisy.
Bogdan Teleaga, Christopher Mitcheltree
Andrew Fyfe, Bogdan Teleaga
Andrew Fyfe, Bogdan Teleaga, Christopher Mitcheltree