Emergent Rhythm

Live performance using real-time AI audio generation and AI image generation

2022

#AI Visual
#Audio Generation
#Music Generation
  • DSC03017-1-1536x864-1
  • aidj3 3-1-2048x1152
  • aidj3 2-2048x1152
  • aidj3 10-1-2048x1152
  • aidj3 12-1-2048x1152
  • aidj3 8-1-2048x1152
  • 20221208-GZ9 6955 s-1-1536x1024-1
  • 20221208-GZ9 6810-1-1536x1024-1

“Emergent Rhythm” is an audio-visual DJ performance using real-time AI audio generation and AI image generation.

This work is the third installment of the AI DJ Project, a series of attempts to explore the future of AI-based music production and live DJ performance. The project began as a “back to back” DJ session with the artist and an AI DJ system, continued with improvised sessions with realtime MIDI AI music generation and now evolved into this work that uses both sound and image generation in realtime.

As the title Emergent Rhythm suggests, in this performance we focus on the musical and visual “rhythms” and recurring patterns that emerge in the interaction between multiple AI models and the artist. The accompanying visuals feature not only the periodicity over time from past to future but also the common patterns across multiple scales ranging from the extreme large-scale of the universe to the extreme small-scale of atomic structures. More than one million images were generated using the Stable Diffusion model for the visuals. The DJ first manipulates multiple models on stage to spontaneously generate rhythms and melodies and then combines and mixes the generated audio loops to create a musical story.

From a DJ session, in which existing songs are selected and mixed, to a live performance that generates songs spontaneously and develops them in response to the audience’s reactions. This project continues to evolve, redefining the act of DJing itself.

Performances

2022.12.8 | MUTEK.JP (Shibuya Stream Hall, Tokyo) 2023.12.9 | Craft Alive (Daikanyama UNIT, Tokyo) 2023.6.14 | Soanr+D/+RAIN Film Festival 23 (Barcelona, Spain) 2023.10.21 | Digital Art Zurich 23 (Kunsthaus, Zurich, Switzerland)

Articles

- Medium – Making of “Emergent Rhythm” — Behind the scenes of a live performance using real-time AI audio generation - Medium – Audio-visual Expression Using Image Generation AI — Development Case Study at MUTEK.JP

Credits

Concept, Machine Learning & Performance: Nao Tokui (Qosmo) Visual Programming: Ryosuke Nakajima (Qosmo) Visual Programming: Keito Takaishi (Qosmo) Dancer: Masumi Endo Movement Director: Maiko Kuno