MASOM: Musical Agent based on Self-Organizing Maps

Kıvanç Tatar, and Philippe Pasquier
Can we use Artificial Intelligence (AI) to create super-human musicians? Can we create an AI that listens to more music than a human could? Can we train AI on the compositions of dead composers and play with them on stage?

Musical agents are AI software making music. Musical Agent based on Self-Organizing Maps (MASOM) is a musical software agent for live performance. MASOM plays experimental music and free improvisation. It learns by listening to audio files such as recordings of performances or compositions. We can train a MASOM agent on a set of music that is so big that it would take more than one human life to listen. Similarly, we can train MASOM agents on dead composers and convert their fixed media piece to interactive musical agents.

MASOM also extracts higher level features such as eventfulness, pleasantness, and timbre to understand the musical form of what it hears. MASOM is limited to the style of what is has listened to and reacts in real-time to what he is hearing. The agent can listen to itself and other performers to decide what to play next. MASOM is designed by Kıvanç Tatar and Philippe Pasquier. Publications:

->Tatar K., Pasquier P., & Siu R. (2018). REVIVE: An audio-visual performance with musical and visual Artificial Intelligence Agents. CHI’18, April 21–26, 2018, Montreal, QC, Canada ACM 978-1-4503-5621-3/18/04. ->Tatar, K. & Pasquier, P. (2017). MASOM: A Musical Agent Architecture based on Self-Organizing Maps, Affective Computing, and Variable Markov Models. In Proceedings of the 5th International Workshop on Musical Metacreation (MuMe 2017). Paper

The research and development of MASOM is still ongoing. Following is the documentation of MASOM‘s previous versions, and public presentations. 

MASOM-Factor v1.03

MASOM joins two media art companies from Istanbul, Ouchhh and AudioFil for a performance at the Ars Electronica Festival 2017 Artificial Intelligence in Linz, Austria:

and for a projection mapping piece on the Facade of Romanian Parliament at the IMapp 2017 Bucharest, Romania:

MASOM version 0.06

This is the current version of MASOM. We released the complete system design in Proceedings of the 5th International Workshop on Musical Metacreation. Paper 

Patar @Barely Constrained by CoCreaTive

Patar is an experimental electronic music project featuring Kıvanç Tatar, Philippe Pasquier, and MASOM. MASOM is a musical agent, an artificial intelligence (AI) architecture, making music. Together, three entities produce a live performance of experimental electronic music, live electroacoustic music, musique concrète, soundscape compositions, through structured improvisation. Patar’s performances experiments with a wide range of electronic music styles within the performance. The performances of Patar oscillate between ambient and noise while exploring the frontiers of the rhythmic and tonal, and using both electronic and acoustic textures.
This performance includes three piece. Each piece includes a different MASOM agent. MASOMs trained on Bernard Parmegiani, David Tudor, and Ryoji Ikeda, respectively. The details of the event

Comparing MASOM's output to random segment playback:
Here, we provide two recordings of MASOM's output in solo performance mode so that the listeners can decide on MASOM's performance.
MASOM's output:
This track is a recording of the output of MASOM trained on Ryoji Ikeda's Test Pattern album
This following track is a recoding of random audio segments from the same album, Bernard Parmegiani's De Natura Sonorum. We let our readers to decide MASOM's performance by comparing the random segment playback to the MASOM's output.

This track is a recording of the output of MASOM trained on Bernard Parmegiani's De Natura Sonorum album.

In this video, we explore the memory of a MASOM trained on Bernhard Parmegiani's De Natura Sonorum.

MASOM version 0.05

Patar Studio Session IV This session had three performers: MASOM (AI), Kıvanç Tatar, and Philippe Pasquier. MASOM is trained on the recordings of Parmegiani and Ikeda. 
Kıvanç Tatar ©2018
 ︎ Bandcamp:->Tatar ->Çekiç
︎ Soundcloud