TR(AI)O @ Piece & Lecture, SIGMUS

My real-time AI interactive improvisation music performance “TR(AI)O – A Generative Improvisation of AI and Human” is selected to be part of the “Piece and Lecture Session” of the 147th SIGMUS (Special Interest Group in MUSic and computer) Summer Symposium. Under IPSJ (Information Processing Society of Japan, similar to IEEE and ACM, which is the largest computer science and information processing research organization in Japan), SIGMUS focuses specifically on the collaborations and intersections between music and information processing as the biggest research community in Japan. As titled, in the “Piece and Lecture” section, along with another 7 Japanese research groups around the University in Japan, we will not only present our ideas but also be required to showcase the work as performances.

So, what is TR(AI)O? It is a human-AI interactive music performance where live music is co-created through a real-time dialogue between human users and AI characters. Guided by the manifesto, “The dialogue is the music,” it explores an alternative algorithmic aesthetic by mapping the LLM’s generative syntax and token IDs directly into musical notation and tempos. Exaggeratedly speaking, we can even argue that it is the “voice/Language” of AI. Embracing machine-driven friction to trigger a theatrical leap of imagination, this scalable framework allows a human player to improvise alongside AI entities, re-envisioning real-time collaboration on stage. Due to the numerous variations of giving different personalities to AIs, mapping mechanisms to different instruments, notes, and reasons, it opens up infinite possibilities in music creation. The dialogue between the human and AI (or 2 AIs themselves) will be displayed in the background to not only create a musical conversation but also an immersive dramatic environment. It can be set up as an intimate dialogue between one human and one AI, but also capable of being expanded to be a symphony version with multiple humans and AI agents involved.

This is TR(AI)O—a live performance by AIs and me.

The 147th SIGMUS Summer Symposium: https://www.ipsj.or.jp/kenkyukai/event/mus147.html

我的人工智慧即時音樂即興互動藝術表演「TR(AI)O – A Generative Improvisation of AI and Human」，將參與於北海道情報大學舉辦第147屆SIGMUS (Special Interest Group in MUSic and computer) 音樂情報科學研討會下的Piece and Lecture Session (作品+演講)活動。隸屬於日本資訊科技與電腦科學領域最大、最具權威的學術組織IPSJ（Information Processing Society of Japan）之下，音樂情報科學研討會是日本國內探討電腦與音樂結合、音樂資訊處理領域規模最大的學術組織。Piece and Lecture Session顧名思義，我與其他七組來自日本各大學及研究機構的音樂電腦研究員兼藝術家，除了就作品發展發表介紹之外，更多了現場展演的部分。這將會是TR(AI)O的公開首演，更是我跨足夢想音樂圈的第一步（開玩笑）。

TR(AI)O就像是現場爵士樂即興演奏般，讓人類跟AI共同創作音樂。有別於當今生成式AI（Generative AI）透過指令（prompt）等待音樂（圖像亦是）被AI生成的主流，TR(AI)O提供的是一種另類使用大型語言模型（LLM）來達到與AI即興即時生成音樂。 AI除了將英文字母對應到音符之外（旋律），透過詞元化（Tokenized）產生律動（節奏），某種程度也可以視為AI的（有聲調的）語言化。所有的對話都是當場生出的，甚至可針對AI進行角色化的設計讓每一場表演基本上都是獨一無二的，相對應的樂器跟音符亦可客製化，構成其無限的可能性。舞台上除了聲響之外，也將與AI（或AI彼此）的對話以投影呈現，增強聲歷其境感。這場看似是表演者一個人的獨奏（solo），其實卻是跟其他兩個AI對話三重奏（trio）。對於我來說，這個架構具有無限可能，可簡化成單人與單AI的深層對話，也可以延伸成多人與多AI的交響曲。

第147回MUS研究発表会: https://www.ipsj.or.jp/kenkyukai/event/mus147.html