Lecture Concert 

JKU, Austria

The Accompanion: an AI-based Piano Accompaniment System

'The Accompanion' is an AI-based piano accompaniment system playing live with a human performer on a single computer controlled grand piano.

The program will consist of three parts with a duration of roughly 30 minutes.

1. A short (< 5min) presentation introducing the system.

2. A demo played by myself (~ 10 minutes)

• E. Grieg: Selections from Peer Gynt Suite 1 Op. 46 (Piano 4 hands version)

• J. Brahms: Hungarian Dance No. 5 (Piano 4 hands version)

3. An open demo (~ 15 minutes) where I will invite to the public to try the system, where they could play some of classical pieces (from J. S. Bach, Mozart, Beethoven, J. Strauss,...)

KAIST AI Performance Team

Human-AI Piano Relay Performance (Pf. Jonghwa Park)

• J. Haydn: Piano Sonata Hob. XVI/34 I. Presto

'Human-AI Piano Relay Performance' system is an interactive music performance system which enables novel musical performances by creating a duet between a human pianist and an AI counterpart, which is based on the score following algorithm and expressive piano performance generation model (VirtuosoNet). A demo of the system will be presented by Jonghwa Park, a Korean virtuoso pianist. More information about the system can be found here.

Flute Performance Accompanied by AI Pianist with Automatic Cue Detection (Fl. Jaeran Choi)

• Ennio Moricone: Cinema Paradiso

‘Automatic Musical Cue Detection’ system is a multi-modal based interactive performance system using motion analysis. In ensemble, the beginning of a piece or a fermata section has weak audio information to refer to, so it mainly relies on "cues" using the performer's gestures and breathing. However, in ensembles with an AI pianist, the absence of a human pianist makes it impossible to apply essential visual interactions such as cues. To solve this problem, this system detects the flutist's motion cues via webcam at the beginning and the fermata section and calculates the optimal timing and the automatic piano starts the accompaniment. Jaeran Choi, a master student at Music and Audio Computing Lab, will perform ‘Ennio Moricone: Cinema Paradiso’ with the system. 

Classical Voice with AI Pianist with Adaptive Accompaniment System (Sop. Yoonji Oh)

• F. Schubert: Heidenröslein, D.257

For an ensemble of a classical singer and an AI pianist with automated operation, this performance incorporates multiple systems including an adaptive piano accompaniment system and real-time lyrics tracking. Yoonji Oh, a Korean soprano, will present 'Heidenröslein' by Schubert with the system.

Real-time Piano Transcription with Visualization (Pf. Jonghwa Park)

Audio-to-score transcription, powered by the autoregressive neural network model will be performed and visualized in realtime. 

Taegyun Kwon is Ph.D. Candidate in the Music and Audio Computing Lab(MACLab) at GSCT KAIST. He researched on piano performance analysis, including realtime piano transcription, alignment and expressive generation. In this work, he developed a real-time piano transcription system, performed by virtuoso pianist Jonghwa Park, and the adaptive piano accompaniment system for singing performance, collaborated with soprano Yoonji Oh.
  Jiyun Park is a Ph.D. student in the MACLab at GSCT KAIST. Her research focuses on interactive music performance including real-time music alignment and singing voice. Her work includes developing a Human-AI piano relay performance system, performed by virtuoso pianist Jonghwa Park, and a real-time lyrics tracking system, collaborated with soprano Yoonji Oh.
  Jaeran Choi is a Master Student at GSCT KAIST. Her research interests include Human-AI musical interaction. In particular, she focuses on multimodal musical cue detection and reactive accompaniment systems. She developed flute-AI Pianist ensemble system using automatic cue detection and will present it with her own flute performance.
  Joonhyung Bae, a Korean artist and Ph.D. Candidate at GSCT KAIST. He is currently a Music and Audio Computing Lab member. He is researching sound-based virtual performer visualization for artistic expression using deep learning.~~~
  Hyeyoon Cho is a second year master student at GSCT KAIST. She received a Bachelor's degree in piano performance at University of Texas, Austin and  Master’s in piano performance at Indiana University. Her research interests include quantization in piano performance and music information retrieval.
  Dasaem Jeong, Assistant Professor in the Department of Art & Technology at Sogang University, South Korea. He obtained his Ph.D. in culture technology from KAIST under supervision of Juhan Nam. His research focuses on various music information retrieval tasks, including expressive performance modeling and symbolic music generation. He developed  VirtuosoNet, a system for generating expressive piano performance.
  Yonghyun Kim is currently pursuing a master's degree at GSCT KAIST. His research areas of interest are Music, Artificial Intelligence, and HCI. He is currently focusing on research that combines multimedia (esp. audio and vision) and AI to enrich human musical experience and creation.
  Juhan Nam is an Associate Professor in the Graduate School of Culture Technology at the Korea Advanced Institute of Science and Technology (KAIST) in South Korea. He is the director of the Music and Audio Computing Lab. He is interested in various topics at the interaction of music, audio signal processing, machine learning, and human-computer interaction.

Special Guests