Ongoing Research Projects

Pianist Multimodal Data Acquisition and Generation

Supported by Yamaha (2023-2026)

The primary objective of this research is to enhance our understanding of piano performance through the lens of multimodal data analysis. The central problem it addresses is the gap in comprehensive data that integrates both the auditory and physical aspects of piano playing. By focusing on the multimodal aspects of piano performance, this study aims to bridge this gap, providing insights into the nuanced interplay between musicians and their instruments. The expected outcomes include the development of advanced AI algorithms for pianist motion generation and a deeper understanding of the intricate dynamics of musical performance. This research is anticipated to pave the way for significant advancements in AI-driven music synthesis, performance analysis, and the creation of more intuitive and human-like interfaces for musical interaction.

Pianist Motion Data Acquisition

The primary objective of this research is to enhance our understanding of piano performance through the lens of multimodal data analysis. The central problem it addresses is the gap in comprehensive data that integrates both the auditory and physical aspects of piano playing. By focusing on the multimodal aspects of piano performance, this study aims to bridge this gap, providing insights into the nuanced interplay between musicians and their instruments. The expected outcomes include the development of advanced AI algorithms for pianist motion generation and a deeper understanding of the intricate dynamics of musical performance. This research is anticipated to pave the way for significant advancements in AI-driven music synthesis, performance analysis, and the creation of more intuitive and human-like interfaces for musical interaction.

Related Publications

• Automated Gap-filling Algorithm for Kinetic Data of Finger Movement: Pianist Hand Motion Cleaning Using Spatio-temporal Imputation
  Taegyun Kwon, Dawon Park, Joonhyung Bae, Jonghwa Park, Jaebum Park, and Juhan Nam
  Proceedings of the 29th Congress of International Society of Biomechanics, 2023 [paper]

• Multivariate Kinematic Analysis of Piano Performance: a Case Study of a Professional Pianist
  Dawon Park, Taegyun Kwon, Joonhyung Bae, Jonghwa Park, Juhan Nam, and Jaebum Park
  Proceedings of the 29th Congress of International Society of Biomechanics, 2023 [paper]