2024 UBICOMP TUTORIAL ON SOLVING THE ACTIVITY RECOGNITION PROBLEM (SOAR)
2024 UBICOMP TUTORIAL ON SOLVING THE ACTIVITY RECOGNITION PROBLEM (SOAR)
Self-supervised and multi-modal recognition of activities from wearable sensors
Feature extraction remains the core challenge in Human Activity Recognition (HAR) - the automated inference of activities being performed from sensor data. Over the past few years, the community has witnessed a shift from manual feature engineering using statistical metrics and distribution-based representations, to feature learning via neural networks. Particularly, self-supervised learning methods that leverage large-scale unlabeled data to train powerful feature extractors have gained significant traction, and various works have demonstrated its ability to train powerful feature extractors from large-scale unlabeled data. Recently, the advent of Large Language Models (LLMs) and multi-modal foundation models has unveiled a promising direction by leveraging well-understood data modalities. This tutorial focuses on existing representation learning works, from single-sensor approaches to cross-device and cross-modality pipelines. Furthermore, we will provide an overview of recent developments in multi-modal foundation models, which originated from language and vision learning, but have recently started incorporating inertial measurement units (IMU) and time-series data. This tutorial will offer an important forum for researchers in the mobile sensing community to discuss future research directions in representation learning for HAR, and in particular, to identify potential avenues to incorporate the latest advancements in multi-modal foundation models, aiming to finally solve the long-standing activity recognition problem.
*This is the tentative schedule and might change due to addition/removal of topics.
Harish Haresamudram
Georgia Institute of Technology
Chi Ian Tang
Nokia Bell Labs
Sungho Suh
DFKI and RPTU
Paul Lukowicz
DFKI and RPTU
Thomas Ploetz
Georgia Institute of Technology
Participants can register via the Ubicomp registration link: here
Contact hharesamudram3 at gatech.edu / thomas.ploetz @ gatech.edu to get more information about the tutorial