Healthcare

Our lab conducts research on multimodal healthcare AI for recognizing rehabilitation actions and quantitatively assessing movement quality in stroke patients. In particular, MMeViT uses data from IMU sensors and RGB-D cameras to recognize upper-limb activities of daily living in home environments, enabling more reliable post-stroke action monitoring.

We also developed RAST-G@, a deep learning model that combines ST-GCN and temporal attention to evaluate rehabilitation movement quality from skeleton sequences and provide structured feedback to both patients and therapists.

These studies are meaningful because they support more objective, consistent, and scalable rehabilitation assessment compared to conventional subjective evaluation methods. In multimodal healthcare, AI is used to jointly analyze different types of data—such as video, inertial sensor signals, and biosignals—to better understand a patient’s condition and behavior. This makes it possible not only to recognize actions, but also to assess recovery progress, detect abnormal movement patterns, and deliver personalized rehabilitation feedback.

Overall, our lab aims to improve the accessibility, objectivity, and continuity of rehabilitation through AI-driven digital healthcare systems for home-based recovery.

This study proposes a home-based rehabilitation assessment and feedback system for stroke patients that combines wearable sensors, an RGB-D camera, a mobile application, and an AI server. At its core, the system uses RAST-G@, a deep learning model that combines ST-GCN and transformer-based temporal attention to evaluate rehabilitation movement quality from skeleton sequences. The authors also built the NRC dataset, which includes upper-limb ADL and ROM exercises scored by licensed physiotherapists.

Results on the KIMORE and NRC datasets show that the system improves assessment accuracy over baselines and provides scalable, quantitative feedback for domiciliary rehabilitation.

This study proposes a multimodal action recognition system for monitoring upper-limb daily living activities of stroke patients in home environments. The system uses IMU sensors and an RGB-D camera, and introduces a deep learning model with suitable preprocessing for handling multimodal rehabilitation data. The authors show that stroke action data is less clearly clustered than that of non-disabled individuals, yet the proposed model can still learn meaningful action patterns.

Overall, the work suggests that multimodal deep learning can support not only action recognition, but also future rehabilitation feedback and assessment for home-based stroke recovery.

Google Sites

Report abuse