Projects

Selected list of projects I have participated in

Multiple Sclerosis (MS) (a progressive autoimmune chronic disease of the central nervous system) experiences few barriers to social interactions and professional careers. People with MS face challenges tracking their end-of-day (EOD) physical functioning outside of clinical settings (e.g., at their homes). Despite existing medications and behavioral interventions to improve people’s functioning, their functioning often goes unchecked in between infrequent clinical visits. This could lead to many negative consequences, including loss of employment, social disability, poor life satisfaction, and reduced physical health. 

Thus, behavior-based healthcare interventions (e.g., physical rehabilitations) can reduce relapses in patients with MS. However, lack of clinician oversight limits the benefit of such interventions when the patients are performing such behavior-based interventions at their homes without clinician guidance. In this regard, the ability to modeling behavior or forecast functioning of people with MS before it deteriorates could enable future user interfaces for clinically validated “just-in-time” interventions (e.g., “day planning” that allows people with MS to plan their activities for the day in advance. For example, such interfaces could notify people with MS (or their caregivers and clinicians) and suggest relevant clinical interventions before their functioning worsens. But lack of precise rules for determining how much data we need to accurately estimate the parameters of such behavior models remains one of the challenges while data collection places a significant burden on the participants (e.g., people with Multiple Sclerosis (MS)).

Heat-related deaths have been increasing recently worldwide. According to the Office for National Statistics, more than 10,000 people died due to heat-related illness since 2017. In Japan, ambulance due to heatstroke was record high for June 2022, also 56,303 deaths in England and Wales in 2022. The mortality rate due to overheating is estimated to increase by 260% by the 2050s. So, it is important to understand heatstroke symptoms in advance. I have been working on this project to develop an intelligent system to predict and prevent heatstroke from human physiological factors. In this project, I have been working to understand work stress, mental stress, and environmental stress under different activities, where it will be possible to model and understand human behaviors under different stressing factors. I worked with Fujitsu General Ltd for data collection. I have been building a reliable system that can effectively identify the factors responsible for heatstroke. We created a dataset with information from 130 individuals by collaborating with a Japanese company to understand physiological factors in different ages and genders. In this project, we also collected data from real-field workers while they were doing their jobs in the summertime.  Real-field data were collected when there were warnings about heatstroke in Tokyo.

This is the project collaborating with Rice University, Kyushu Institute of Technology, and NTT Data Institute of Management Consulting, Inc., Tokyo, Japan. Under this project, we have collected 100 office workers’ real-field multimodal data by collaborating with a company in Japan, where the dataset contains objective physiological and behavioral sensor data, and work activities and work environment information with different psychological state information. The dataset depicts the status of real office workers states performing their everyday work under real-life stressors. This project's work focuses on understanding office workers stressors, including physical and mental exhaustion, due to different behavioral, work environmental, workplaces, and work types issues, which is also a social problem. Ongoing work focuses on revealing work and behavioral patterns and routines between Office work and Telework using a real-world multimodal dataset including physical activity sensors data, work activity, work environment, work engagement, and psychological status data. We are working to build models for revealing daily work and behavioral patterns while people are working at home and office. We are building interpretable and personalized models to predict the different mental states of office workers and analyze their day-to-day behavior and psychological state to reveal the factors in the workplace that affect their psychological states.

Activity recognition models show high performance when trained and tested with data collected in controlled environments like experiments. However, when used in real life, the performance of these models tends to drop significantly. We study three causes for this: usage of different devices, therefore data missing, sensor orientation inconsistency: within and among subjects, and new user scenario.

 We introduce a system of integrating activity recognition and collecting nursing care records at nursing care facilities as well as activity labels and sensors through smartphones, and describe experiments at a nursing care facility for 4 months. A system designed to be used even by staff not familiar with smartphones could collected enough number of data without losing but improving their workload for recording.  By using activity recognition based on sensor data and historical data of nurses activities, we aim to reducing documentation workload by partially filling the nurse’s records for the day. We explored activity recognition to optimize the work of nurses at nursing homes by reducing the time they spent on documentation tasks at real nursing care facilities. Moreover, we demonstrated the near future prediction to predict the next day’s activities from the previous day’s records, which could be useful for proactive care management.

Activity recognition with low strength sensor is daunting task due to the lack of proper data collection in real life. We investigated the performance of the LoRaWAN sensor in both laboratory environment and real nursing care environment. LoRaWAN sensor has some special criteria that is LoRaWAN gateway is able to support up to 20,000 IoT devices. It can operate successfully at ranges exceeding 15 km in sub- urban settings and more than 2 km in dense urban environments. We explore the sensing capability of LoRa, both theoretically and experimentally in a real nursing care center in Japan. 

★ Explore eSense Earable for Detecting Head and Mouth Related Behavioral Activities 

Intaking activities are also very important for nurse care of elderly people, because they need to record and manage the amount of intake, and swallowing ability which is one of the measures of health for elderly people. We try to recognize head and mouth-related behavioral activities (Eating / Speaking/ nodding, etc).