Project
2023-2024 Projects
[NIH U01] Monitoring Acute and Longer-term Effects of Cannabis on Psychomotor Performance in Daily Life in Medical Cannabis PatientsPIs: Sang Won (Grace) Bae and Tammy ChungMethods to monitor acute cannabis effects in real-time could reduce potential cannabis-related harm by raising a person's awareness of possible cannabis-related impairment, for example, on psychomotor performance. Toward this goal, this project will collaborate with individuals who use cannabis for therapeutic purposes in monitoring, using their smartphone, the acute therapeutic and adverse effects of cannabis use, and will combine this fine-grained data with follow-up over 1-year because cannabis effects may differ at shorter and longer time scales. In line with NIDA priorities, this project's novel combination of smartphone symptom monitoring and longer-term follow-up will guide the creation of real-time interventions to reduce cannabis-related impairment and harm.
[NIH R21] Smartphone Sensors to Detect Shifts toward Healthy Behavior during Alcohol TreatmentPIs: Sang Won (Grace) Bae and Tammy ChungThis secondary data analysis project will use the unique combination of already collected phone sensor data in the context of an alcohol clinical trial to gain new insight into when and how young adults' routines change in response to alcohol intervention. Phone sensor data provide objective markers of daily routines, which can reveal with low burden, shifts toward alternative behaviors associated with positive response to alcohol treatment. Results will inform personalized interventions based on phone sensor data to optimize digital intervention effects.
Brain State Inference and Brain Disorder Biomarkers Study Using Multimodal Neural Imaging EEG PlatformPIs: Sang Won (Grace) Bae and Feng LiuThe prevalence of substance overdose has significantly increased since the onset of the COVID-19 pandemic. According to the National Survey on Drug Use and Health (NSDUH), 19.7 million American adults (aged 12 and older) grappled with Substance Use Disorder (SUD), and 1 in 8 adults faced both alcohol and drug use disorders concurrently. While past research indicates that substance addiction is a brain disorder with abnormal neuroimaging patterns, the conventional measurement of addiction relies heavily on questionnaires, introducing subjectivity and potential errors. This project aims to establish an evidence-based and objective framework by delving into subjects' EEG multimodal neuroimaging fingerprints and gait information. By leveraging a systematic multimodal neuroimaging approach encompassing resting state EEG signal, evoked response potential EEG signal, and Electrooculography (EOG), we aim to provide an objective and comprehensible means of gauging SUD severity and incidence while enhancing our insights into relapse behavior. Our approach incorporates EEG, EOG, and mobile sensing data for early prediction and estimating the severity stages of SUD.
[NIH R21] Smartphone Sensors to Detect Shifts toward Healthy Behavior during Alcohol TreatmentPIs: Sang Won (Grace) Bae and Tammy ChungThis secondary data analysis project will use the unique combination of already collected phone sensor data in the context of an alcohol clinical trial to gain new insight into when and how young adults' routines change in response to alcohol intervention. Phone sensor data provide objective markers of daily routines, which can reveal with low burden, shifts toward alternative behaviors associated with positive response to alcohol treatment. Results will inform personalized interventions based on phone sensor data to optimize digital intervention effects.
Brain State Inference and Brain Disorder Biomarkers Study Using Multimodal Neural Imaging EEG PlatformPIs: Sang Won (Grace) Bae and Feng LiuThe prevalence of substance overdose has significantly increased since the onset of the COVID-19 pandemic. According to the National Survey on Drug Use and Health (NSDUH), 19.7 million American adults (aged 12 and older) grappled with Substance Use Disorder (SUD), and 1 in 8 adults faced both alcohol and drug use disorders concurrently. While past research indicates that substance addiction is a brain disorder with abnormal neuroimaging patterns, the conventional measurement of addiction relies heavily on questionnaires, introducing subjectivity and potential errors. This project aims to establish an evidence-based and objective framework by delving into subjects' EEG multimodal neuroimaging fingerprints and gait information. By leveraging a systematic multimodal neuroimaging approach encompassing resting state EEG signal, evoked response potential EEG signal, and Electrooculography (EOG), we aim to provide an objective and comprehensible means of gauging SUD severity and incidence while enhancing our insights into relapse behavior. Our approach incorporates EEG, EOG, and mobile sensing data for early prediction and estimating the severity stages of SUD.
Designing an Empathetic Chatbot for Enhancing Student Academic Success, Balancing Campus Life, and Well-Being (Part II)
PIs: Sang Won (Grace) Bae and Ting LiaoThis research initiative seeks to validate the hypothesis that interactions with an empathetic chatbot enhance participants' trust, engagement, and satisfaction in cases where the chatbot facilitates the initial screening phase for student advising. The designed chatbot does actually conduct text-based conversations with students, gathering their questions and concerns before their scheduled meetings with academic advisors (please keep in mind that this project does not include in-person advising sessions)."Leveraging AI in Sports with Explainable E-Coaching"
We are excited to invite you to participate in the study on the use of smart devices in improving performance and injury prevention in sports leveraging athlete and coach communication, trust, and empathy. You will be asked to share your experiences of using smart technologies to monitor performance, and evaluate the explainable AI (XAI) e-coaching dashboard we developed. Your participation will help us develop a real-time XAI e-coaching system that leverages passive sensing and real-world data to improve performance in sports.ELIGIBILITY:
- Being a college student athlete.
- For more information on this opportunity, click here.
MULTIMODAL SENSING
PIs: Sang Won (Grace) Bae and Feng Liu
PREDICTION OF MARIJUANA INTOXICATION
Collaborators funded by NIH: Tammy Chung (Rutgers, PI), Brian Suffoletto (Stanford), and Anind Dey (University of Washington)
PREDICTING STUDENT ENGAGEMENT
PIs: Sang Won (Grace) Bae and Ye Yang (Amazon)
"Mobile Sensing and Behavior Modeling for Mental Health"
We invite you to participate with researchers at the CARE AI Lab conducting investigations on the relationship between facial behaviors and cognitive function. All activities will be conducted via smartphone in everyday setting, so you will not need to be in the lab to complete them. Together, we can develop detection models that leverage passive sensing and real-world data to improve mental health.Participants can earn up to $135 in compensation.ELIGIBILITY:
- Must possess an Android smartphone with Android 9 or above.
2022 Fall Projects
Brain State Inference and Brain Disorder Biomarkers Study Using Multimodal Neural Imaging EEG PlatformPIs: Sang Won (Grace) Bae and Feng LiuSubstance overdose has risen significantly since the outbreak of the COVID-19 pandemic. According to the National Survey on Drug Use and Health (NSDUH), 19.7 million American adults (aged 12 and older) battled with Substance Use Disorder (SUD), and 1 out of every 8 adults struggled with both alcohol and drug use disorders simultaneously. Previous research shows substance addiction is a brain disorder that can demonstrate abnormal neuroimaging patterns, however, the traditional measurement of addiction status is mainly based on questionnaires, which can be subjective and prone to error. This project aims to develop an objective evidence based framework by exploring subject’s EEG multimodal neuroimaging fingerprints and gait information. To provide an objective and interpretable way to assess the severity and incidence of SUD, as well as improve our understanding of relapse behavior, we apply a systematic multimodal neuroimaging approach utilizing a multiple modal signal, which includes resting state EEG signal, evoked response potential EEG signal and Electrooculography (EOG). We propose to use EEG, EOG, and mobile sensing data for early prediction and severity stage estimation for SUD.
Smart & Connected Spaces: A Testbed for IoT Data Innovation in SSEPIs: Sang Won (Grace) Bae, Philip Odonkor, Steven Hoffenson, Changyue Song and Eman AlOmarFrom wireless home sensors to wearable devices, the Internet of Things (IoT) promises new opportunities for tailored experiences and targeted interventions with respect to sustainability and livability. Realizing this potential, however, requires fundamental understanding of what types of data empower this capability and how—in terms of volume, velocity, and veracity. Thus, there is a critical need for IoT tools and resources to allow for data collection, processing, inference, adaptation, and learning experiences within SSE. The Smart & Connected Spaces project aims to develop an IoT testbed to support sensor-driven research in SSE. Specifically, it will create digitally expressive smart spaces within SSE, affording faculty the ability to explore the reciprocal relationships between people, space and technology. With extensive backgrounds in multi-sensor modeling, engineering design, human-computer interactions, and big data analytics, the PIs on this project are uniquely positioned to address the cyber-physical challenges of implementing and utilizing the proposed testbed.
Smart & Connected Spaces: A Testbed for IoT Data Innovation in SSEPIs: Sang Won (Grace) Bae, Philip Odonkor, Steven Hoffenson, Changyue Song and Eman AlOmarFrom wireless home sensors to wearable devices, the Internet of Things (IoT) promises new opportunities for tailored experiences and targeted interventions with respect to sustainability and livability. Realizing this potential, however, requires fundamental understanding of what types of data empower this capability and how—in terms of volume, velocity, and veracity. Thus, there is a critical need for IoT tools and resources to allow for data collection, processing, inference, adaptation, and learning experiences within SSE. The Smart & Connected Spaces project aims to develop an IoT testbed to support sensor-driven research in SSE. Specifically, it will create digitally expressive smart spaces within SSE, affording faculty the ability to explore the reciprocal relationships between people, space and technology. With extensive backgrounds in multi-sensor modeling, engineering design, human-computer interactions, and big data analytics, the PIs on this project are uniquely positioned to address the cyber-physical challenges of implementing and utilizing the proposed testbed.
2022 Spring/Summer Projects
Developing an Explainable E-Coaching Dashboard to Enhance Quality of Communication, Trust, & Empathy Between Athletes and CoachesPIs: Sang Won (Grace) Bae and AnaMaria LaccettiThe objective of this research study is to develop an explainable e-coaching dashboard system to enhance the use of smart devices (i.e. mobile applications, wearables) to improve performance and injury prevention in sports. The goal of the e-coaching dashboard system is to establish user trust in sport performance algorithms, enhance the quality of communication between athletes and coaches, and give coaches the ability to empathize with athletes through shared decision making and explainable AI methods.
Mobile Sensing and Behavior Modeling Framework for Mental HealthPIs: Sang Won (Grace) Bae and Rahul IslamThe goal of this study is to investigate daily behaviors and patterns to understand physical and mental health using smartphones. Mobile sensing has been used to predict mental states (e.g., depression) in the field of human-computer interaction. However, there is a lack of understanding in terms of functional impairments. In this study, we expect to find relationships between emotional and cognitive changes in mental health with the help of passive sensing in real-world settings.
A Deployable Privacy-Preserving Sensing Framework to Empower Students & Teachers PI: Sang Won (Grace) Bae As online learning becomes more prominent, it is increasingly difficult for instructors to gauge the learning states of their students in real time, thus missing the opportunity to evaluate their teaching effects. While existing approaches suggested systems that require high computing power in detecting student engagement, there is a gap of broad adoption of systems that meet the needs of both students and teachers. Therefore, in this project, we propose an easily-deployable sensing framework, which unobtrusively senses students’ behavioral signals to see if they are fully immersed in online learning. In this proof of concept study, we collected behavior markers from students to validate the feasibility of the app in online learning. The implications of these results allow us to move on to the next stage of expanding the framework’s scalability with larger data sets by validating its performance throughout a variety of contexts and tasks.
Educational Implications of Human-Centered Explainable AI (HCXAI) for Non-CS Students & HCXAI in Learning at ScalePIs: Sang Won (Grace) Bae and Gi Woong ChoiWhile many studies have focused on implementing AI for adaptive learning or learning analytics, the HCI community requires education to reach the broader audience in non-programming ways. We highlight how researchers and educators in the HCI community can integrate AI into human lives, leveraging the concept of XAI into HCI education in a less technical way for students. Such integration will help students become familiar with AI, improving an essential competency (AI literacy) needed for the era of AI.
Mobile Sensing and Behavior Modeling Framework for Mental HealthPIs: Sang Won (Grace) Bae and Rahul IslamThe goal of this study is to investigate daily behaviors and patterns to understand physical and mental health using smartphones. Mobile sensing has been used to predict mental states (e.g., depression) in the field of human-computer interaction. However, there is a lack of understanding in terms of functional impairments. In this study, we expect to find relationships between emotional and cognitive changes in mental health with the help of passive sensing in real-world settings.
A Deployable Privacy-Preserving Sensing Framework to Empower Students & Teachers PI: Sang Won (Grace) Bae As online learning becomes more prominent, it is increasingly difficult for instructors to gauge the learning states of their students in real time, thus missing the opportunity to evaluate their teaching effects. While existing approaches suggested systems that require high computing power in detecting student engagement, there is a gap of broad adoption of systems that meet the needs of both students and teachers. Therefore, in this project, we propose an easily-deployable sensing framework, which unobtrusively senses students’ behavioral signals to see if they are fully immersed in online learning. In this proof of concept study, we collected behavior markers from students to validate the feasibility of the app in online learning. The implications of these results allow us to move on to the next stage of expanding the framework’s scalability with larger data sets by validating its performance throughout a variety of contexts and tasks.
Designing an Empathetic Chatbot for Enhancing Student Academic Success, Balancing Campus Life, and Well-Being (Part I)
PIs: Sang Won (Grace) Bae and Ting LiaoThis research initiative seeks to validate the hypothesis that interactions with an empathetic chatbot enhance participants' trust, engagement, and satisfaction in cases where the chatbot facilitates the initial screening phase for student advising. The designed chatbot does actually conduct text-based conversations with students, gathering their questions and concerns before their scheduled meetings with academic advisors (please keep in mind that this project does not include in-person advising sessions).Educational Implications of Human-Centered Explainable AI (HCXAI) for Non-CS Students & HCXAI in Learning at ScalePIs: Sang Won (Grace) Bae and Gi Woong ChoiWhile many studies have focused on implementing AI for adaptive learning or learning analytics, the HCI community requires education to reach the broader audience in non-programming ways. We highlight how researchers and educators in the HCI community can integrate AI into human lives, leveraging the concept of XAI into HCI education in a less technical way for students. Such integration will help students become familiar with AI, improving an essential competency (AI literacy) needed for the era of AI.
2021 Summer Projects
Understand User Preference Using Multimodality to Deliver Personalized RecommendationsPIs: Sang Won (Grace) Bae, Feng Liu, and Ye YangDelivering personalized recommendations can improve the effectiveness of user satisfaction. To do this, understanding user preference is critical to developing such recommender systems, however, existing studies mainly utilize high-cost devices and high computation in detecting preference. In this work, we propose a multimodal framework in which behavioral expressions and neural signals are captured by low-cost portable electroencephalography (EEG) devices in identifying a user’s preference. We found that EEG combined with behavior features improves the preference detection, specifically whether a user likes or dislikes the given images in controlled experiments. Further, we introduce a richer set of objective markers leveraging EEG-based neural features and behavior markers that contribute to preference detection. We demonstrate the multimodal-based preference detection using the commercialized portable EEG which can provide an efficient way to approach a user’s preference detection in designing personalized recommendation systems in real-world settings.
PIs: Sang Won (Grace) Bae and Ye YangAs mental health is critical for human well-being, recent years have witnessed sharp increase in the amount as well as variety of mHealth apps to help user self-care in their daily lives. Existing studies start to explore the adoption, effectiveness, and impact of mHealth apps, however, most such studies rely heavily on data collected from controlled experiments, and there is lack of quantitative studies reflecting user experience and perceptions from the end-user’s perspective. To fill the gap, we analyze user review comments from 20 mHealth apps including AI-chatbot, particularly for mental health, extracted from Google Play Store. We highlight challenges and opportunities to design Human-Centered AI systems to better support user experiences and perceptions for at-risk population.
PIs: Sang Won (Grace) Bae and Ye YangAs mental health is critical for human well-being, recent years have witnessed sharp increase in the amount as well as variety of mHealth apps to help user self-care in their daily lives. Existing studies start to explore the adoption, effectiveness, and impact of mHealth apps, however, most such studies rely heavily on data collected from controlled experiments, and there is lack of quantitative studies reflecting user experience and perceptions from the end-user’s perspective. To fill the gap, we analyze user review comments from 20 mHealth apps including AI-chatbot, particularly for mental health, extracted from Google Play Store. We highlight challenges and opportunities to design Human-Centered AI systems to better support user experiences and perceptions for at-risk population.
2021 Tri-State ExploreCSR Sponsored by Google Research
Designing Human-Centered AI Systems to Improve Student EngagementDetecting Student Emotion and Flow in College Students During the Online Courses – Objectivity-Oriented Approach in the Context of Coding Practice for Engineering Classes (Received Best Poster Award)