Arshia Khan, Ph.D.

Professor & Director of Graduate Studies, Dept. of Computer Science

email:; Phone: 218-726 7971, Office: HH 327B

Lab: Dementia and Elderly Care Robotics and Sensing Lab - Vkh 17;

Swenson College of Science and Engineering

University of Minnesota Duluth


Humanoid Therapeutic Robotics and Wearable Sensors: My research interests span the robotics, medical informatics, and biomedical engineering requiring interdisciplinary collaborations with experts such as cardiothoracic surgeon, neuropsychologists, roboticist, psychologist, physical therapist, dietician, nurse and occupational therapist. Given my engineering background and interest in medicine, my research has evolved into various segments of biomedical engineering such as robotic assistive technology, sensor based assistive mobile technology, and personalized medicine.

Assistive therapeutic robot to address BPSD in dementia care: Humanoid robot administering reminiscence therapy

Dementia Friendly Intelligent Living Space

We are investigating the feasibility of combining assistive therapeutic robots, an intelligent pill dispenser, and wearable sensors to complement human caregiving in a intelligent space equipped with sensors and a dementia friendly atmosphere.

Robotic Assistive Technology for Dementia

Robotic assistive technology: Providing assistance and companionship while monitoring older adults diagnosed with Mild Cognitive Impairment (MCI)

The aim of this interdisciplinary research project is to develop an innovative and novel robot assistive technology to help reduce the challenges and burden faced by caregivers and individuals affected with ADRD. A dedicated autonomous robot can learn the daily living activities of the affected individual by following the individual and then assist the person as her/his ability to maintain the daily living activities are reduced due to the progression of dementia. We are working on a companion robot that would be assigned to an individual when diagnosed with MCI. This robot will serve as a companion to the person by interacting with the person and as the disease progresses, the robot will remind and assist the individual of the daily living activities.

Robotic Assistive Technology

More recently I have been working in the area of robotic assistive technology in the areas of recovery after open heart surgery and helping maintain quality of life in individuals affected with Alzheimer's disease.

Recovery after OHS: I am currently collaborating with Dr. Mary Boylan, a cardiothoracic surgeon from St. Lukes Hospital in developing a robot assistive technology to help patients recover after open heart surgery. We are using Baxter the humanoid robot in exploring impedance control to help a patient out of bed. I have secured the Whiteside grant to conduct a proof of concept study to explore the use of robotics in helping individuals recover after open heart surgery.

Robotics and Dementia: In another project, I am working on using a robot to follow an individual recently diagnosed with mild cognitive impairment and apply machine learning to learn the patterns of daily living so at an advanced of dementia the robot can prompt and remind the individual of their activities of daily living.

Happy Times: Aiding Recall of Deeply Embedded Memory :

This study is based on research conducted by Neurologist, Dr. Oliver Sacks, where music is used to recover deeply embedded memories. This mobile solution seeks to recreate events through use of music, images and textual descriptions. The objective is to stimulate patient memories affected by Alzheimer’s Disease(AD). An iOS application was developed that uses cues such as images/pictures, of an important event of choice paired with a series of music clips associated with the event creating a multimedia slide show. By doing so, the affected person is encouraged to watch and listen, as they become distressed when unable to recall events. The application was developed to provide a calming effect on the individual. As the disease progresses, the individuals affected with AD start to withdraw from social situations and demonstrate uncharacteristic anger or frustration. Irrespective of their stage in AD, all people were able to respond and react to musical stimulation. Cues help jog deeply embedded memories related to an event; for example, incorporating pictures of important events such as a wedding or birthday into a slide show can help the affected individual calm down. This calming effect potentially helps the affected individual recall associated memories.

Prompting Mechanisms:

The goal of this study is to evaluate prompting tools/devices to rate them based on their ability to enhance compliance and the least amount of stress/annoyance. The healthcare field is infiltrated with various types of devices from wearable sensors to mobile robots. These devices are being used to enhance and augment healthcare delivery in tasks such as reminding, prompting, assisting and supervising individuals affected with chronic ailments. There is little done in the area of evaluating these devices and rating them in their ability to enhance compliance, their effectiveness, their levels of annoyance/tolerance by humans, user preferences, user trust, friendliness and helpfulness. We have selected 4 devices in the range of wearable to mobile autonomous devices- Pepper a mobile autonomous robot, Cozmo a miniature mobile robot with limited dexterity, a tablet and the Apple watch. We have developed an application that will run each of the selected devices in order to analyze their effectiveness, friendliness, compliance, annoyance, trust and user preferences. The application we developed reminds individuals to correct posture every few minutes.

In this user study that is conducted with healthy participants, we will test our app that we have created to help remind individuals to maintain correct posture, with the selected 4 devices. We will to examine the reaction of the participants to prompting for posture correction using each of these devices. We will use the Empatica wearable sensor, that will be placed on the participants wrist to measure the stress levels when prompted by the devices such as the Pepper, Cozmo, tablet and the Apple watch.

The objective of this study is to investigate various(4) devices based on each of the following criteria:

1. Compliance level of the participants to the prompting

2. Level of stress/annoyance caused to participants

3. Participant preference

4. Effectiveness

5. Trust ability

6. Helpfulness

7. Friendliness

Bipolar Depression Druid: A Framework to Monitor, Track and Predict Bipolar Depression

Bipolar Disorder affects approx. 5.7 million adult Americans(NIHM). It is the 6th leading cause of disability in the world(WHO) driving a need of constant monitoring. Early detection is the key to the prevention of adverse consequences of a bipolar episode that can in an extreme situation lead to suicidal attempts. With the wide use of mobile devices, there is a great potential to harness the power of size, mobility, convenience, cost efficiency and easy access to efficiently augment and complement the management of chronic illnesses such as

Bipolar Depression. This study is developing a mobile solution that uses wearable sensors such as heart rate and sleep patterns to monitor bipolar depression. The solution integrates data from the wearable wireless sensors to track the patient vitals with self reported data on mood patterns and medication adherence to identify the prodrome to predict a bipolar episode.

Informing Care Decisions and Aiding Children with Autism in Understanding Emotions

MyHeifer is an Autism Spectrum Disorder(ASD) intervention application aimed at better understanding patients’ behavioral patterns and informing healthcare decisions, easing caregiver burden and providing an emotional outlet for patients. Children with ASD often struggle with the complexity of human communication because of the array of verbal and non-verbal communication methods at play. Because of this, technological interventions can be a valuable tool for communicating with children with ASD because of their simplicity. Hence the MyHeifer application seeks to provide an un-complicated environment for children with ASD to express and explore their emotions. Children preform “actions” or “interactions” which are classified as either positive or negative behaviors. Through these interactions children learn various ways to react to situations. The choices children make are collected and serve as a basis for future healthcare decisions. Because communication is often difficult for children with ASD, utilizing data from past actions or interactions helps caregivers anticipate and understand the challenges to make better emotional and behavioral connections in individual patients in order to address personalized care needs.

Prevent Decubitus Pressure Ulcers: Preventive Weight Shifting Guide, Monitor, and Tracker

Tracker App for Wheelchair Users with Spinal Cord Injuries

This project uses 256 wireless pressure sensors along with accelerometer sensor in the prevention of pressure ulcers among wheelchair users. A Weight Shifting App has been developed to prevent pressure sores in wheelchair users with spinal cord injuries. Wheelchair users with spinal cord injury have a higher risk of developing pressure ulcers due to limited mobility and the countless hours they spend in the wheelchair, exerting pressure on the points of interface between the bony structure and the wheelchair cushion. The areas of interface that are under prolonged pressure lack blood flow, causing the tissue to breakdown, leading to a decubitus or pressure ulcer. Approximately 28.9 % wheelchair users in communities, 27% in nursing homes, and between 5 and 30% of hospitalized patients develop pressure sores. Nearly 70% of elderly patients develop pressure ulcers, which in turn significantly increase the healthcare management and costs and can be a cause of pain, discomfort, loss of independence and mobility in not only the elderly but also the younger patients. An app was designed and developed to notify, walk through the process of weight shifting and also track the movements of the patient performing weight shifting using the accelerometer. The app was tested to find that the forward lean movements can be tracked accurately using the accelerometer but the lateral movements although tracked could not verify that the weight shifting was correctly performed. In order to fix this issue the pressure map is used as an additional sensor to track and monitor the weight shifting accurately.

Pressure Ulcer Prevention: Clinical study

In this user study we will test an app that we have created to help perform weight offloading(see below). This study will be conducted with healthy patients to verify that the forward lean and lateral lean movements can be tracked accurately using the accelerometer and a pressure mat to ensure that the weight shifting/offloading was correctly performed. The following are two objectives that will be attempted to address in this study by observing and tracking the weight offloading data gathered in this study:

1. How can we verify weight offloading?

2. Can prompting an individual help them remember to weight offload?

The specific aims of this study are to test our app to verify our device can verify weight offloading is actually happening and if reminding a person to weight offload will help them remember to weight offload on their own even when they are not prompted to weight offload.

Weight offloading/shifting: is a precautionary pressure relief mechanism to prevent pressure ulcers. This process restores blood flow and offers relief to the parts of the skin that are weighted down due to body weight. Weight offloading involves lifting one leg at a time or lateral trunk lean or forward trunk lean, wheelchair pushups and wheelchair lean.

Manic Attack Prediction

Patients suffering from Bipolar disorder (BD) experience repeated relapses of depressive and manic states. Our research is based on the design and development of a continuous, autonomous sensor fusion based monitoring framework to identify and predict state changes in patients suffering from bipolar disorder.

IRB is approved for a user study that deals only with healthy participants and has limited scope as identified in the objectives below:

Investigating how psychological changes (Mood), affect physiological responses.

This project would involve:

1. Studying the correlation between heart rate variability and mood.

2. Studying the correlation between Electrodermal activity and mood.

Sensor Based Assistive Mobile Technology

In the area of sensor based assistive mobile technology my projects utilize wearable sensors to track heart rate, blood pressure, body surface temperature, oxygen saturation, location, accelerometer, and pressure sensors to monitor and track various physiological conditions that play a role in prevention of pressure ulcers, tracking, monitoring and management of bipolar disorder, detection of wandering in patients affected with dementia, sensors for gait analysis for force treadmill and .

In collaboration with medical student Danish Imtiaz and Dr. Adriana Seelye, a neuropsychologist from the Department of Veterans Affairs, I am exploring the use of EMF sensor in predicting behavioral and psychological symptoms of dementia (BPSD). Most studies conducted in this area are reactive in identifying the BPSD while we are exploring a proactive approach of prediction by combining reminiscence therapy and EMF sensor with mobile technology to help predict the onset of a BPSD episode.

I am also exploring the use of a combination of physiological and geolocation sensors in predicting wandering in individuals affected with Alzheimer's Disease (AD). Continuing my research on Alzheimer's, we are attempting to aid recall of deeply embedded memory, by building upon the research conducted by Neurologist, Dr. Oliver Sacks, where music is used to recover deeply embedded memories. A mobile application was developed by combining cues such as photos, music and videos from an event of significance importance to the affected individual, into a multimedia presentation to help jog deeply embedded memories related to this event.