About The Project

Statistics show that among one in fifteen adults and three in seven elderly people experience metal illness. It includes many different conditions that vary in degree of severity, ranging from mild-to-moderate to severe. We aim to lower the statistics, using our technology by introducing them to a virtual assistant, Emily. It’s functioned to talk to them, give emotional support and remind them of their allergies, emergency numbers, favorite song, favorite movie etc. We were inspired to work on a chatbot as such since dementia, depressed and anxiety people have trouble finding someone they can talk to and remind them of their favorite things. Our survey tells us that the caregivers of people with dementia need to be trained in a way that can test their patience, which made us go for a virtual assistant. It can repeat anything asked several times. For the lonely and depressed, we hoped they could find someone who could talk to them anytime of the day, thus leading us to add a therapy conversation.

Our solution gives them a virtual person who can assist them throughout the day, like a friend they hoped for. Unlike traditional chatbots, our solution uses an avatar, making it seem like the user is conversing with a friend via video-call. The bot has been trained to console lonely, depressed and people with dementia, they can talk about anything possible to the bot and the bot will be there to listen to them, anytime. They also seem to struggle in keeping up with a conversation, the bot will help them with it by asking about their day and reminding them of their favourite things, to end up getting them into a conversation. People with dementia ask questions repeatedly, as humans, it becomes difficult to answer over again, but a bot will be able to repeat itself as many number of times. They also tend to forget things they enjoy, the bot can help by reminding them of their allergies and emergency contact shared by the user, in times of danger. People with anxiety can overcome it by talking to a virtual assistant, which can keep conversations confidential.

The next steps of our project will be to implement: Patient Mode and Caregiver mode. Unfortunately, we lost a significant amount of data while training our chatbot due to DialogFlow crashing randomly, hence, we plan to train our chatbot/videobot again for much more richer responses once we can collect the data again. We also plan on adding API features like weather app, calendar integration, scheduling appointments and daily schedule, etc. In addition, a few games for the elderly people would help in engaging them with the chatbot even further. We also plan on adding a FAQ's page which will provide the Caregiver with the latest breakthroughs and information for Mental health issues. We plan on testing it with patients diagnosed with mental illness for more realistic test data. This data will help us improve our chatbot even further and will in turn help us more in meeting the required needs for people who are suffering with mental illnesses.

The first test we took was to send out a survey form to check how people of different age-groups would accept our idea for a digital human based chatbot. After getting numerous positive responses, opinions and feedbacks, we could go ahead to develop our chatbot. The second test we conducted was to check how the emotional support feature of the chatbot was working. Due to COVID-19 restrictions, we couldn't go and physically survey people. So, we created a link that people could use to talk to Emily (Our digital human chatbot). The majority of the responses we got stated that the emotional support feature still needed more fine-tuning. Finally, in our third test, we sent out the complete chatbot model via an online link which included: user-registration, free-chat, and storing and retrieval of data into a firebase. Around 75% users were happy with our chatbot model and told that it was better than normal bot conversations as they felt that it was similar to talking to someone over video call.

We agreed to work on Tailwind and React as our frontend. For backend, we used a bit of Express Js to have a small token server running. Apart from that, we have used DialogFlow as our main "brain" of the project, and for storing data with authentication, firebase helped a lot. The key point in our project is the Virtual Human Avatar, for which we had to use a platform called Uneeq. It's a paid service but they were generous enough to give us students 30days free trial extension which we are utilizing now to submit our idea. So, our front-end first gets connected to Firebase for Authentication. Next it gets redirected to our Profile Page. To start Virtual Human Avatar, we need a token server running. Uneeq provides easy integration with DialogFlow, hence we just need to start the avatar. For each details/intents, we store data into Firebase for easy retrieval and storage of new data. This is the workflow for our entire project.

We have a GitHub repo for our project. For security reasons, we can't expose it to public. But we can of course provide screenshots of the same!