Embedded Files
After we finished the implementation of our web application, we asked a total of seven people to try out our product. We first interviewed three users after they tried our product, and we presented their comments in our final presentation on Thursday, April 29th. The next day, we created a survey, and asked four more people to test our product. Overall, most of them make positive comments, except a few people think that our chatbot system can be improved more.
After we finished the implementation of our web application, we asked a total of seven people to try out our product. We first interviewed three users after they tried our product, and we presented their comments in our final presentation on Thursday, April 29th. The next day, we created a survey, and asked four more people to test our product. Overall, most of them make positive comments, except a few people think that our chatbot system can be improved more.
Survey
Survey
We sent out surveys to those who tested our project, and here's the result.
How does the website look?
How does the website look?
Surprisingly, they all give positive response on our website's appearance.
Surprisingly, they all give positive response on our website's appearance.
To what extend does the chatbot sound natural?
To what extend does the chatbot sound natural?
Using the chatbot system that is inspired by Eliza and some NLP analysis, we are happy that our chatbot sounds mostly natural (non-robot-like).
Using the chatbot system that is inspired by Eliza and some NLP analysis, we are happy that our chatbot sounds mostly natural (non-robot-like).
Which feedback feature did you use in evaluating your chat sessions?
Which feedback feature did you use in evaluating your chat sessions?
This one also shocks us because we thought the word cloud and mood chart are probably the most useful ones. However, it seems that the topic that is identified is the most helpful one.
This one also shocks us because we thought the word cloud and mood chart are probably the most useful ones. However, it seems that the topic that is identified is the most helpful one.
Was the chatbot successfully able to see your emotions and respond to it accordingly?
Was the chatbot successfully able to see your emotions and respond to it accordingly?
One of the user told us that the feedback system is pretty accurate, and whenever she typed something that is negative, the chatbot is able to see her problem and replied with interesting responses.
One of the user told us that the feedback system is pretty accurate, and whenever she typed something that is negative, the chatbot is able to see her problem and replied with interesting responses.
Would you recommend this software to those who have mental disorders?
Would you recommend this software to those who have mental disorders?
Our last question is to ask whether the user will recommend the app to a real patient, and 6 out of 7 people say "Yes", with only 1 person say "Maybe". Therefore, we can conclude that our app is quite successful.
Our last question is to ask whether the user will recommend the app to a real patient, and 6 out of 7 people say "Yes", with only 1 person say "Maybe". Therefore, we can conclude that our app is quite successful.
In the end, we received two responses saying that our chatbot has some questionable responses. Here are their comments:
It keeps saying the same sentence sometimes.
I wrote "I'm gay" and it responded "How do you feel about gay?" As funny as this is, perhaps work a bit on the linguistic aspect.
Of course, our chatbot is not perfect. Nonetheless, we believe that in the future, we can add more NLP analysis to make the chatbot's response more reasonable and clear.
Interview
Interview
We also interviewed three UR students and asked them to provide their thoughts on our project. Here are their comments.
User K-R
User K-R
“The glorified chatbot I believe will be very useful in terms of making people go deeper into how they see the world. It's something that makes people talk about what they want to talk about and further elaborate on the issue at hand because they are able to detect certain things in relation to your emotion and can find out about some of the core issues in people. I wish it had something that realtime shows your emotions however based on the mood of the conversation because it will detect things that people might not be able to see for itself at that moment. The word cloud feature is neat, however should include more nouns instead of emotions because people in therapy tend to carry negative emotions, and having that being shown directly towards the end might not click with some people.”
User S-L
User S-L
“I think the idea of creating a chatbot to conduct mental therapy for people is great. To be honest, I am not comfortable talking to a real therapist because it’s a bit intimidating. Using this chatbot makes me more comfortable because I know I am not talking to a real therapist. However, I think the script of the chatbot could be improved. Sometimes when I ask a question, it asks a question back, that is kind of funny. If you guys could have a team of real therapists to work on the script, this project would be more amazing.”
User N-P
User N-P
“I think it fits pretty well into my skeptical idea of what that kind of therapy entails; I’ve seen actual therapists that are like ‘tell me more...interesting...but how does that make you feel?’ For parts of it it’s pretty transparent about what it’s doing, but it’s not ineffective. I appreciate it starting with ‘How is your day going?’ It’s pretty daunting to just sit down in a therapist’s office and they say ‘what brings you here today?’ I’ll be honest, I had very low expectations, since part of therapy is the human element; someone to talk things through with; the only time it broke my expectation is when i asked it to change topics, and it didn’t register the change in topics.”
Page updated
Report abuse