Embedded Files
A Closed-Caption Aid for Lecture Halls
Motivation:
Throughout the US alone, over 13.2 Million 3.3 million people are affected by ADHD and Auditory Processing Disorder (ASD) respectively. It has also been found that a large majority of college students have trouble focusing in the classroom as well. With so many information in lectures being conveyed based on focusing and understanding what is being said to someone those who are struggling have a hard time learning. Many students struggle to understand the information being taught to the within lecture settings, especially those who are not auditory learners. This led to us asking:
How Might We Make Lectures More Engaging and Accessible for College Student?
Human-Centered Design:
Initially we decided to focus on designing a system for people exclusively with ADHD and ASD. So our initial systems were being created with only one user input involved. However through expert outreach and surveys, we learned that more than just these populations are affected by this problem. Through speaking with Language Speech Pathologists and therapists we decided on learned how the issue can be tied to not just the person receiving information but giving it as well. This lead us to deciding our user to be students who struggle with focusing on what an instructor is saying in a classroom setting. As the professors would have to interact and use this system too, we had to keep them in mind with future iterations making the professors stakeholders.
30 user survey responses
10 user interviews
4 expert interviews
35 early prototype tests
5 current prototype tests
7 design iterations
Solution
Solution
Our eventaul final design was this app seen on the left. Students would be able to click the "Start Class" button to begin receiving the captions. With the size button they could also change the size of the text.
System Diagram
The teacher will wear a wireless microphone that acts as the audio input to our custom Raspberry Pi device. This device acts as the brain of the entire system where an opensource machine learning API called DeepSpeech processes the audio and creates a text output. The Raspberry Pi then also acts as a servo to upload the text output to a destination where the student's device will pull from. The student's device will be able to pull said text and alter the size as it appears on their own screen.
Design Goals:
Increase the users' understanding of a speaker
Accessible and customizable
Intuitive/easy to use
Does not distract other students
Results:
Intuitive for the Student and Teacher
Intuitive for the Student and Teacher
Achieved
Verbal Eyes has been designed so that it can be easily be used by both teacher and student. With the teacher being able to activate the system with only the push of the button and the student accessing by only clicking one on their phone very little technological knowledge is needed to use the system.
Customizable
Customizable
Not Achieved
A level of customizability is available with changing the size of the text but the scope with which was set out has not been achieved. In the future we wish to implement the choices of text and background color.
Multi-User Availability
Multi-User Availability
Not Achieved
With the current setup only one user at a time is able to access the Verbal Eyes system. By implement a dedicated server or finding a better method of communication can this be achieved.
Accessible to All Students
Accessible to All Students
Achieved
Anyone one who has access to a smart device is able to use our system
No Non-User Distraction
No Non-User Distraction
Achieved
Implementation of the system onto users own personal device allows those who do not choose to use it to not be subjected to closed captions
Future Directions:
Add additional customizability (colors, languages, access to transcript)
Develop a server so more users can connect to the system at once.
Use an advanced speech to text vocabulary library for accurate transcription.
Acknowledgements and references
We would like to thank Dr. Joseph Samosky and the Art of Making staff, as well as our experts and the students who gave us feedback.
Meet the Team
Meet the Team
Meet the Team
From Left to Right: Reece Bashore, Connor McDermond, Sophia Buda, Kate Bonifacic
Acknowledgements:
Acknowledgements:
We would like to thank Dr. Joseph Samosky and the Art of Making staff, as well as our experts and the students who gave us feedback.
Contact Us: VerbalEyes@pitt.edu
Contact Us: VerbalEyes@pitt.edu
Page updated
Google Sites
Report abuse