CrowdSpeak

CrowdSpeak is a crowd-sourced transcription program that automatically transcribes the user’s video and subsequently utilizes crowd-sourced feedback to critique the user’s tone, pronunciation, and accuracy. This provides users an outlet to independently conduct oral practice outside of a school or formal setting.

Goal: Make auto transcriptions algorithms better using crowdsourced user input. Our product will produce a human sounding transcript with fixes and comments.

As English becomes more and more prevalent as a second language used around the world, it has become imperative that resources are available for individuals to independently practice their English proficiency. While it is quite formulaic to practice writing and written grammar, the task of effectively practicing and refining spoken English is much more ambiguous.

Crowdspeak was implemented using HTML5, CSS and Javascript. The responsive layout on CrowdSpeak was implemented using Bootstrap. Transcription is provided on-the-fly with the Google Web Speech API, while video is recorded simultaneously with MediaSource, a Javascript-based video recording API.

More Tech info

Should the CrowdSpeak project be continued in the future, there are a few features we would add. The first would be incentives for crowd sourcers. Due to the nature of how they edit the transcripts, it is a bit tedious to do without any reward. Ideally, we would pay them a small monetary value for their work. Amazon’s Mechanical Turk is a successful model for how we could approach this. Users will pay crowd sourcers for their services.

Another feature we would add would be graphical information. Users would be able to see a graph of their improvement and a graph of their common speech issues. Due to the wide range of comments that can be selected via radio buttons by crowd sourcers, storing that data and graphing it should not be too hard.

Once the English language version is feature-complete, we would like to expand to other languages. The speech-to-text API we used supports over 80 different languages, which should allow us to rapidly expand support to additional languages.

Finally, we would touch up our user interface and create a mobile app so people could use CrowdSpeak on the go. We would also integrate it with social media. This would allow users to share their progress online and hopefully increase awareness of our app