In this Project, we hope to make an extended reality (XR) application to facilitate learning English anytime and anywhere.
Need Finding - Emphasising
Interview
Interviewees
Melody:
6 months service in Cambodia
Helping primary school to college students
Nam:
2 service trips to Cambodia
Met the poorest kids in slumps
RUPP (Cambodia) student:
"Excellent English speakers are rare... But we know we need it."
Ordinary Cambodia Student...
Mother tongue: Khmer
Around them:
English speaking tourists
Chinese merchants
Learn little English at school
Enjoy any entertainments on phones
Insights
Digital literacy:
Familiar with smartphones
Mainly for entertainment
English Proficiency:
Simple vocab
Need mapping of vocab to daily items
Motivation:
Knows the importance for upward mobility
Discouraged by boring lessons
Points of View (PoV):
PoV: Students in underdeveloped regions need an engaging way to to communicate/socialize with others at work and in life
Users: Students in underdeveloped regions
Observation: Need engaging learning experience
Need from insight: To communicate/socialize with others at work and in life
Review of Existing solution (Pros & Cons)
Pros
Textbook:
Cheap
Portable
Variety of contents
Video:
With Pronunciation
Easy to access anywhere
Online Class:
With Pronunciation
Interactive
Cons
Textbook:
Non-interactive
Fixed contents
Video:
Non-interactive
Fixed contents
Online Class:
Can not be accessed anytime
Need Finding - Ideation & Design Justification
Augmented Reality feasibility and suitability:
Interactive
Pronunciation
Access anytime anywhere
Unlimited Contents
Before the project, we also brainstormed some lifelong learning concepts and further extend in English learning perspective.
Scenario 1: Nervous teen approaches fluent English speaker at a social event. He stumbles and forgets words and feels embarrassed. With the AR app, he can find forgotten vocabulary.
Scenario 2: In a restaurant workplace, the young waiter approaches English-speaking tourists with menu and notepad. Teen struggles to answer questions due to language barrier. Teen uses AR app to learn relevant vocabulary and phrases such as the food.
Design Validation - Speed Dating
Respondent insights
The scenarios are appropriate (basic English used for social situations at work & in life)
Can help application of learn vocab
Fun and stress-free way to learn on-the-go
Feedback/suggestions
Could add more information about the vocabulary
Gamification or fun stuff to motivate users
Careful choice of vocab based on their life situation
Prototyping: Unity Gamification of AR learning experience
Prototype Design and Implementation:
Features:
Real time object scanning and classification
Vocab information: e.g. Example sentences and definition
Text-to-Speech: Pronunciation of the words and voice-over of the examples
Game Missions
(Extra feature to be implemented: user contribution to identify and submit customed scanned object data)
Implementation:
Unity 3D Game Engine in AR: Canvas System, Visual Scripting, Game Mission and user contribution features
Unity Barracuda to import .onnx object recognition models
Customized and trained YOLO object recognition models
Google Voice sample for text-to-speech feature
Prototype Demonstration Video
Evaluation - User Testing
Questionnaires filled by the tester and the results of the survey
1. User-friendly:
All respondents agree/strongly agree that the prototype user-friendly.
3. Social Communication:
100%/75% agree that the app is helpful for language learning/social communication.
2. Achieving the goal of lifelong learning:
All respondents see the app being used frequently/for a long time.
4. AR Enhancement:
All respondents agree that using AR technology makes the app better.
Contribution and Reflection
My major role in this English learning AR project was to create a working prototype to illustrate the immersive learning experience and the possibility of lifelong learning with Augmented Reality technology.
In the ideation stage, we all brainstormed some ideas about how and who we should help for their lifelong learning goals and explored different possibilities of the solutions.
After the idea validation of the solution, I started to make the AR prototype application with Unity. After gathering information of the vocab content of the application, I implemented the object recognition features, Augmented reality environment set-up in the gamified application, text-to-speech features, and the game elements features. Then, we test and fix the user interfaces with actual devices.
Then, we build the prototype and deploy into the actual mobile devices for demonstration video and user testing. I also recorded the demonstration video.
At the early stage, one of the group mates proposed an alternative lifelong learning idea which is about the education among prisoners. It was a great idea but I suggested that it very hard for us to work on the user research, user testing and validation. And we aborted this direction of development and focused on the lifelong learning of teenager which specifically on English learning after further narrowing down our scopes.
It is a hard decision and we learnt a lot in terms of team-working and cooperation with a better project planning by understanding and identifying what should the project goals be, who are the target users, what we want to do and what we can do to achieve the goal.
I was also inspired by the finding from learning experience of students and teenagers living in less developed countries. In this project, we understand that new technologies are only the prerequisite. To achieve the goal of the product, we need to understand the users. I recalled one thing learnt by the seniors, "Do not get married to your ideas". Sometimes we think the technologies or ideas are so fancy and good but that may not be needed and unable to address to the actual user needs. After understanding the needs, we can actually make people's life better with the knowledge we have. And that is what makes the society advance.