Story Completion Tasks
Scenario 1:
During this summer break, Charlie’s university has implemented an intelligent AI Robot with an education system which will assist teaching. It is the first lecture of this semester. Charlie walks into the classroom. He heard his classmates whispering: There will be an In-Class Monitoring by using the Classroom Behavior Analysis System:
When students walk into the classroom, the system records their attendance by Facial Recognition Technology. It will then be generated as engagement data stored in Classroom Attendance System [15].
During the class, the system will capture students’ affective (emotional) states, such as when a student is either confused or confident, using an Emotion Recognition Technology [1,5,7,8]. Then the analysis report will be created for teachers to understand students’ emotional readiness and cognitive needs for their learning [5,7,8].
Each student’s learning material and feedback will be generated via real-time analysis in class [1,4,10,12], to build a profile of the student’s progress in each class. Across a term, this data will generate a personal “Education Profile", which aims at improving students' learning.
Now the class bell rings, Charlie feels …
Scenario 2:
The lecturer said: “let’s have a group discussion”. Charlie then heard another whisper from his classmates:
During the in-class groupwork practice, the system will also identify what each student is doing and saying, by using Facial Recognition and Voice Recognition. Eye Tracking Technology will be used to capture the amount of time that each student is focused on each learning resource [2,9,10]. Then the analysis report will be created by the Eye Tracking for the teachers to understand the learning progress of the class and the efficacy of the chosen learning recourses, and therefore to adjust the teaching strategy to better suit students' needs.
Now the classroom becomes noisy and full of conversations, Charlie …
Scenario 3:
On another day, Charlie learns that there is a digital “Education Profile” for every student. All their education-related data (data of in-class emotion, voice, face, gesture, academic score, etc.) will all kept in the “Education Profile” and will provide real-time analysis during class:
It will automatically identify and analyze individual academic weaknesses and strengths [3] based on each student’s “Education Profile”. Then it will suggest a personalized learning pathway. It will also automatically generate individual recommendations of learning resources for each student, targeting the identified weaknesses in their current profile.
The Education Profile will also work with an Academic Early Warning System [6,11,14]: It has a high accuracy of predicting when a student is likely to drop-out or fail from a subject. It captures the learner’s mindset and how it changes over time, such as a student’s attitude towards a lecture changed from positive to negative and unwillingness to engage increased. It has strong indicators to track students’ progress and identify changes in each student’s learning confidence and motivation toward a course. The system can automatically trigger an alert to ask for timely interventions to support students’ learning, such as providing targeted feedback to teachers.
The Education Profile will also work with an Intelligent Evaluation System for students: It provides automatic assessment for student’s homework and exam, then automatically sends the results into the “Education Profile” and provide analysis of the weakness of student’s study. It can also provide customized learning suggestions targeting the identified weaknesses for each student.
Later that day Charlie is in a group exercise on algebra, on a topic he finds quite hard. He knows the Education Profile is going to be used to see how he and his class are doing …. What happens next?
Table 1. Story Prompts Justification – Proposed or Existing AIEd Artifacts
Reference List:
[1] Karan Ahuja, Dohyun Kim, Franceska Xhakaj, Virag Varga, Anne Xie, Stanley Zhang, Jay Eric Townsend, Chris Harrison, Amy Ogan and Yuvraj Agarwal. 2019. EduSense: Practical classroom sensing at Scale. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3, 3, 1-26.
[2] Karan Ahuja, Deval Shah, Sujeath Pareddy, Franceska Xhakaj, Amy Ogan, Yuvraj Agarwal and Chris Harrison. 2021. Classroom Digital Twins with Instrumentation-Free Gaze Tracking. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, Yokohama, Japan, Article 484. http://dx.doi.org/10.1145/3411764.3445711
[3] Daniela Altun, Christopher Krauss, Alexander Streicher, Christoph Mueller, Daniel Atorf, Lisa Rerhaye and Dietmar Kunde. 2022. Lessons Learned from Creating, Implementing and Evaluating Assisted E-Learning Incorporating Adaptivity, Recommendations and Learning Analytics. In International Conference on Human-Computer Interaction. Springer, 257-270.
[4] Sinem Aslan, Nese Alyuz, Cagri Tanriover, Sinem E Mete, Eda Okur, Sidney K D'Mello and Asli Arslan Esme. 2019. Investigating the impact of a real-time, multimodal student engagement analytics technology in authentic classrooms. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1-12.
[5] Mohamed Ez-Zaouia, Aurélien Tabard and Elise Lavoué. 2020. EMODASH: A dashboard supporting retrospective awareness of emotions in online learning. International Journal of Human-Computer Studies 139, 102411.
[6] Dragan Gašević, Shane Dawson, Tim Rogers and Danijela Gasevic. 2016. Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success. The Internet and Higher Education 28, 68-84.
[7] Kenneth Holstein, Bruce M McLaren and Vincent Aleven. 2017. SPACLE: Investigating learning across virtual and physical spaces using spatial replays. In Proceedings of the Seventh International Learning Analytics & Knowledge Conference. 358-367.
[8] Kenneth Holstein, Bruce M. McLaren and Vincent Aleven. 2018. Student Learning Benefits of a Mixed-Reality Teacher Awareness Tool in AI-Enhanced Classrooms. In Springer International Publishing, 154-168.
[9] Stephen Hutt, Kristina Krasich, James R. Brockmole and Sidney K. D'Mello. 2021. Breaking out of the lab: Mitigating mind wandering with gaze-based attention-aware technology in classrooms. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1-14.
[10] Adam Linson, Yucheng Xu, Andrea R English and Robert B Fisher. 2022. Identifying Student Struggle by Analyzing Facial Movement During Asynchronous Video Lecture Viewing: Towards an Automated Tool to Support Instructors. In International Conference on Artificial Intelligence in Education. Springer, 53-65.
[11] Sungjin Nam and Perry Samson. 2019. Integrating Students’ Behavioral Signals and Academic Profiles in Early Warning System Springer International Publishing, 345-357. http://dx.doi.org/10.1007/978-3-030-23204-7_29
[12] Yanyi Peng, Masato Kikuchi and Tadachika Ozono. 2023. Development and Experiment of Classroom Engagement Evaluation Mechanism During Real-Time Online Courses. In International Conference on Artificial Intelligence in Education. Springer, 590-601.
[13] Xingran Ruan, Charaka Palansuriya and Aurora Constantin. 2023. Affective Dynamic Based Technique for Facial Emotion Recognition (FER) to Support Intelligent Tutors in Education. In International Conference on Artificial Intelligence in Education. Springer, 774-779.
[14] Wanli Xing, Xin Chen, Jared Stein and Michael Marcinkowski. 2016. Temporal predication of dropouts in MOOCs: Reaching the low hanging fruit through stacking generalization. Computers in human behavior 58, 119-129.
[15] N Selwyn. 2019. Digital lessons? Public opinions on the use of digital technologies in Australian schools. Melbourne, Monash University. Retrieved May 10, 2020, 2019-2001.