Reflecto

How might a new tool use multimodal classroom analytics to help teachers reflect, collaboratively with trusted colleagues, on their classroom practices?

When students use intelligent tutoring software, teachers monitor the class and provide one-on-one help where needed. Photo courtesy of Carnegie Learning, Inc.

Challenge

An important way that K-12 teachers engage in professional learning is by reflecting on their own practices, for example, by reviewing video recordings from their class sessions. Yet, even with video, effective reflection is hampered by the fact that it is nearly impossible for teachers to notice and remember all that happens during any given classroom period. It is likely that classroom analytics can fill in some of the blanks in teachers’ observations and thereby support effective reflection.

Solution

Working with teachers, we designed, implemented, and pilot-tested a new analytics-based tool, Reflecto, for K-12 teachers, to be used in conjunction with intelligent tutoring software.

Conducted design-based research with teachers to understand teachers’ current reflection practices and how they relate to student learning, their needs and preferences regarding the use of analytics for reflection. We also probed, with teachers, what tool designs might prove effective.
Designed and developed Reflecto, a new teacher-facing reflection tool powered by multimodal classroom analytics to help teachers reflect, collaboratively with trusted colleagues, on their classroom practices.
Conducted a classroom pilot study with two teachers to get a first impression of whether the new tool is useful to them.

Location sensors record the teacher's position during class sessions. A "visit detection algorithm" distills teachers' visits to students from the location data.

At the beginning of a Reflecto session, teachers select possible trends in their classroom practices to explore with data.

An interactive seatmap visualizes the teacher's trajectory through the classroom, including visits to students.

Discoveries

Multimodal analytics help relate teacher actions and student learning: By combining analytics derived from location sensors (placed in classrooms) and from the log data that captures students’ interactions with an AI-based tutoring system, the tool supports open-ended data exploration by teachers, focused on confirming the presence of desirable trends or the absence of undesriable trends in teachers' practice.
Hypothesis-driven exploration of multimodal analytics can spur useful reflection. We found that Reflecto helps teachers validate and challenge existing knowledge, surface surprises, and, under appropriate conditions, translate reflection into instructional change. The teachers found Reflecto helpful. The study also revealed challenges related to confirmation bias, sensemaking, and tool usability.

Impact

The project contributes a new analytics-based tool for the smart K-12 classroom, focusing on a scenario in which the use of analytics has great promise but is underexplored, namely, teacher reflection after class.
Analytics-based tools that help teachers reflect are rare and often not tailored to teachers’ needs. The project shows that exploring trends with multimodal data (student interaction data and teacher position data) can be a strong basis for collaborative teacher reflection.
It thus has the potential to improve a form of K-12 classroom instruction that is already quite effective, namely, learning with ITS, an increasingly common learning environment.

Publications

Borchers, C., Wang, Y., Karumbaiah, S., Ashiq, M., Shaffer, D. W., & Aleven, V. (2024). Revealing networks: Understanding effective teacher practices in AI-supported classrooms using transmodal ordered network analysis. In B. Flanagan, B. Wasson, & D. Gašević (Eds.), Proceedings of the 14th Learning Analytics and Knowledge Conference, LAK 2024 (pp. 371-381). New York: ACM.
Falhs, A. C., Borchers, C., Echeverria, V., Yang, K., Rummel, N., & Aleven, V. (2025). How expertise levels shape preferences and reflection needs: towards AI reflection systems for teacher empowerment. In Tammets, K., Sosnovsky, S., Ferreira Mello, R., Pishtari, G., Nazaretsky, T. (Eds.), Proceedings of the Twentieth European Conference on Technology Enhanced Learning, ECTEL 2025 (pp. pp 120–125). Lecture Notes in Computer Science, vol 16064. Cham, Switzerland: Springer. https://doi.org/10.1007/978-3-032-03873-9_11
Karumbaiah, S., Syamkumar, A., & Aleven, V. (2025). Exploring Out-of-Tutor Interventions by Teachers in Supporting Student Learning with AI Tutors. In A. Rajala, A. Cortez, R. Hofmann, A. Jornet, H. Lotz-Sisitka, & L. Markauskaite (Eds.), Proceedings of the 19th International Conference of the Learning Sciences, ICLS 2025 (pp. 1709-1713). International Society of the Learning Sciences. https://repository.isls.org/handle/1/11348
Shou, T., Borchers, C., Karumbaiah, S., & Aleven, V. (2023). Optimizing parameters for accurate position data mining in diverse classrooms layouts. In M. Feng, T. Käser, & P. Talukdar (Eds.), Proceedings of the 16th International Conference on Educational Data Mining (pp. 310–317). Bengaluru, India: International Educational Data Mining Society.
Yang, K. B., Borchers, C., Falhs, A.-C., Echeverria, V., Karumbaiah, S., Rummel, N., & Aleven, V. (2024). Leveraging multimodal classroom data for teacher reflection: teachers’ preferences, practices, and privacy considerations. In: R. Ferreira Mello, N. Rummel, I. Jivet, G. Pishtari, & J. A. Ruipérez Valiente (Eds.), European Conference on Technology Enhanced Learning, EC-TEL 2024 (pp. 498-511). Cham, Switzerland: Springer Nature. https://doi.org/10.1007/978-3-031-72315-5_34
Yang, K., Zhao, Y., Borchers, C., Fang, Y., Kim, J., Butt, A., Popescu, O., Gupte, S., Chiu, A., Shen, P., Rummel, N., & Aleven, V. (2025). Reflecto: A teacher reflection tool leveraging multimodal learning and teaching analytics. Interactive Event at the 26th International Conference on Artificial Intelligence in Education, AIED 2025 https://osf.io/preprints/edarxiv/2mdj3

Page updated

Report abuse