LAK 2023 Workshop - Collaboration Analytics

Collaboration is central to learning. However, analytic methods applied to analysis of small group collaboration are still in research stages and have yet to have significant impact in supporting students and educators in the classroom. In addition, collaboration analytics methods have been developed across a wide range of field, focusing on different aspects of group interaction, and cognitive, social, and affective states. 

This half-day interactive workshop will bring together a diverse group of researchers who are working with student collaboration data and developing collaborative analytics. Participants will have the opportunity to share their methodology as well as learn about other approaches that may come from different perspectives of collaborative analytics. In a series of guided discussions and interactive sessions, participants will be able to work with their own and others’ sets of data to try different approaches to analyzing student collaborative work.  

Workshop Organizers:

Peter W. Foltz, University of Colorado Boulder

Sadhana Puntambekar, University of Wisconsin

Jamie Gorman, Arizona State University

Jason Reitman, University of Colorado Boulder

Sidney D’Mello, University of Colorado Boulder

Overview of Collaboratiom Analytics

Learning is inherently collaborative and social (e.g., Bransford et al., 2000; Vygotsky, 1978). As preparation for careers, graduates will be expected to work closely with others (Levy & Murmane, 2005). Indeed, 94 percent of employers surveyed as part of the MetLife Survey of the American Teacher characterized working in teams as either “very important” or “absolutely essential” (Markow & Pieters, 2011).  The need for developing collaboration skills has also been reflected in national and international educational standards for math and science (NGA, 2010; NGSS, 2013) as well as for developing skills for the 21st century workforce (Griffin, McGaw & Care, 2012). These standards emphasize engaging students in collaborative knowledge-building and problem solving to develop disciplinary ideas and reasoning through a range of activities and types of tasks. Despite this increased emphasis on incorporating collaborative learning throughout the curriculum, techniques for teaching, assessing, and providing feedback are not widely implemented in classrooms or well embedded within curricula. This is largely due to the fact that it can be challenging for a teacher to orchestrate rich collaborative learning activities in the classroom and monitor and support teams of students all interacting simultaneously in real-time.

Learning analytics applied to student interaction data provide a means to instrument, measure, and understand the rich collaborative experiences that can unfold in educational settings. Methods for measuring collaborative learning have been researched, developed, and implemented across different disciplines using varied theoretical and methodological perspectives (e.g., Suthers et al., 2013). These include:   

• Skills frameworks, such as the internationally-recognized PISA (OECD, 2015) and ATC21s (Griffin, Care & McGaw, 2012), provide detailed descriptions of human-human collaborative problem-solving skills and links to concrete indicators of behaviors related to those skills. 

• Computer Supported Cooperative Learning (CSCL) has worked to understand how individuals learn in groups, how groups of learners construct shared knowledge, and how technology interacts with that learning (e.g., Dillenbourg, 1999; Puntambekar, Erkens & Hmelo-Silver, 2011; Wise & Jung, 2019.)

• Conversational agent and Natural Language Understanding research has analyzed student discourse to mine the rich linguistic content generated by students and incorporated frameworks such as “academically productive talk”, APT (Michaels & O’Conner, Kumar & Rosé, 2010) which examines discourse moves to support and facilitate collaborative conversations where students share and build on each other’s ideas.

• Team Science has emphasized measuring real-time constructs of individual and team cognition and the dynamics of change over situations (e.g., Cooke et al., 2013; Gorman et al., 2020) and team communication frameworks have focused on communication style to address both team cognition and peer mentoring theory that how teams communicate can impact their functioning more than how much they communicate (e.g., Marlow, Lacerenza, & Salas, 2017).

• Distributed cognition views cognition as processes that go beyond any individual’s mind and incorporates other individuals and artifacts of the work environment to examine them as an interacting whole system (e.g., Hutchins, 1995; Wright et al., 2000). 

• Multimodal, multiparty methods have focused on characterizing the rich sources of information from modalities such as gesture, body movement, eye-gaze, paralinguistics, body movement and linked these markers to social/cognitive/affective states related to collaborative performance (e.g., Praharaj et al., 2019).

Each perspective provides valid, yet slightly different analytic-based windows that elucidate our understanding of collaboration as whole.  However, research efforts seldom incorporate more than one perspective or technical approach. The workshop will bring together researchers from different perspectives to discuss their approaches as well as work interactively and hands-on with shared data sets.

References

Bransford, Brown, D. A. and Cocking, R. eds. (2000) How people learn: Brain, mind, experience, and school committee on developments in the science of learning, National Academy Press: Washington, DC.

Brannick, M.T. and Prince, C. (1997), An overview of team performance measurement, in Team performance assessment and measurement. Theory, methods, & applications, M.T. Brannick, E. Salas, and C. Prince, Editors., Lawrence Erlbaum Associates: Mahwah, NJ. p. 3-16.

Cooke, N.J., Gorman, J. C., Myers, C. W. and Duran, J. L.  (2013), Interactive team cognition. Cognitive Science,. 37(2): p. 255-285.

Dillenbourg, P. (1999), What do you mean by collaborative learning, in Collaborative-learning: Cognitive and computational approaches, P. Dillenbourg, Editor.: Oxford. p. 1-15.

Griffin, P., Care, E. and McGaw, B. (2012). The changing role of education and schools, in Assessment and teaching of 21st century skills, P. Griffin, B. McGaw, and E. Care, Editors. 2012, Springer: Heidelberg. p. 1-15.

Gorman, J.C., Grimm, D.A., Stevens, R.H., Galloway, T., Willemsen-Dunlap, A.M., & Halpin, D.J. (2020). Measuring real-time team cognition during team training. Human Factors, 62, 825-860.

Hutchins, E.L. "How a cockpit remembers its speed." Cognitive science, 1995, Vol. 19, pp. 265-288.

Kumar, R. and Rose, C. P. (2010) Architecture for building conversational agents that support collaborative learning. IEEE Transactions on Learning Technologies, 4(1): p. 21-34.

Markow, D., & Pieters, A. (2011). The MetLife survey of the American teacher: Preparing students for college and careers. New York, 

Marlow, S. L., Lacerenza, C. N., & Salas, E. (2017). Communication in virtual teams: A conceptual framework and research agenda. Human Resource Management Review, 27(4), 575-589.

Michaels, S. and O’Connor, C. (2015) Conceptualizing talk moves as tools: Professional development approaches for academically productive discussion, in Socializing intelligence through talk and dialogue, L.B. Resnick, C.S.C. Asterhan, and S.N. Clarke, Editors., American Educational Research Association. p. 33-248.

OECD, PISA 2015 Collaborative Problem Solving Framework, (2015), Organisation for Economic Cooperation and Development (OECD).

Praharaj, S., Scheffel, M., Drachsler, H. and Specht, M. (2019). Literature Review on Co-Located Collaboration Modeling Using Multimodal Learning Analytics. Can We Go the Whole Nine Yards? IEEE Transactions on Learning Technologies. 14, 3 (2019), 367–385.

Puntambekar, S., Erkens, G., & Hmelo-Silver, C. (Eds.). (2011). Analyzing interactions in CSCL: Methods, approaches and issues (Vol. 12). Springer Science & Business Media.

Suthers, D., Lund, K., Rosé, C. P., Teplovs, C., & Law, N. (2013). Productive multivocality in the analysis of group interactions. New York: Springer.

Vygotsky, L., (1978)  Mind in society: The development of higher psychological processes, Cambridge. MA: Harvard University Press.

Wise, A. F., & Jung, Y. (2019). Teaching with analytics: Towards a situated model of instructional decision-making. Journal of Learning Analytics, 6(2), 53-69.

Wright, P.C., Fields, R.E. and Harrison, M.D. (2000). Analyzing Human-Computer Interaction as Distributed Cognition: The Resources Model. Human-Computer Interaction, Vol.15.