Big Data, Research Challenges, & Science Convergence in Educational Data Science

The Third Workshop of the Learner Data Institute

A Half-Day Hybrid Workshop @ EDM 2022

July 27, 2022
13:30 - 16:15 British Summer Time (BST)
Durham, UK

Workshop URL: https://sites.google.com/view/learnerdatainstitute/ldiedm

For our past workshops, see LDI @ EDM '21 & LDI @ EDM '20!

Keynote Speaker

Richard G. Baraniuk
C. Sidney Burrus Professor of Electrical and Computer Engineering
Founder and Director, OpenStax
Rice University

Keynote Title: "Opening Learning Data While Releasing None of It"

Keynote Abstract: Despite a century of progress in learning science, there is a major roadblock in education research. We do not understand enough about learning to improve outcomes for all students, in large part due to the complex factors that drive student outcomes and the lack of large-scale studies. Over the past few years, a unique opportunity for large-scale learning science has arisen, as millions of students and teachers transition to digital learning. This talk will overview some of key challenges involved in transporting learning science into this space, including protecting student privacy and tradeoffs in research study size, depth of insight, and cost.

Agenda

All times British Summer Time (BST) (UTC + 1)

Note: All papers linked below are penultimate drafts. Please do not cite papers without permission from the authors.

13:30 - 13:40 – Introduction + Keynote Speaker Intro (Stephen Fancsali and Vasile Rus);

Progress Report on The Learner Data Institute (Vasile Rus, Stephen E. Fancsali, Philip Pavlik, Jr., Deepak Venugopal, Steve Ritter, Dale Bowman, and The LDI Team) [PDF]

13:40 - 14:20 – Keynote: "Opening Learning Data While Releasing None of It" (Rich Baraniuk)

14:20 - 14:30 – BREAK

14:30 - 14:45 – Using autoKC and Interactions in Logistic Knowledge Tracing (Philip I. Pavlik Jr.) [PDF]

14:45 - 15:00 – Exploring Predictors of Achievement-Goal Profile Stability during Mathematics Learning in an Intelligent Tutoring System (Leigh Harrell-Williams, Christian Mueller, Stephen Fancsali, Steven Ritter, Xiaofei Zhang and Deepak Venugopal) [PDF]

15:00 - 15:15 – Generating Multiple Choice Questions with a Multi-Angle Question Answering Model (Andrew Olney) [PDF]

15:15 - 15:30 – Clustering learners’ feedback processing patterns based on their response latency (Wei Chu and Philip Pavlik) [PDF]

15:30 - 15:45 – NeTra: A Neuro-Symbolic System to Discover Strategies in Math Learning (Anup Shakya, Vasile Rus, Stephen Fancsali, Steve Ritter and Deepak Venugopal) [PDF]

15:45 - 16:00 – Predicting Transfer in a Game-Based Adaptive Instructional System (Khanh-Phuong Thai, V. Elizabeth Owen and Ryan Baker) [PDF]

16:00 - 16:15 – Exploring Differences in Performance between Knowledge Tracing Methods & Gaming the System Behavior (Husni Almoubayyed and Stephen Fancsali.) [PDF]

Workshop Summary

The Third Workshop of the Learner Data Institute (LDI) builds on the success of two previous, virtual workshops (at EDM 2020 & EDM 2021) and seeks to bring together researchers working across disciplines on data-intensive research of interest to the educational data science and educational data mining communities. In addition to welcoming work describing mature, data-intensive or “big data” research and emerging work-in-progress that spans traditional academic disciplines, the workshop organizers welcome case studies of interdisciplinary research programs and projects, including case studies of learning engineering efforts pursued by universities, learning technology providers, and others (both successful and unsuccessful), as well as position papers on important challenges for researchers harnessing “big data” and crossing disciplinary boundaries as they do so.

We convene researchers and developers from diverse fields who seek to “harness the data revolution” in educational data science and “grow convergence research,” aligning with (at least) two of the U.S. National Science Foundation’s “10 Big Ideas” for emerging research and development opportunities. “Convergence builds and supports creative partnerships and the creative thinking needed to address complex problems” (NSF’s 10 Big Ideas: Growing Convergence Research), and we expect that bringing together highly experienced researchers, as well as students and early-career researchers, will stimulate substantial growth and interest in state-of-the-art, data-intensive, transdisciplinary or “convergent” approaches to solving vexing societal problems related to education. We also seek to explore the big data and learning engineering frameworks that will enable convergent solutions.

Questions & Areas of Interest

• How can we use massive and diverse datasets generated by adaptive instructional systems (AISs) to address core questions and challenges in learning science and engineering?

• Are learners, teachers, and learning science researchers successfully interacting with cyber-learning technologies?

• What are some critical challenges with respect to scaling the development of AISs across many domains and perhaps millions of learners?

• What are the limitations of AISs and adaptive components of instructional systems?

• Which aspects of learning are best handled by humans and which ones by cyber-learning technologies (and how do we enhance the interaction of the two)?

• How can data from student and teacher interactions with cyber-learning technologies, in and outside the classroom, be collected in ways consistent with best practices—e.g., with respect to data fidelity, security, reliability, privacy, human subject research protocols, school policies, parental consent, HIPAA, FERPA, etc.?

• Methodology, infrastructure, and workflows for “big data” and data-intensive educational research

• Inter/multi/trans-disciplinary approaches to data-intensive educational research

• Case studies of successful & unsuccessful efforts to practically harness insights from large datasets in settings where learning takes place (e.g., case studies of “learning engineering” efforts)

• Emerging challenges for researchers working across disciplines with large datasets

• Use-cases, workflows, and case studies (illustrating the need) for (possibilities of extensions to existing) data infrastructure for research leveraging learner data, including data repositories, (open source) software and statistical libraries, innovative use of cloud computing resources, etc.

Important Dates

• Submission Deadline: June 24, 2022, 11:59PM (Anywhere on Earth; AoE) June 10, 2022, 11:59PM (Anywhere on Earth; AoE) June 3, 2022

• Acceptance Notification: July 6, 2022

• Workshop: July 27, 2022

Submission Types

The Workshop Committee solicits two types of submissions addressing Questions & Areas of Interest listed above, including both mature research as well as work-in-progress. Papers will be peer-reviewed by members of the Workshop Committee.

• Research & Development papers (4-6 pages): suitable for ~15-20 minute presentations

• Position papers (2-3 pages): describing approaches to convergence research (see below), emerging challenges (e.g., that the LDI might take on collaboratively with authors with future funding), “wishlist(s)” for transformative learning applications, resources like data repositories, and other infrastructure that would fuel innovative work (e.g., that LDI could collaboratively develop with future funding). Shorter presentation timeslots will be allocated for position papers.

We hope that all papers, but especially position papers, will spark conversations and interactions to drive future collaborations between LDI researchers and workshop participants.

Submission Logistics

Use the EDM 2022 Microsoft Word template or LaTeX template. See this page for additional EDM resources.

Submission System (EasyChair): https://easychair.org/conferences/?conf=ldiedm2022

More About the Workshop

The proceedings of the International Conference on Educational Data Mining (and related conferences, including the International Conference on Artificial Intelligence in Education, the ACM Conference on Learning at Scale, and Learning Analytics and Knowledge) demonstrate inherent linkages across traditional and emerging academic disciplines and research areas. Whether efforts are described as interdisciplinary, multidisciplinary, or trans-disciplinary, providing solutions to compelling challenges faced by learners, those individuals and institutions that facilitate learning, and other learning stakeholders must draw on expertise across boundaries of disciplines as diverse as, but not limited to, psychology, cognitive and learning science(s), mathematics, computer science (e.g., machine learning, artificial intelligence), statistics, human-computer interaction, public policy, education, neuroscience, social work, moral and political philosophy, and any of a number of sub-fields and research areas at the intersection of these disciplines.

The need for such multi/inter/trans-disciplinary solutions is even more relevant today as the vast and diverse repositories of digital data available can make such solutions viable. Indeed, recognizing both substantial scientific challenges and the need for innovative scientific frameworks to solve them, the U.S. National Science Foundation has identified the notion of “convergence” research as one of ten “big ideas” for its on-going investment strategy. Two attributes are crucial to NSF’s notion of convergence research, namely that such research is “driven by a specific and compelling problem” and emphasizes “deep integration across disciplines.” Such integration is achieved when:

“... experts from different disciplines pursue common research challenges, [and] their knowledge, theories, methods, data, research communities and languages become increasingly intermingled or integrated. New frameworks, paradigms or even disciplines can form sustained interactions across multiple communities” (Convergence Research at NSF).

The NSF-funded LDI focuses on such science convergence solutions for major challenges in learning with technology.

About the Learner Data Institute

The Learner Data Institute (LDI) is an NSF-funded “data-intensive research in science and engineering” (DIRSE) initiative (NSF Award #1934745) seeking to set out compelling, specific, big data research challenges for educational data science researchers and large-scale scientific and data convergence approaches to address them.

The LDI will help us learn: (1) how to transform a far-flung group of interdisciplinary researchers, developers, and practitioners into a community of practice that can fully exploit the data revolution through data and science convergence; (2) how adaptive instructional systems and data science can be used as research vehicles to further our understanding of how learners learn; (3) to explore the human-technology partnership with data and data science to improve learners’ and teachers’ ability to employ technology in a way that facilitates learning, while at the same time improving the affordability, effectiveness, scalability of these systems; and (4) more generally, how to extend the frontiers of data science to include: new methods of data collection and design; more interpretable machine learning methods (e.g., by combining deep learning with more interpretable inference frameworks like Markov Logic); scalable new algorithms (e.g., for joint inference in Markov Logic Networks); and methods for identifying causal mechanisms from unstructured, semistructured, and structured data.

More specifically, LDI contributors from university-based research groups, industry, and government are focusing on cutting-edge, big data approaches to assessment, learner modeling, instructional design, modeling subject-area domains in instructionally useful ways, socio-cultural aspects of learning, ethical aspects of working with learner data, and the human-technology frontier, among other areas of interest.

Workshop Committee

Vasile Rus, Ph.D., University of Memphis (Co-Chair)

Stephen E. Fancsali, Ph.D., Carnegie Learning, Inc. (Co-Chair)

Dale Bowman, Ph.D., University of Memphis

Jody Cockroft, AA, BS, CCRP, University of Memphis

Art Graesser, Ph.D., University of Memphis

Andrew Hampton, Ph.D., Christian Brothers University

Philip I. Pavlik Jr., Ph.D., University of Memphis

Steven Ritter, Ph.D., Carnegie Learning, Inc.

Deepak Venugopal, Ph.D., University of Memphis

[Ad-hoc reviewers will be drawn from the group of LDI contributors and broader community as necessary.]