The workshop includes panel talks by the organizers, who have experience in data collection tool design. Then, there will be discussion sections to sketch out designs of potential Evidence-Based Iterative (EBI) processes. The EBI process will be able to help LA researchers, practitioners, and other stakeholders update the design of the data collection system in consideration of research contexts and findings from previous iterations. We are planning to write a white paper based on those discussions and publicly share it.

Organizers ourselves will be panels. We are a diverse group of scholar-practitioners in the learning analytics community, and there will be at least 5 different systems including a learning management system (LMS) for the gameful courses and an online learning platform providing feedback for learners and live report for instructors – and challenges in building these systems – described during the workshop. After an introductory presentation on what iterative process is and why we need it, organizers will start a 60 minutes long panel talk. During the panel talk, they will share (a) the design processes and deployments they went through with the tools, (b) the data they capture, and how it might have changed over iterative development/deployment, and (c) potential theoretical lenses which they have associated with their tools and how those lenses have changed through the design and deployment process. All participants will be encouraged to participate in the panel via questions addressed to the organizers.

Here is a list of panels and the moderator!

• Heeryung Choi, University of Michigan (moderator)

• Christopher Brooks, University of Michigan

• Caitlin Hayward, University of Michigan

• Neil Heffernan, Worcester Polytechnic Institute

• Dragan Gasevic, Monash University.

• Kirsty Kitto, University of Technology Sydney

• Abelardo Pardo, University of South Australia

• Phil Winne, Simon Fraser University

Sure! The second section of the workshop will be a 30 minute discussion between participants, where you will form a group of 4 to 5 around specific data collection challenges chosen from the responses to the survey. Participants will (a) share own struggle or how they managed to overcome such struggle during data collection or data analysis and (b) find similar and different components of these experiences. The goal of this discussion is to understand similarities and differences in approaches and what caused those and to identify the common top three challenges of your experiences.

The third section of the workshop will be another 30 minute group discussion between participants. The goal of this second discussion is to sketch out frameworks (e.g., iterative research workflow) of an iterative process which could work as a potential solution. Participants will be encouraged to think about following questions during the sketch-out: What is identified as a key issue causing the failure or difficulty of ideal data collection? What is the context of the data collection and data analysis they would like to focus on? How can an iterative design process solve the issue? How would researchers know if the designed framework is working? The workshop will be concluded with participants’ presentation of their framework design.

A. Participants should apply to the workshop by submitting ½ to 1-page write-up on their previous struggle of data collection to the Google Form below. As long as you submit a meaningful response, you will be able to participate in the workshop! (Don't forget to sign in to your Google account to see the form!)