myImpact Trials Process

The myImpact trials process resemble action-research inquiries based on Plan-Do-Study-Act (PDSA) cycles of continuous improvement to design, deliver, and assess learning. During a trial, a team of teachers chooses a learning design or practice focus that they hypothesize will make a difference in student engagement and academic learning. 

Teachers collect three observations of each criterion in both the engagement and academic rubrics in the baseline data collection period and each of the trial periods.

Baseline Data Collection

The first data collection event is a baseline data collection period which establishes a gauge for comparing trial data. Teachers collect observational data on the preset engagement and teacher-developed academic criteria that will be used throughout the trials process. Teachers do not change the learning design or their instructional practices for this baseline data collection period. The baseline data collection period is usually two or three weeks long to allow teachers to collect three observations of each criterion in both the engagement and academic rubrics.

The myImpact trials process supports the use of two high-leverage instructional strategies: clear learning goals and student self-assessment with the clear learning goal. If these practices are already in place before the trials process is started, then teachers maintain those practices during the baseline data collection period. Teachers note the clear learning goal and student self-assessment tool during data recording into the Student Learning Goal and Student Self-Assessment fields on the Data Collection page.

If these two practices are not already in place, teachers are not expected to enter any information into the Student Learning Goal, Student Self-Assessment, or Strategy Tried fields on the Data Collection page.

Trial 1

The myImpact trials process supports the use of clear student learning goals by creating a Student Learning Goal field in the Observation block where teachers input their classroom observations. If clear student learning goals are not part of the team’s daily practice, we highly suggest that clear student learning goals becomes the focus for Trial 1.

The trial collection period is usually three weeks long. The focus learning design or practice should be included into each day’s lesson. The first week, teachers should include the learning design or instructional practice in lessons without collecting observations. This period is meant to allow teachers and students to become somewhat acclimated to something new. Teachers then collect three observations of each criterion in both the engagement and academic rubrics over the following two weeks while still including the learning design or instructional practice in daily lessons.

Trial 2

The myImpact trials process supports the use of student self-assessment with clear student learning goals by creating a field for Student Self-Assessment in the Observation block where teachers input their classroom observations. If student self-assessmentis not part of the team’s daily practice, we highly suggest that student self-assessment with clear student learning goals becomes the focus for the Trial 2.

The trial collection period is usually three weeks long. Teachers continue the learning design or instructional practice that was the focus of Trial 1 if teachers determined that it had a positive effect on student engagement and/or academic progress. 

The Trial 2 focus learning design or practice should be included into each day’s lesson. The first week, teachers should include the learning design or instructional practice in lessons without collecting observations. This period is meant to allow teachers and students to become somewhat acclimated to something new. Teachers then collect three observations of each criterion in both the engagement and academic rubrics over the following two weeks while still including the learning design or instructional practice in daily lessons.


Trial 3 or any trial after student learning goals and student self-assessment are already in place

The Trial 3 collection period is usually three weeks long. Teachers continue the learning design or instructional practice that was the focus of Trials 1 and 2 if teachers determined that it had a positive effect on student engagement and/or academic progress. 

The Trial 3 focus learning design or practice should be included into each day’s lesson. The first week, teachers should include the learning design or instructional practice in lessons without collecting observations. This period is meant to allow teachers and students to become somewhat acclimated to something new. Teachers then collect three observations of each criterion in both the engagement and academic rubrics over the following two weeks while still including the learning design or instructional practice in daily lessons.

Commonly Asked Questions

How do we know we are rating our indicator students similarly? (inter-rater reliability)

The data being collected does not have the precision of a measure, like a ruler or a thermometer. You are collecting indications that will be assumed to be connected to changes in learning design. Each teacher knows their students and how students respond. You are using your professional observation skills to maintain intra-rater reliability, meaning the degree of observation consistency by a single teacher. 

When you analyze the data you will be looking at the movement of observations from combined rubric levels 1 and 2 to combined rubric levels 3 and 4.

If it is helpful, the team can have another discussion getting a sense of what each engagement and academic criterion looks like in each of the team member's classrooms. All team members can come to some common understandings of the observable features of each criterion collected as data.

I didn’t know what to enter into the Student Learning Goal, Student Self-Assessment, or Strategy Tried columns of the Data Collection fields. 

Those fields remain blank for the baseline data collection observations if those are not aspects of a teacher’s normal designed learning or instructional practices. The myImpact trials process emphasizes Clear Learning Goals and Students Self-Assessment as intended focuses for Trials 1 fand 2 because those two practices are fundamental to students developing as expert learners.

Do I use a new row or use the same row to enter data for a single student?

Use a new row for each day that you collect observations. 

If I didn’t make an observation, should I give a 1 level rubric score? 

In the event that you were not able to observe a particular criterion on a certain day, please choose “Not observed” from the dropdown menu in that criterion field. 

One of the criteria we chose isn’t observable. 

Option 1: If the team has four criteria chosen, just drop the criterion that is not observable for the trials. Collect data on the remaining three criteria for the remainder of the trials.

Option 2 (use this option only during baseline data reflection): If there is enough time to collect additional baseline data, revise the criterion into an observable statement. Collect baseline data on that criteria before the Trial 1 data collection window begins. 

I didn’t have the opportunity to collect criteria observations.

It is important to design lessons to provide yourself the explicit opportunity to collect observations on the agreed-upon criteria. That may mean planning to collect data during particular lessons or making slight modifications to lessons so they include the opportunity to collect observations on criteria.

You may find collecting all engagement and/or academic criteria during one lesson difficult. The team can agree that we can split observations across different lessons. For example, I might be able to collect “Initiates and completes instructional tasks” on a Monday, but won’t collect “Participates in discussions” until the Wednesday lesson.

Some of my students are not at school every day or have dropped my class. 

Option 1: Reducing the number of indicator students is an option if there is not enough time to choose new students to observe. Strive to collect as much data across as many indicator students as possible.

Option 2: If there is enough time to collect baseline data on new students, identify other students to observe.

I couldn’t collect three observations in each of the criteria. 

Our goal is to collect three or more observations across all the criteria during each trial. That may not be feasible for each criterion in each trial. Try your best to collect as much as possible, balancing both engagement and academic observations.