Progress Monitoring
Ed Direction Data Fellows: Asynchronous Module
April 29, 2022
Embedded Files
Welcome to the Progress Monitoring Optional Asynchronous Module.
Helpful Instructions and Reminders:
You will work through this module by scrolling through this learning space. To expand documents and slide decks that are included, you can click on the gray arrow at the top right corner of each item.
Feel free to focus on the pieces of this module that are most relevant to your topics of interest.
Please complete the Exit Ticket and Module Completion Form at the end of the module. We will use your submission to track completion.
Please contact datafellows@eddirection.org if help is needed.
Click on the button to the left to open a Note Catcher, which is mirrored to follow the content as it is presented on the Learning Space. As you navigate through this module, you are welcome to use this optional tool to capture your notes.
Refer to your note catcher each time you see this icon.
Session Outcome: This optional asynchronous module will provide additional information and resources to support progress monitoring of goals set by the RSSP team. More specifically, this module will walk through the components of a measurement plan and explore how data dashboards are used in progress monitoring.
Success Criteria: Participants will be able to apply their understanding of the following as they serve as Data Fellows:
How to effectively set up a measurement plan linked to their LEA’s SMART goals
The primary types of data and how each can be used in progress monitoring both student performance and implementation efforts
The role of data dashboards in progress monitoring
As with all Data Fellows training modules, this module contains elements that allow you to see practices in action, name the components that contribute to the effectiveness of the practices, and do a task that brings the practices to life.
Where We’ve Been, Where We’re Headed
Over the last 2 months, we’ve discussed what it means for districts to be data literate and have a strong data culture. We have also started to unpack how to set SMART goals and monitor progress of both student performance and implementation. Finally, we’ve discussed how to design a data dashboard and how to select a data platform.
This module contains both new information as well as refreshers on content previously introduced to the cohort. This content is designed to recap prior learning while also providing a new layer of learning with additional resources to support you in your role.
NAME IT: Progress Monitoring
NAME IT: Progress Monitoring
Monitoring student performance and implementation helps insure the efforts your district is taking to address your priority strategies and initiatives are having the desired impact. When we fail to monitor progress, we run the risk of getting to the “finish line” and realizing that all our hard work has not resulted in real, lasting change.
The goal of progress monitoring is to increase student outcomes by making informed decisions based in real-time data. Progress monitoring should occur at the classroom, school, and LEA level. As a Data Fellow, you will support your district’s RSSP team in ensuring the “right” data are collected and accessible to stakeholders so that progress can be monitored at regular intervals.
Pause and Reflect: On a scale of 1-10, how would you rate your district’s current approach to progress monitoring at the classroom, school, and district level? What supports are in place to support progress monitoring? What necessary structures and/or resources are missing?
NAME IT: Measurement Plans
NAME IT: Measurement Plans
A measurement plan provides the roadmap for measuring progress towards your SMART goals and ensures that you have an actionable use for every data source you collect.
A well written measurement plan articulates what data needs to be collected, when, and by whom.
In previous RSSP cycles, districts learned that undefined measurement plans made efficient and effective improvement reviews difficult. As a result, LEAs are focusing on creating goals and measurement plans that include assessments to provide data during improvement cycles (as opposed to only EOY information) and that outline an assessment schedule to provide sufficient time for collecting, cleaning, and analyzing relevant data. (More here.)
Source: TEA
TEA guidance in 1E points out the following about goals and measurement:
Goals and measures need to be set for each of the district’s aligned strategies and initiatives under each priority.
Student outcome goals should include at least one prioritized student group.
Effective measurement plans provide the roadmap for measuring progress towards your SMART goals and ensure that you have an actionable use for every data source you collect. They can serve as the starting point for data dashboard design. With a clear picture of what needs to be assessed and by when, the Data Fellow can move from a measurement plan to building an effective data dashboard.
Pause and Reflect: What in this section has confirmed what you know about measurement plans? What surprised you or added to your understanding of measurement plans?
Components of an Effective Measurement Plan
Effective measurement plans contain four main components. They capture SMART goals, identify which assessments will be used, specify when assessments will happen, and identify who will be assessed.
In addition to naming these components, the measurement plan should include a data calendar to ensure ample time for data collection, data cleaning, and data analysis for each assessment. This calendar should be created by backwards mapping milestones from assessment administration or data collection dates. Your district data calendar should be reviewed regularly and updated as needed to reflect assessments/data that are being used for progress monitoring. Keep in mind that developing (and updating) a data calendar should be done in consultation with all key staff members who either produce, collect, or use the data.
The Role of the Data Fellow
To ensure the measurement plan comes to life, the Data Fellow will:
Support teams in drafting SMART goals.
Ensure there is a Data Calendar that clearly outlines when assessments take place, when data will be available, time to clean and organize the data, and time to analyze the results.
Work with their team and 1a TAP to identify who will be assessed and at what level.
Pause and Reflect: In which of these roles do you feel most confident as a Data Fellow? For which might you need more support? What additional questions do you have?
Useful Resources
Goals and Measurement Plan Template
This is a template to help you think through your measurement plan to closely monitor SMART goals. Adapted from TEA.
1.B Student Outcome Data Analysis for Priortization.pdfStudent Outcomes Data Analysis for Prioritization
These slides provide an overview of how to use data insights to prioritize progress monitoring.
Things to Consider:
Be sure to consult your Assessment Inventory when building out the measurement plan.
When creating a data calendar, backwards plan from the time of the assessment and/or data collection. Don’t underestimate the amount of time it takes to clean data. The calendar must allow adequate time for the data to be cleaned and prepped.
Take time to explore how data sets are linked and how students are rostered (this will help with the eventual setup of a data dashboard by the Data Fellow).
The Measurement Plan will directly inform how the Data Fellow structures the LEA’s Data Dashboard. Any data that is being collected and used to monitor progress should be accessible via the Dashboard.
To learn more about setting measurement plans and updating data calendars, read through this resource shared in the March Asynchronous Module.
DO IT: Explore Resources
DO IT: Explore Resources
Explore these resources that you may choose to use in your role as a Data Fellow. As you review the information and tools, consider how each might be most useful in your district. Also make note of any changes you might need to make to ensure the tool/information is relevant for your local context.
SEE IT: Data Sources Deep Dive
SEE IT: Data Sources Deep Dive
The Lost Pines RSSP team sat down to review student performance data three weeks into their first cycle of improvement. As they unpacked the data, they quickly identified that the priority actions they chose to implement were not having the desired impact on student learning. Very few student groups were showing gains, and some were even showing a decline. As the team discussed potential reasons for this, they came to a standstill. While some had hypotheses about why the data looked like they did, the team had no evidence to confirm or contradict these hypotheses.
After some consideration, the team identified a hole in their measurement plan. While they planned to collect implementation data on both capacity and fidelity of implementation of the priority practice to address their district need, the first data collection point wasn’t for two more months. They quickly realized they needed to adjust the timeline and collect implementation data immediately so they could have a data-driven conversation about why they were not seeing more positive results in performance data.
NAME IT: Performance vs. Implementation Data
NAME IT: Performance vs. Implementation Data
Effective Measurement Plans contain:
Student Performance Measures
Implementation Measures (Capacity + Fidelity)
Pause and Reflect: What shifts in their measurement plan did Lost Pines determine they needed to make? Does your current measurement plan account for both student performance and implementation data?
To learn more about student performance data and implementation data, review through this resource shared in the March Asynchronous module.
DO IT: Consider Your Data Sources
DO IT: Consider Your Data Sources
Consider your district’s current measurement plan. Make note of additional data sources you could use to measure progress. Consider both student performance data and implementation data.
NAME IT: Lead and Lag Measures
NAME IT: Lead and Lag Measures
Effective Measurement Plans monitor both lead and lag measures. We use both lead and lag measures to monitor the goals we set, regardless of what it is we want to achieve. Read more about how lead and lag assessment measures are used differently in the table below.
When we set goals, including our overarching goals (Wildly Important Goals, or WIGs), we use both lead and lag measures to evaluate our success. While lag measures may help us determine whether we were successful overall, lead measures allow us to monitor our progress, course-correct, and refine our approaches in real time. Using lead data is imperative to school improvement because it allows us to make more strategic and less reactive decisions about services and supports to improve student learning (Foley et al. 2008).
While it’s easy to define what your lag measure will be (often our lag measures are our end of year assessments), careful consideration must go into determining what appropriate lead measures are for monitoring progress towards your SMART goal.
We use four categories to identify lead measures for our priority goals:
Process
Capacity
Fidelity
Student Outcomes
With each category, we consider questions that can help us determine which lead measures to use to propel us towards achieving our goals.
Process: Are improvement and implementation actions being done or adjusted as necessary to continuously get closer to bringing priorities to life?
Capacity: Are the appropriate supports being provided to build the knowledge, mindsets, and skills for those implementing to succeed?
Fidelity: Is implementation of the strategies and initiatives improving over time?
Student Outcomes: Is student learning accelerating as demonstrated through student assessment growth?
We can ask the above questions to strategically plan with our end goal in mind. Through considering process, capacity, fidelity, and student outcomes, we have a road map of the data we may need to collect periodically (including baseline data) to monitor our progress. See the table below for an example in selecting relevant Lead measures related to implementing high dosage math tutoring for economically disadvantaged middle schoolers.
Adapted from TEA.
Pause and Reflect:
Why shouldn’t we just analyze student performance Lead measures if our goal relates to student performance outcomes?
What are some hazards of not collecting enough progress monitoring data? Are there any hazards to collecting too much progress monitoring data?
Read more about lead and lag measures in this document from AchieveIt.
See examples of lead and lag measures in practice here.
NAME IT: Using Measurement Plans to Design a Data Dashboard
NAME IT: Using Measurement Plans to Design a Data Dashboard
When you have a strong, comprehensive measurement plan, you have already done a lot of the deep thinking required to create a clear and functional data dashboard. Everything that is needed to build your dashboard is outlined in your measurement plan!
Some things to consider:
Lag measures (related to your “Wildly Important Goal” or “S.M.A.R.T. Goal”) aren’t dynamic on a dashboard - they’re reported infrequently, perhaps comparing data yearly.
Lead measures (related to implementation and short- or medium-term student performance) will need to be collected more frequently, and your dashboard should show comparisons from each of these data points.
Pause and Reflect:
Lag measures are the end of cycle or end of year measure. They are typically reported out once and are shown in comparison to previous year(s) data.
What implications does this have for your data dashboard?
How might you visualize your lag measures?
Lead measures will typically show comparisons over time and are collected at multiple times throughout the year.
What implications does this have for your data dashboard?
How might you visualize your lead measures?
DO IT: Wireframe a Dashboard
DO IT: Wireframe a Dashboard
Step 1: Review the completed measurement plan with lead measures. Write a lag measure to monitor the stated goal (HINT: Check out the WIG).
Adapted from TEA.
Step 2: Use the Dashboard Design Template to wireframe a data dashboard that will facilitate progress monitoring. Be sure to think through which visuals will lend themselves to displaying the data.
Step 3: Sketch out a dashboard with the visuals you chose. Consider where the less dynamic lag data will live and where the lead measures will be displayed with consistent updates. Don’t get bogged down in making it look pretty - just simple lines, boxes, and circles will do!
Module Complete!
Module Complete!
Congratulations on completing the Progress Monitoring module. Please complete the Exit Ticket and Module Completion forms by clicking on the links below. We will use the information you submit to track your completion.
Page updated
Report abuse