Embedded Files
Welcome to the "Structuring Data for Visualizations" Optional Asynchronous Module.
You will work through this module by scrolling through this learning space. To expand documents and slide decks that are included, you can click on the gray arrow at the top right corner of each item.
Feel free to focus on the pieces of this module that are most relevant to your topics of interest.
Please complete the Exit Ticket at the end of the module. We will use your submission to track completion.
Please contact datafellows@eddirection.org if help is needed.
Click on the button to the left to open the note-catcher, which is mirrored to follow the content as it is presented on the Learning Space. As you navigate through this module, you are welcome to use this optional tool to capture your notes.
Session Outcome: This optional asynchronous module will focus on how to best collect RSSP data in order to facilitate high quality dashboard creation.
Success Criteria: More specifically, Data Fellows will be able to:
Identify best practices in designing implementation data collection tools with their RSSP teams
Understand the importance of focusing on the "backend" during data collection
Create compelling dashboard visualizations that utilize both implementation and performance data
Anticipated time to complete this module: 90 minutes
SEE IT: Case Studies
SEE IT: Case Studies
Below are case studies detailing the actions taken by two RSSP teams as they attempt to collect the optimal data for their RSSP dashboards. As you read the case studies, carefully consider:
Which district is set up to have a more efficient data cycle & better analyses?
What are the key moves that district made that set it up for success?
What can your district do to improve the "backend" of its RSSP data processes?
Case Study #1: Freemont ISD
Case Study #1: Freemont ISD
A Strong Start
Freemont ISD is a large suburban district in Northern Texas that serves over 18,000 students across 20 campuses. The RSSP team has set a Wildly Important Goal to increase the percentage of 9th grade students who are designated as "Approaches" on the STAAR math assessment from 63% to 68%, and they have chosen High Quality Instructional Materials as their intervention strategy.
Throughout the summer, the RSSP team collaborated on which data, if collected, would provide them with the most accurate forecast for their progress towards achieving their wildly important goal. In consultation with their superintendent, they determined that a combination of implementation and performance data collected via walkthrough forms, teacher and student surveys, and common 8th grade math assessments would give them the best possible information to help them determine if they were on track to accomplish their WIG.
Delegating Tasks
After determining the data they would need and the tools required to collect the data, they delegated the creation of each tool to a different member of the team to complete asynchronously. One member was charged with drafting the survey for teachers, another for creating the student survey/exit-ticket, and another with creating the walkthrough form. Luckily, common assessments were embedded within the HQIM selected by the district, so there was no need to task a committee member to coordinate common assessments among teachers.
As the different data collection tools were being designed, the RSSP team tasked the Data Fellow with creating a dashboard that would use the performance and implementation data to display:
Prior year STAAR math scores by teacher and by student subgroup
Current unit assessment scores by teacher and student subgroups
Walkthrough Form summaries by teacher and campus
Teacher survey summaries by campus
The relationship between implementation measures and student unit assessment scores
The Data Fellow selected Google Data Studio as his visualization platform and began creating a dashboard wireframe.
A Rocky Finish
As the school year began, the RSSP team worked with principals and teachers to implement the selected HQIM and the other data collection tools they had created to track fidelity.
As the Data Fellow prepared for the Mid-Cycle Stepback in October, he started collecting all the different sets of RSSP data that were being generated and worked to incorporate them into his dashboard. However, when he loaded the different data sets into Google Data Studio, he had significant trouble blending them. Frustrated by the setback, he instead decided to load each data set individually and use them to create separate visualizations. This strategy seemed to work initially, and the Data Fellow was able to create visualizations that summarized the data sets individually. However, he had significant trouble when attempting to create visualizations that displayed the relationship between measures from different datasets. Because the datasets were not blended, it was impossible to create visualizations that displayed the relationship between implementation and performance measures.
With the Mid-Cycle Stepback approaching quickly, the Data Fellow was under considerable stress and did not know what to do.
Case Study #2: Fairfield ISD
Case Study #2: Fairfield ISD
A Strong Start
Fairfield ISD is a small rural district in Northern Texas that serves approximately 800 students across 4 campuses. The RSSP team has set a Wildly Important Goal to improve the number of 8th grade students who are designated as "Approaches" on STAAR reading from 76% to 85% and they have chosen High Quality Instructional Materials as their intervention strategy.
Throughout the summer, the RSSP team collaborated on which data, if collected, would provide them with the most accurate forecast for if they were on track achieve their Wildly Important Goal. In consultation with their superintendent and Data Fellow, they determined that a combination of implementation and performance data collected via walkthrough forms, teacher surveys, student exit tickets, and common ELA district assessments would give them the best possible information to help them determine if they were on track to accomplish their WIG.
Creating Wireframes
In order to track their progress, the RSSP team requested that the Data Fellow build a dashboard that displayed the following:
Prior year STAAR reading scores by student subgroup
Current unit assessment scores by student subgroups
Walkthrough Form summaries by campus
Teacher survey summaries by campus
The relationship between implementation measures and student unit assessment scores
Before discussing how each of the data collection tools would be created, the Data Fellow worked with the RSSP team to create a data set wireframe so that the entire team could develop a shared understanding of what they ultimately wanted their data set to look like. The team decided that they wanted to create one “master” spreadsheet that broke down all of the data they were collecting at the student level. They reasoned that if they wanted to be able to see the relationships on their dashboard between their implementation efforts and student outcomes, they would need to have the backend data connected. They wanted to be able to see how student performance outcomes were influenced by implementation measures.
This meant that for every student listed in the spreadsheet, they needed to have the corresponding teacher’s walkthrough scores and survey results, along with the corresponding principal’s survey results on the same row. Furthermore, if they wanted to be able to break down student performance by student characteristics, they would also need to ensure that their spreadsheet contained the appropriate student characteristic data on each row for every student ID.
Focusing on the Backend
As they were designing their master spreadsheet, the Data Fellow flagged the difficulty of creating such a comprehensive spreadsheet from so many different data sources. Up to this point, not much thought had been invested in understanding the process by which data would be taken from the separate sets of data generated from the tests, forms, and surveys and loaded into a single spreadsheet. After a series of internal discussions, the team began brainstorming how they could make the backend data organization process as efficient as possible for the Data Fellow. This led the team to, where possible, create tools and processes that automatically put data into a Microsoft or Google spreadsheet, enabling the Data Fellow to copy and paste values collected by the tools directly into the master spreadsheet.
Beginning with the backend data in mind, the RSSP Team worked together to design the teacher walkthrough form, the teacher survey, and the student survey/exit ticket. They also determined that they would use the common assessments embedded within the HQIM to collect data for student performance outcomes. A key challenge that emerged with this decision was that the HQIM assessments were to be administered on paper and graded by hand. To overcome this problem, the RSSP team committed to working with principals to instruct teachers to record their assessment data within Google Sheets. Teachers would then email their student assessment scores to the campus administrative assistant, who would then compile the test scores by teacher name in one email and send them to the Data Fellow.
Being Proactive
As the Data Fellow searched for a list that contained the information of student demographics and characteristics, he discovered that no such list existed. Instead, that information was separately embedded in student individual profiles, and school counselors and administration were the only people who had access to view and edit the profiles. In order to organize this information into a usable data set, the Data Fellow collaborated with the counseling departments of each campus. They collectively determined that the most effective way to create the data set would be for the Data Fellow to send each counseling department the list of student ID’s in a Google Sheet for which he needed demographic/characteristic information, and each department would then independently add the requested information to the Google Sheet.
Everything Falls into Place
As the school year began and RSSP data started rolling in, the Data Fellow worked hard to combine the data from all of the different sources into one master spreadsheet. This was a painstaking process that took a lot of time. After obtaining and adding the necessary lists and reports to the master spreadsheet, the Data Fellow began using the HQIM assessment score emails from the campus administrative assistants to add the teacher names and common assessment scores to the spreadsheet. After teacher names and common assessment scores were entered, the Data Fellow moved on to copying and pasting the information collected from the teacher surveys, walkthrough forms, and the student surveys/exit-tickets into the spreadsheet.
The process for combining all the data from different sources into one spreadsheet took an unfathomable amount of time. However, once it was complete, the Data Fellow had created an extremely robust data set that could create a dashboard capable of providing invaluable insights to the RSSP team.
With the hard work completed, the Data Fellow loaded his spreadsheet into Google Data Studio and began creating the RSSP dashboard. Because the dataset effectively combined all the different data sources, creating the visuals requested by the RSSP team was fairly simple. After a couple days, the Data Fellow had designed a strong Data Dashboard that could act as an effective foundation for discussions during the Mid-Cycle Stepback.
NAME IT: Fairfield ISD Materials
NAME IT: Fairfield ISD Materials
In this section, you can examine all of the different data collection tools that were used by Fairfield ISD, the data sets generated by those tools, the master data set created from the smaller data sets, and the final dashboard the Data Fellow created. You will additionally be shown how to create the key visualizations displayed in the RSSP dashboard.
Attendance Report
Attendance Report
The Data Fellow used attendance reports to obtain student ID's
Prior Year STAAR ELA
Prior Year STAAR ELA
The Data Fellow received access to the prior year STAAR scores for the RSSP focus students
Master Data Set
Master Data Set
This is the comprehensive spreadsheet that includes all of the cleaned raw data that the RSSP team collected. Because all RSSP implementation data and student performance data are contained in the same spreadsheet, the Data Fellow is able to create robust visualizations in his dashboard.
Fairfield ISD Dashboard
Dashboard Discussion
During the Mid-Cycle Stepback, the two largest take-aways that emerged from the data discussion were as follows:
Across almost all measures, Springview Jr. High outperforms Canyon Jr. High.
This finding should lead Fairfield ISD's RSSP team to discuss why Springview Jr. High is performing so well (relatively) on both implementation and performance measures. The RSSP team should also have meaningful conversations around why Canyon Jr. High consistently performs lower. In order to fully understand what is happening at each of these schools, qualitative tools like interviews and focus groups should be utilized.
Teacher Self-Expressed Implementation Fidelity is the best predictor of student performance on common assessments.
This intuitively makes sense -- the teachers who do the best at implementing the curriculum have the students who do best on assessments that are aligned to the curriculum. What makes this so interesting is that the other methods of measuring implementation fidelity -- walkthrough forms and student surveys -- were less accurate. This insight should spur discussions within the RSSP team around why teacher self-expression is so accurate and what the team can do to enhance the validity of walkthrough forms.
Visualization Descriptions
Below are instructions on how to create the visualizations found in Fairfield's Dashboard.
Average 2021-2022 STAAR Scaled Scores by Student Characteristic
Average 2021-2022 STAAR Scaled Scores by Student Characteristic
This visualization was created by selecting "Campus" as the dimension, "Characteristic 1" as the breakdown dimension, and "Prior year STAAR Score" as the metric. Note: Ensure that the visualization is displaying the average value. This can be done by selecting the symbol immediately left to "Prior year STAAR Score" and selecting the "AVG" option.
How to Create the "Approaches" Line
How to Create the "Approaches" Line
When editing your visualization, select the word "Style" on the top right of the screen and scroll down until you see the words "Add a reference line". Select the blue + sign and manually enter the value at which you want the line to appear. Using Google, you can find out what the Scaled Score for "Approaches" is for your specific test year. In our case, the scaled score value for Approaches for the year these data come from is 1587. We then label the reference line "Approaches", select the box "Show Label", and choose our desired line width.
Average 2021-2022 STAAR Scaled Scores by Student Characteristic
Average 2021-2022 STAAR Scaled Scores by Student Characteristic
This visualization displays similar information to the one above, except that the breakdown dimension has been changed to "Ethnicity", allowing us to see the STAAR scores breakdown by ethnicity instead of student characteristic.
Common Assessment Scores
Common Assessment Scores
This visualization was created by selecting "Campus" as the dimension, "Characteristic" as the breakdown dimension, "Common Assessment 3 Score" as the metric, and sorting the data by "Common Assessment 3 Score".
Walkthrough Form Summary
Walkthrough Form Summary
This visualization was created by selecting "Campus" as the dimension and "Teacher Walkthrough Total Score" as the metric.
Teacher Survey + Common Assessment 3 Score Scatterplot
Teacher Survey + Common Assessment 3 Score Scatterplot
This visualization was created by selecting the scatterplot option under the "Add a chart" selector. Then "Teacher" was selected by the dimension, "Common Assessment 3 Score" as the metric for the X axis, and "Teacher Self-Expressed Implementation Fidelity for the Y axis.
DO IT: Build Your Own Visualizations
DO IT: Build Your Own Visualizations
The dataset below is similar in structure to Fairfield's master data set. Many of the values, however, have been changed. Use this adapted data set to create:
A scatterplot that demonstrates the relationship between student self-interest in the curriculum and their common assessment 3 score
A scatterplot that demonstrates the relationship between walkthrough form total and common assessment 3 scores
A scatterplot that demonstrates the relationship between teacher self-expressed curriculum implementation fidelity and common assessment 3 scores
A bar chart that summarizes common assessment 3 scores by campus, broken down by student characteristics
A bar chart that summarizes common assessment 3 scores by campus, broken down by student ethnicity
Module Complete!
Module Complete!
Congratulations on completing the module. Please complete the Exit Ticket form by clicking on the link above. We will use the information you submit to track your completion.
Page updated
Report abuse