Working with Data
Use Data to Make Decisions through Inquiry and Improvement Cycles
Tips and tools to help schools better understand and make decisions based on data.
People and partners in NYC Community Schools have access to all kinds of data, which at times can feel overwhelming. Consider these tips for understanding and acting on data easily:
Inquiry protocols to help people and partners identify trends in the data
Mental models, which include biases and beliefs, and how they influence decision making
Graphic organizers for root cause analysis
Improvement Cycles to spur change over time
Inquiry Protocol: Help People and Partners Identify Trends in the Data
When NYC Community Schools use the New Visions data tools or other tools to explore trends in attendance, behavior, and academic performance, they often apply the Notice and Wondering protocol from Data Wise to spur initial reflections and to guide groups in inquiry.
People and partners first focus on what they notice in the data to ensure team members base reflections on the data, rather than anecdote, and remain objective.
What do we notice?
- What do we notice about the data?
- What trends do we see?
- What subgroups, if any, seem to have different outcomes than others?
o 11th grade boys have the highest chronic absence rate in school.
o 4th grade has experienced an uptick in chronic absenteeism this year.
o 2nd grade ELL students have had increases in behavioral incidents this year.
o 8th grade students with an IEP pass State Math Tests at higher rates than non-IEP students.
Team members begin the “wondering” protocol by asking probing questions about the connections they see in the data. Through this, they identify a problem of practice to solve. For example, the uptick of chronic absenteeism in 4th grade is due to girls missing school on Fridays.
Once schools have identified a problem they wish to address, they explore the possible root causes driving the problem:
What do we wonder?
- What might be the underlying factors contributing to the trends we see?
- What might be the connection between … and …?
- What seems to be one of the key challenges in our school?
Consider Biases and Beliefs
How can people and partners ensure that students who need more support receive it in a culturally responsive way? Are the systems in the school optimized to work for the most vulnerable? What beliefs exist and how do these affect decisions that are made about different types of students?
The iceberg is a tool that helps schools reflect on the underlying patterns and beliefs that produce biases, which may impact the lens through which people and partners analyze data and organize supports for youth.
Tip: Stay “Low” on the Ladder of Inference When Solving Problems of Practice
The Ladder of Inference helps educators and practitioners base opinions on objective data. To stay “low” on the ladder of inference means to focus on observable data and experiences and to not jump to conclusions.
Graphic Organizers for Root Cause Analysis
Graphic organizers are a great way for teams to identify the root cause driving a trend in the data i.e. what is the reason for girls missing school on Fridays, which drives the uptick in chronic absenteeism in the 4th grade?
Graphic Organizer: The Fishbone Diagram
Use the Fishbone Diagram to understand root cause and identify factors contributing to the problem i.e. what is the reason for girls missing school on Fridays, which drives the uptick in chronic absenteeism in the 4th grade?
Record possible causes contributing to the problem in the Fishbone Diagram. Categorize the causes to help identify themes (i.e. teacher actions, parent knowledge, transportation, etc.). Use the diagram to think about the variety of possible factors driving the problem.
FISHBONE DIAGRAM (insert)
The Five Whys
The Five-Whys is a strategy that helps educators and partners brainstorm root causes driving a problem i.e. what is the reason for girls missing school on Fridays, which drives the uptick in chronic absenteeism in the 4th grade?
Asking the question “why” helps educators and partners identify barriers to the desired outcome and make connections between cause and effect.
Five Whys: Step-by-step
1. Identify the problem.
2. Start asking “why” questions related to the problem.
3. Continue asking “why” to further unpack the reasons driving the root cause of the problem.
Improvement Cycles to Spur Change Over Time
Once root causes and underlying factors have been identified, educators and partners in NYC Community Schools choose one driver of the problem to address. Identify an intervention that team hypothesizes will work.
o Team notices in New Visions Attendance Heat Map an uptick in chronic absenteeism in 4th grade.
o Team digs deeper and notices that 4th grade girls are missing school on most Fridays.
o Team uses Fishbone and Five Whys tools to determine causes of these Friday absences.
o Team identifies one cause to focus on: 4th grade girls are avoiding a morning class, and convincing parents to keep them home.
o Team determines strategy to address problem: determine problem with class or schedule and connect with families about the importance of attending school every day.
Plan-Do-Study-Act Cycle (PDSA)
NYC Community Schools use the Plan-Do-Study-Act improvement cycle to refine interventions and find out what solutions work to turn the curve on the trend in data.
§ What changes are to be made i.e. what are the action commitments?
§ What will be different or stay the same over the next four to six week cycle?
§ State the objective
§ State the predictions
§ State the plan (include who, what, where, and when)
§ Identify how data or feedback will be collected
§ Analyze the data collected
§ Compare results to predictions i.e. what actually happened?
§ Summarize results and take-a-ways
§ Carry out the plan
§ Document observations
§ Record data and solicit feedback
Schools and partners use the following PDSA template to track iterations of the intervention they want to refine and monitor over a four to six week period.