Data Inquiry & Analysis

Use Data to Make Decisions through Inquiry and Improvement Cycles

Tips and tools to help schools better understand and make decisions based on data.

People and partners in NYC Community Schools have access to all kinds of data, which at times can feel overwhelming.

Consider these tips for understanding and acting on data easily:

Inquiry protocols to help people and partners identify trends in the data

Mental models which include biases and beliefs, and how they influence decision making

Graphic organizers for root cause analysis

Improvement Cycles to spur change over time

Inquiry Protocol: Help People and Partners Identify Trends in the Data

NYC Community Schools use the New Visions Data Tools to explore trends in attendance, behavior, and academic performance and apply the Notice and Wondering protocol from Data Wise to spur initial reflections and to guide groups in inquiry.

Notice

People and partners first focus on what they notice in the data to ensure team members base reflections on the data, rather than anecdote, and remain objective.

What do we notice?

      • What do we notice about the data?
      • What trends do we see?
      • What subgroups, if any, seem to have different outcomes than others?

Examples:

o 11th grade boys have the highest chronic absence rate in school.

o 2nd grade ELL students have had increases in behavioral incidents this year.

o 8th grade students with an IEP pass State Math Tests at higher rates than non-IEP students.

Wondering

Team members begin the “wondering” protocol by asking probing questions about the connections they see in the data. Through this, they identify a problem of practice to solve.

Once schools have identified a problem they wish to address, they explore the possible root causes driving the problem:

What do we wonder?

      • What might be the underlying factors contributing to the trends we see?
      • What might be the connection between … and …?
      • What seems to be one of the key challenges in our school?

Mental Models

Consider Biases and Beliefs

    • How can people and partners ensure that students who need more support receive it in a culturally responsive way?
    • Are the systems in the school optimized to work for the most vulnerable?
    • What beliefs exist and how do these affect decisions that are made about different types of students?

The iceberg is a tool that helps schools reflect on the underlying patterns and beliefs that produce biases, which may impact the lens through which people and partners analyze data and organize supports for youth.

The Ladder of Inference helps educators and practitioners base opinions on objective data. To stay “low” on the ladder of inference means to focus on observable data and experiences and to not jump to conclusions.

Tip: Stay “Low” on the Ladder of Inference When Solving Problems of Practice


Graphic Organizers for Root Cause Analysis

Graphic organizers are a great way to identify

root causes behind the patterns you see.

Two Graphic Organizers

                    • The Five Whys
                    • Fishbone Diagram

Use the Fishbone Diagram to understand root cause and identify possible factors contributing to the problem.

Record possible causes contributing to the problem in the diagram. Categorize the causes to help identify themes (i.e. teacher actions, parent knowledge, transportation, etc.). Find the template for the Fishbone Diagram here.

The Five-Whys is a strategy that helps educators and partners brainstorm root causes driving a problem i.e. What is the reason for the uptick in chronic absenteeism in the 4th grade?

Asking the question “why” helps educators and partners identify barriers to the desired outcome and make connections between cause and effect.

Improvement Cycles to Spur Change Over Time

Once root causes and underlying factors have been identified, educators and partners in NYC Community Schools choose one driver of the problem to address. Identify an intervention the team hypothesizes will work.

Example:

o Team notices an uptick in chronic absenteeism in 4th grade in Attendance Heat Map.

o Team digs deeper and notices that 4th grade girls are missing school on most Fridays.

o Team uses Fishbone and Five Whys tools to determine causes of these Friday absences.

o Team identifies one cause to focus on: 4th grade girls are avoiding a morning class, and convincing parents to keep them home.

o Team determines strategy: Find out more about problem with class and connect with families about the importance of attending school every day.

Plan-Do-Study-Act Cycle (PDSA)

NYC Community Schools use the Plan-Do-Study-Act improvement cycle to refine interventions and find out what solutions work to turn the curve on the trend in data. Schools and partners use the following PDSA template to track iterations of the intervention they want to refine and monitor over a four to six week period.