Embedded Files
Introduction
By this point in your Data Fellows journey, you have probably crafted a few data stories. You have collected data from a variety of sources, discerned trends through analysis, and used those trends to create a narrative about what stakeholders can do to positively alter student learning trajectories.
While every system story has its unique nuances, there are a few core "plots" that drive most of problems we experience in our districts. These common plots are known as system archetypes. Through the pioneering work of scholars like Jay Forrester, Peter Senge, and Donella Meadows, researchers have identified the structures of universal problems that drive much of the dysfunction we see.
In this module, we will introduce you to the five system archetypes that are most relevant to your work as a Data Fellow. By developing an understanding of what these system archetypes look like, you will be in a better position to identify them within your districts. Instead of crafting your data stories from scratch (so to speak), you will be able to use these archetypes as references and guides. Pairing system archetypes with the data you collect is a powerful strategy that can generate deep insights into the intransigent problems your district is experiencing. It will also help you identify the most impactful levers of change that, if effectively targeted, can cultivate enduring positive change for teachers and students.
Module Overview:
In the sections below, we will present each archetype by first describing its dynamics -- how stakeholder actions come together to perpetuate the problem pattern. We will then provide an example of what the archetype looks like in an RSSP setting. After providing an RSSP example, we will describe how you can use data to identify and target the key leverage points within each archetype.
Much of the content in this module has been adapted from Daniel Kim's article System Archetypes I.
Drifting Goals
Dynamics
Drifting Goals describes the all-too-familiar scenario of setting an ambitious goal, only to watch it diminish in rigor over time. The process typically follows the subsequent steps:
We set an ambitious goal to improve outcomes (change the status quo)
We implement interventions designed to produce improvement according to our goal
There are long delays between intervention implementation and realized outcomes
The delay in achieving outcomes weakens intervention support and increases pressure to reduce or eliminate the goal
We reduce or eliminate the goal (and subsequent interventions) according to the pressure
The weakened goal and interventions fail to produce improved outcomes
We remain at the status quo
Example
An RSSP team sets a SMART goal to improve student proficiency by 5% by the end of the school year. In order to accomplish the ambitious goal, they enact HQIM as an acceleration strategy. After a full year of implementation, student scores do not change in any meaningful way. Frustrated with the intervention, teachers describe that HQIM is no more effective than what they were previously doing, and they have the short term data to prove it. Knowing that teachers and students are still getting used to HQIM as a core part of classroom instruction, the RSSP team believes that if HQIM can be implemented for another 1-2 years, growth in student scores will begin to emerge. However, due to the pressure they are receiving and lacking any district data to support their intuition that student growth is correlated with strong HQIM intervention, the team relents and drops their efforts to ensure HQIM is implemented with fidelity. Throughout the remainder of RSSP, student scores remain the same.
How to Use Data to Approach the Problem
Step 1: Daniel Kim describes that, “A critical aspect of evaluating a ‘Drifting Goals’ scenario in an organization is to determine what drives the setting of the goal(s)... The relative strength of each potential influence will determine whether the [rigor of the goal] will drift up, down, or oscillate”.
We recommend first comparing your district's current RSSP goals to its previous RSSP goals. If the goals have decreased in rigor, then identify the stakeholder groups that exerted the most influence in changing each goal (RSSP team, principals, teachers, TEA, TAPs, etc).
You can then collect qualitative data –surveys and interviews–from the influential groups to determine their motivations and reasonings for changing the implementation and performance goals.
Step 2: Use the insights produced from the data collection to determine if the changes to the RSSP goals were needed adjustments or if they more closely resemble the drifting goals archetype. Kim explains that:
Making adjustments to initial goals is not inherently wrong. Sticking to the original goal purely for its own sake is as misdirected as changing the goal at every whim. But distinguishing between legitimate goal adjustments and the drifting goals structure can be very difficult – it’s easy to rationalize adjustments as “needed corrections”.
Step 3. If your district is caught in a drifting goals structure, then you can work with your RSSP team to design a new system for goal setting that exists outside of current decision-making structures. For example, if your team ties its district performance goal to state averages, then internal stakeholders will have little ability to diminish the goal's rigor. This could look like the following: "3rd graders at Bennion Elementary will achieve STAAR reading scores that are 2% higher than the statewide average."
Fixes that Fail
Dynamics
Fixes that Fail details the dangers of short-sighted solutions that ultimately cause more harm than good. The usual steps of Fixes that Fail are:
We observe a problem symptom
We fail to discern the root cause that produced the symptom
We create a "fix" that targets the symptom
The "fix" appears to work in the short term
The "fix" causes delayed unintended consequences that aggravate the root cause and exacerbate the problem symptom
Example
School leaders and teachers have noticed that since returning to school from the pandemic, students have been misbehaving at increased rates – coming late to class, wandering the halls during school, and actively disrupting instruction. Instead of working directly with students to understand the underlying motivation behind their behavior (which is that students feel their time in class is boring and not meaningful), the school decides to implement a stricter discipline policy – penalizing participation scores, referring students to the office, and issuing detentions and suspensions more frequently -- in order to “crack down” on student behavior. While these issues are not directly related to RSSP, the school has asked the RSSP team to create a dashboard to help track discipline data.
After instituting the harsher discipline policy, the problem symptom (student behavior) initially improves because students are afraid of being punished. However, student fear eventually turns into resentment, and their resentment into defiance. The frequency of student infractions increases because students are no longer deterred by fear of punishment. Students would rather express their feelings through defiant behavior than not express them at all.
How to Use Data to Approach the Problem: How to use data to approach the problem.
Step 1. Kim notes that, "In most instances of Fixes that Fail, people are usually aware of the negative consequences of applying a quick fix. But the pain of not doing something right away is often more real and immediate than the delayed negative effects. If the long-term/short-term trade-off were indeed one- for-one, where solving one problem today would create another one tomorrow, this strategy might be tolerable. But the reinforcing nature of unintended consequences ensures that tomorrow's problems will multiply faster than today's solutions."
If your district is caught in a fixes that fail structure, Kim recommends openly conveying to leaders that pursuing short-term fixes will create more headaches than they are worth.
Instead of jumping to quick fixes, take the time to talk with those who are the heart of the problem. Use qualitative data to understand the why behind problematic behaviors and identify potential long term solutions.
Step 2. Collaborate with all involved stakeholders to select the long-term solution that has the highest likelihood of targeting the underlying problem and producing long-term success.
Using collaboration and transparency when crafting a solution will help cultivate trust and buy-in from all stakeholders. When people understand that you are not interested in pursuing short term solutions and are enlisting their help in fixing the problem indefinitely, it can help spark a mindset shift within them. No longer concerned with short-term efforts, stakeholders can become collectively committed to attacking the heart of the problem.
Step 3. Implement the long-term solution
At the beginning of implementation, set expectations for all who are involved that significant time will pass before their efforts will begin to pay off. This helps temper any expectations that change will happen fast and prevents stakeholders from growing impatient too quickly.
Growth and Underinvestment
Dynamics
Growth and Underinvestment explains why some promising initiatives produce initial flashes of promise, only to be followed by long periods of stagnating outcomes. The classic Growth and Underinvestment steps are
We implement an intervention intended to improve system performance
System performance improves
As system performance improves, we fail to adequately invest in the capacity to sustain improvement.
System performance outpaces capacity, leading to stagnation or collapse.
Example
At the beginning of RSSP, a school district selected HQIM as their intervention strategy for 7th grade math and invested in effective HQIM training for its teachers. Throughout the year, teachers received regularly-occurring high quality professional development designed to help them implement HQIM with fidelity in their classrooms.
At the end of the school year, the district saw a 4% increase in average student math proficiency for 7th grade students. Happy with the success, the district continued implementing HQIM but reallocated the dollars for HQIM training to other district priorities. Throughout the next two years, teachers endeavored to implement HQIM with fidelity but lacked the support they enjoyed during year one. Student math scores remained stagnant, and district leaders begin to wonder if reinvesting in HQIM is worth the cost, or if the first year performance bump was simply an anomaly.
How to Use Data to Approach the Problem
Step 1: Kim describes that the key to overcoming Growth and Underinvestment problems is to "develop a way of assessing capacity relative to demands before the performance indicator starts to suffer".
The fundamental problem with growth and underinvestment is that we do not receive the information we need about capacity until it is too late to invest in capacity.
In the example above, the district leadership team had to wait for EOY student performance results to know if teachers were able to implement HQIM effectively without on-going professional development. Once it became clear that teachers needed that support, the school year had already concluded.
To obviate the problem, we recommend developing clarity on which behaviors produce the results desired by your school district. From there, create a measurement system that tracks the capacity of stakeholders to sustain (or increase) that behavior.
Step 2: Determine how to best invest in increasing stakeholder capacity relative to current and future demands.
When it becomes clear that stakeholder capacity will not be able to meet the increased demands placed upon them, carefully use resources to make investments that actually increase stakeholder capacity.
Poorly designed PD or untrained instructional aids are unlikely to help teachers improve classroom instruction.
Limits to Success
Dynamics
Limits to Success describes the situations where the actions that produce growth in performance suddenly begin to backfire. The process usually goes through the following actions:
We enact an intervention intended to improve system performance
The system improves
We ramp up the intervention anticipating perpetual growth
The intervention continually produces the result until a constraint is triggered
The constraint caps system improvement and often drives improvement levels back down
Example
At the beginning of RSSP, a school district selected differentiation and scaffolding as their acceleration strategy and built a performance monitoring system that tracked implementation fidelity. To support struggling teachers, the RSSP team created a peer coaching model where teachers who were implementing differentiation and scaffolding well could provide mentorship to those who needed it. At the conclusion of year one, the district saw a 5% increase in average student reading proficiency.
Happy with the success, the district continued implementing differentiation and scaffolding and maintained the performance monitoring and peer coaching systems, encouraging teachers to continue investing their time and energy into increasing students scores another 5% during the year. As the school year continued, teachers began to experience burnout from attempting to maintain the performance they had achieved the previous year. As teachers hit their breaking points, they began to take more time off, invest less energy into lesson preparation, and were unable to spend as much time with students after school. Consequently, student scores began to fall.
Seeing student scores fall, the district continued to push teachers to implement the acceleration strategies that had led to so much success in the previous year. Unfortunately, the harder the district pushed, the more teachers experienced burnout, and the more classroom instruction suffered. The year ended with a 2% decline in student reading proficiency.
How to Use Data to Approach the Problem
Step 1: Kim notes that "When the times are good and everything is growing rapidly, we trend to operate with an 'if it ain't broke, don't fix it' attitude. By the time something breaks, however, it may be too late to apply a fix."
If you suspect that your district is susceptible to getting caught in a Limits to Success structure, you should reflect with your RSSP team on the following question: "What kind of pressures are building in the district as a result of our improvement efforts"? Thinking through that simple question will likely key your team into the largest potential threats that will jeopardize continued school improvement.
Step 2. Investigate which potential threats are actual threats by collecting qualitative data from relevant stakeholders.
Step 3. Design proactive efforts to provide relevant stakeholders relief before they run up against their limits. Foreseeing and addressing limits before they are met can be the difference between abrupt failure and prolonged success.
Shifting the Burden
Dynamics
Shifting the Burden outlines the scenario of when a group of stakeholders become dependent on support provided by an outside source.
Stakeholders observe a problem symptom
Stakeholders develop an internal (self-sufficient) solution
An external intervention is created to help support the internal solution
The external intervention requires less effort from local stakeholders and undermines the internal solution
Local stakeholders become dependent on the external intervention at the expense of the internal solution
Internal capacity is weakened
Example
An RSSP district has a lackluster data culture. Its leaders and teachers do not set SMART goals, track implementation efforts, and they fail to share implementation and performance data in a timely manner with those who need it. The district has expressed a willingness to do a better job in using data to spur school improvement and has applied for a TEA-sponsored Data Fellows program to help them.
TEA provides the district with a Data Fellow as a temporary external intervention to help bolster their data culture and practices. Throughout her time at the district, the Data Fellow guides the RSSP team in setting goals and collecting, analyzing, and sharing data to drive continuous improvement. Unfortunately, instead of internalizing the core lessons and skills the Data Fellow attempts to share with leaders and teachers, they begin to believe that working with data is "What the Data Fellow does for us". Teachers and leaders reflexively leave most data work to the Data Fellow and fail to build data structures and protocol into district policy and practice. After the grant money ends and the Data Fellow is no longer there to maintain an effective data culture, the district falls back into its old habits.
How to Navigate the Archetype
Step 1. In the Shifting the Burden archetype, the external solution does a good job of addressing the identified problem. The difficulty comes when the external solution is either 1. Temporary or 2. Meant to build the capacity of stakeholders to solve their own problems. This is why the first step in navigating the Shifting the Burden structure is to develop clarity about the purpose of the external intervention.
Step 2. If the external intervention is indeed temporary and/or meant to build local stakeholder capacity, we recommend that implementers of the external intervention adopt a gradual release model that slowly transfers ownership of implementation over to internal stakeholders.
In the example above, the Data Fellow could begin her time at the district by facilitating data discussions for district leadership teams. After a few months, she could begin asking other members of the district leadership team to facilitate while she provides supplementary support. By the end of the program, members of the district leadership team should all be proficiency in facilitating data discussions without any support from the Data Fellow.
Your RSSP Archetype
With a clear understanding of the different system archetypes that relate to RSSP implementation, you can now use the note-catcher to:
Identify the archetype that most effectively limits student outcomes
Develop a data strategy to identify the archetype's leverage points
Create a plan to target the archetype's leverage points with your RSSP team.
Final Thoughts
Learning to view problems through the lens of archetypes empowers us to recognize the cycles in which we are caught and target the engines that drive the dysfunction. When we recognize the archetypes around us and collect data that substantiate them, we can craft powerful data stories that direct our stakeholders to the strategies that are most likely to achieve long-term success.
Module Complete!
Module Complete!
Congratulations on completing the module. Please complete the Exit Ticket form by clicking on the link above. We will use the information you submit to track your completion.
Page updated
Report abuse