Data-decision Process

Tiered Fidelity Inventory Item: 1.13 Data-based Decision Making

It is important to understand the positive, proactive, and instructional philosophy of discipline and PBIS and to be committed to the work. Schools must also blend commitment and proven practices with strong leadership and effective school improvement processes to promote deep and lasting change (U.S. Department of Education, 2014).

The ongoing Data Based Decision Making (DBDM) Process for PBIS is similar to most school improvement, problem solving, or decision-making models and includes collecting data to identify current status, analyzing data, identifying desired outcomes, developing an implementation plan including selecting strategies, determining indicators of results, and establishing an evaluation plan. This thinking is paramount for any improvement effort and remains at the heart of your PBIS work.

More specifically, the procedure for unfolding your schoolwide work is outlined below.

Collect & Chart Data
- Focus on a specific question and work to answer, where are we now?
- Gather data pertinent to answering the question at hand (staff survey; records of office referrals, detentions, suspensions, etc.; observations).
Analyze and Prioritize
- Where are the gaps in your current status compared to where you want to be?
- What data can help you drill deeper to further define the who, where, when, why and how?
Develop S.M.A.R.T. Goals
- Define your outcome goals so that they are Specific, Measureable, Achievable, Relevant, and Time Bound.
- Short term and long term goals must be written to be observable and therefore measurable.
- Resource: S.M.A.R.T. Goals Worksheet
Select Strategies
- What strategies will most efficiently and effectively get you to your desired goals?
- Strategies selected should be written to into an action plan with aligned goals, steps resources, timeline and communication plans.
Determine Results Indicators
- Select indicators that help your team answer
  - Are we implementing the plan with fidelity?
  - Is implementation having desired impact on student outcomes?
Evaluate Plan
- Analyze collected data to determine progress toward meeting goals
  - Have we implemented our plan with fidelity?
  - Have we achieved our goal or are we making adequate progress toward achieving our goal?

Deciding How to Decide

As your work progresses, you will encounter decisions that need to be made by the team as well as larger decisions, made by the entire staff, regarding the development and implementation of new approaches. Without effective tools to make these decisions, you may get bogged down in indecision or prolonged discussions and even division.

Who Decides?

There are some decisions where it just doesn’t make sense to involve everyone. Identifying an individual point person can be very appropriate when that person has much relevant knowledge or authority. Decision-making by the administrator or selected team member may be very appropriate. Similarly, team decisions also may be appropriate when representation of the school is needed, but involving the entire staff is impractical, unnecessary, or too time consuming. However, you will want to take the time to involve the entire staff when the issue is one that everyone needs familiarity with or that requires the support of each person to carry out. This is essential when decisions are being made about new procedures or approaches to be implemented by all.

How to Decide

Once you have considered who needs to decide, the second question is how to decide. The common approaches for making decisions are listed Resource: Common Approaches to Making Decisions. When divisiveness is present, schools would be best to revisit mission, vision, guiding principles, or beliefs to foster harmony before moving on to make decisions using a consensus model. Effective leaders understand the importance of striving for consensus, but also understand that not all decisions should be made by the entire group or by using consensus strategies. In that case a majority rule or unilateral decision will be made. In addition, when the group is “stuck” a smaller group can be assigned to bring suggestions back to all. This would be an example of participative or representative decision-making.

Resource: Common Approaches to Making Decisions

"Data is like garbage. You’d better know what you are going to do with it before you collect it."

- Mark Twain

What is data-based decision making?

Decisions that are based on data help us to create the ideal learning conditions for our students. Many of our decisions can be made without collecting data systematically. However, at times a strategic data-based decision making process may be needed. A formal data-based decision making process can help us identify important variables related to our teaching faster and more efficiently than relying on our memory to recall important events. This is especially true for teachers who often deal with many challenges that require their attention simultaneously.

How can teachers use data-based decision making?

Quick, time-efficient strategies can be used when a teacher wants to collect data on one or more students. For instance, placing a handful of paper clips in one pocket and transferring one paper clip to another pocket every time a behavior is observed is a relatively simple strategy that can be used during instruction. Another strategy may include analyzing assignment scores after class by summarizing these data. In more complicated situations, assistance from paraprofessionals or other school personnel who can spend time observing and recording information may be necessary for data collection purposes. Some schools have organized student improvement teams so that teachers can ask other school personnel to conduct observations while they are teaching class. Observers in these schools include school psychologists, behavior support specialists, paraprofessionals, counselors, fellow colleagues, or administrators. Each school can create strategies to enhance data-based decision making systems by tailoring these strategies to their particular school's characteristics and needs. For more information about embedding data-based decision making into school systems, consult the School-wide Positive Behavior Support Module.

Data-based decision making can be used in many different ways with individual students or classrooms. Questions about student academic performance and both problematic and appropriate behavior can be answered by collecting information systematically. Examples of questions to investigate might include the following:

Is the lesson plan too difficult for most students?
Are students experiencing enough success on independent work tasks?
Does the number of homework assignments completed increase when students receive reinforcement (extra free time, bonus points, etc.)?
Are the new classroom management strategies related to decreases in student misbehavior?
Are there times of day or specific activities when the students are more likely to be on-task?
Is there a difference in academic scores on in-class exams when students complete a review activity that summarizes major points the day before?
Has the student engaged in more social interactions with her friends and this month since the peer-mentoring plan started?

In addition to student behavior, you can collect data on your own behavior as well. Master teachers are constantly reviewing their performance and trying new strategies. Self-monitoring and self-management strategies can help you improve your instructional and classroom management skills. Data can be collected on your own behavior by using a small counting device that you can click every time you engage in a behavior. For instance, you could keep track of the frequency of verbal praise given to students. Other adults can come observe your class and collect observational data while you are busy teaching. Teaching students to collect data is a valuable learning activity and asking for their feedback about lessons provides valuable information that can be used to improve your teaching. Types of questions to investigate about your own work might include the following:

Am I providing reinforcement to students on a ratio of 4 positives for every demand or correction?
Which lesson did the students prefer and find more interesting?
Am I moving around the room and using proximity to decrease problem behavior?
Am I using pre-correction on a regular basis in math class?
Am I attending to students when they raise their hand to answer a question?
How often do I praise students for correct answers when they yell out the answer instead of raising their hand?
Am I providing examples throughout the lecture?
Are my classrooms expectations clear to the students? How many of them can tell me these class rules?
How predictable is my classroom schedule? How often do I let the students know ahead of time that we will be changing activities?
How much time do I allow for students to work independently? Is this too much or too little?

How do you decide what to measure?

The decision to measure something is made when you are concerned about some event (for example, individual student or class academic progress, student conduct in class) or when you want to evaluate a new strategy or intervention. The first step is to define the behavior of interest. It is always very important to create a clear definition, especially when asking someone else to assist in collecting data. If the behavior is defined well, it will include:

A brief description of the behavior,
Information about what the behavior looks like (topography), and possibly what it does not look like (so as to not get confused with similar behaviors),
Details about the frequency, length or duration of the behavior, and/or
Information about the behavior's intensity.

If the concern is about an academic behavior, it is important to establish a certain criterion that describes the behavior of concern. For example, you may be interested in any score that is below 50% on a student's test. You can then count the number of times a student scores below 50% on the tests (note that this excludes all other scores, such as homework assignments. Good definitions will be written in such a way that someone who hasn't seen the behavior will be able to understand and observe it.

Resource: Good vs. Bad definitions examples

Once the target behavior has been defined, you can choose a measurement strategy that best fits the type of behavior you are observing.

Which measurement strategy should I use?

Some measurement systems provide an exact measure of a behavior's occurrence, while others provide a general estimate or proportion of the behavior's occurrence. The type of measurement system chosen depends upon a number of different issues including what the behavior looks like (topography), the frequency of the behavior, and the time and energy the person observing can dedicate to observing and recording the data. In general, elaborate and complicated measurement methods for collecting data can yield greater accuracy. Nevertheless, this is only true when the method is used as intended. We can achieve greater accuracy with a method that is not as rigorous but is a better fit for the time, energy, and r of those responsible for using it. In addition if data are not used on a regular basis to make decisions, the quality of the recording often decreases and the process is viewed as a waste of time.

Each measurement strategy has advantages and disadvantages. Some measurement strategies are more accurate but are difficult to implement. In some cases, you may choose to use variety of methods to measure different aspects of the same behavior depending upon how much time you have to invest. Follow the guidelines below to identify the best measurement strategy for different types of behaviors. Please note that the methods have been placed in order of least to most difficult to implement. The links in this section will take you to a description of different types of measurement tools within the Data-based Decision-making Module.

Choose a measurement strategy by asking the following questions:

Does the behavior generate a product (for example, a written assignment, a clean table, or papers on the floor)? If the answer is yes, click here for Permanent Product recording.
Can you easily count every time that the behavior occurs (for example, raising your hand? Can you easily identify when the behavior starts and when it ends? Would this behavior be easy to count (or does it occur so frequently that it would become complicated to track)? If the behavior is easy to count, click here for Event Recording.
Does the behavior occur so often that it may be difficult to count each occurrence (for example, blinking, or is it difficult to tell exactly when the behavior starts or when it ends (for example, reading)?

If the answer is yes, ask these additional questions:

When the behavior occurs, does it last for a while (for example, writing)? If the answer is yes, click here for Momentary Sample recording.
Does the behavior happen so quickly that it is hard to catch (for example, swearing or making gestures)? If the answer is yes, click this link for Partial Interval recording.
Is it important to know that the behavior continues without interruption (for example, studying)? If the answer is yes, click here for Whole Interval recording.

Do I want to measure how long it takes for the behavior to begin (for example, how long it takes for a student to respond to a request)? If the answer is yes, click here for Latency (Time to respond).

Do we want to measure how long the behavior lasts (for example, how long the student spends reading)? If the answer is yes, click here for Behavior Duration.

If you are collecting data in a team setting, the next step is to identify:

Who will be recording the behavior
Exactly when and where they are to be recording or observing the behavior, and
How often observations will occur.

The person observing should be in fairly close proximity to the student they are observing to make sure all instances of the behavior can be seen. However, the person observing should also try to be as discrete as possible while observing the student, so as to not influence the occurrence of the behavior. It can be helpful to discuss types of student behavior or times when a behavior will not be counted. For instance, a teacher and paraprofessional may be collecting data on how often a student places his head on his desk. The teacher may not think that it is a problem for the student to put his head on his desk during breaks. However, if the paraprofessional counts all occurrences of the student's behavior while the teacher only records occurrences during lessons, there may be problems later interpreting the data. The number of times the student's head was on his desk is in part reflective of the way in which each observer recorded the data.

Deciding how often observations will occur depends upon a number of factors including the time available to collect data, the concern or strategy being evaluated, and the type of behavior. A common mistake is to assume that behavior must be collected all the time, every day. Creating a highly intensive data collection schedule is not usually necessary. However, it is crucial for everyone measuring the behavior to be clear on when the behavior is to be measured so that everyone's numbers are comparable. It is also important to think about how long observations will be conducted. You will need to observe long enough to identify a clear pattern of behavior over time and to feel confident that you have gathered enough information to answer your questions.

How are data summarized to evaluate new strategies and interventions?

The way to summarize data efficiently for decision making is to create a visual summary using a graph. A graph is a visual representation of the occurrence of behavior over time. After instances of behavior are recorded on a measurement form, the information is summarized and then transferred to a graph. Graphing the data you are collecting helps to organize the information and identify important patterns related to behavior. The information collected before any interventions have been implemented is referred to as a baseline. The information collected during the implementation of interventions is referred to as intervention data. Baseline data are compared to intervention data to determine whether the intervention you are conducting is resulting in positive outcomes (for example, increases in academic achievement, increases in positive social skills, and decreases in problem behavior (Freeman & Tieghi-Benet, n.d.).