TFI 2.12 EVALUATION: Fidelity Data
Tier 2 team has a protocol for ongoing review of fidelity for eah Tier 2 practice.
Embedded Files
Systems support the accurate and durable implementation of practices by staff, the efficient use of data for decision-making, and achievement of outcomes. Staff are trained to prevent many student behavior problems as well as to deal with disruptive behaviors in a proactive and positive manner. Systems provide the procedures and infrastructure to support and maintain new evidence-based practices.
Systems support the accurate and durable implementation of practices by staff, the efficient use of data for decision-making, and achievement of outcomes. Staff are trained to prevent many student behavior problems as well as to deal with disruptive behaviors in a proactive and positive manner. Systems provide the procedures and infrastructure to support and maintain new evidence-based practices.
TFI 2.12 Big Ideas
TFI 2.12 Big Ideas
In element, TFI 2.12, the Tier 2 Team will create a process to collect, analyze and report fidelity throughout the year for each Tier 2 intervention identified in TFI 2.5.
In element, TFI 2.12, the Tier 2 Team will create a process to collect, analyze and report fidelity throughout the year for each Tier 2 intervention identified in TFI 2.5.
Action planning includes: gai.....
Action planning includes: gai.....
When to Collect Fidelity Data
When to Collect Fidelity Data
Tier 2 Teams should consider collecting Fidelity Data:
A. When a student's response to the intervention is Questionable or Poor,
AND
B. Throughout the year to ensure the intervention is being implemented as intended by all those involved.
Questionable or Poor Response to the Intervention
Questionable or Poor Response to the Intervention
In TFI 2.11 the Tier 2 Team created a process to identify when a student was having a Questionable/Poor response to the intervention, and what to do in response. STEP 1 of the process developed in TFI 2.11 was to evaluate the fidelity of the intervention’s implementation prior to making any changes to the student's intervention. If the intervention is not being implemented with fidelity, then all students are at risk for not meeting their goals.
In TFI 2.11 the Tier 2 Team created a process to identify when a student was having a Questionable/Poor response to the intervention, and what to do in response. STEP 1 of the process developed in TFI 2.11 was to evaluate the fidelity of the intervention’s implementation prior to making any changes to the student's intervention. If the intervention is not being implemented with fidelity, then all students are at risk for not meeting their goals.
Throughout the Year
Throughout the Year
Tier 2 Teams should also monitor fidelity throughout the year. Fidelity should be monitored for multiple students within the intervention (randomly selected) and multiple staff involved in intervention implementation (teacher(s), facilitator(s), coordinator(s), etc.).
How to Monitor Fidelity
Information for this section was adapted from Missouri Ch. 4 & 5
Once an intervention is developed and being implemented full scale, several aspects will need regular attention and consideration to ensure maximal effects and benefits from the selected treatment. There are multiple ways Tier 2 Teams can monitor Fidelity to implementation:
Once an intervention is developed and being implemented full scale, several aspects will need regular attention and consideration to ensure maximal effects and benefits from the selected treatment. There are multiple ways Tier 2 Teams can monitor Fidelity to implementation:
Monitoring Fidelity of Intervention Implementation
Monitoring Fidelity of Intervention Implementation
Monitoring Social Validity of Interventions
Monitoring Social Validity of Interventions
By adopting multiple ways to assess Fidelity, Tier 2 Teams will be better equipped to monitor and respond to fidelity data.
Monitoring Fidelity of Intervention Implementation
Two simple methods schools can use to assess implementation includes: a) Daily/Weekly Progress Report Form Review and b)Direct Observation. Each method has particular strengths and potential limitations. School teams will need to consider and develop a process that provides evidence the intervention is being implemented as intended, but at the same time it is not too time consuming to complete.
Two simple methods schools can use to assess implementation includes: a) Daily/Weekly Progress Report Form Review and b)Direct Observation. Each method has particular strengths and potential limitations. School teams will need to consider and develop a process that provides evidence the intervention is being implemented as intended, but at the same time it is not too time consuming to complete.
DAILY/WEEKLY PROGRESS REPORT FORM REVIEW
DAILY/WEEKLY PROGRESS REPORT FORM REVIEW
In the event that student data indicates a questionable or poor response to the intervention the first question school teams should consider is whether all elements of the program are occurring. One way to measure this is by reviewing student Daily/Weekly Progress Reports. The intervention coordinator along with the Tier 2 Team can examine 3 to 5 of the most recent progress reports to verify several elements of the program. Review of student Daily Progress Reports will provide answers for the following questions:
In the event that student data indicates a questionable or poor response to the intervention the first question school teams should consider is whether all elements of the program are occurring. One way to measure this is by reviewing student Daily/Weekly Progress Reports. The intervention coordinator along with the Tier 2 Team can examine 3 to 5 of the most recent progress reports to verify several elements of the program. Review of student Daily Progress Reports will provide answers for the following questions:
Sample CI/CO DPR Fidelity Tracking Tool
- Did the student check-in (if applicable)?
- Were points awarded for each class period?
- Did the student check-out (if applicable)?
- Was student data totaled and recorded (e.g., Advanced Tiers Spreadsheet)?
- Did the caregiver sign and return the Daily Progress Report?
If review of a student Daily Progress Report provides evidence these elements occurred, the school team has some confidence the main intervention components were in place and the student participated appropriately. If a student Daily Progress Report indicates an area of low implementation, a member of the Tier 2 Team should be designated to provide re-teaching as needed (e.g., student, teacher, and/or parent).
If review of a student Daily Progress Report provides evidence these elements occurred, the school team has some confidence the main intervention components were in place and the student participated appropriately. If a student Daily Progress Report indicates an area of low implementation, a member of the Tier 2 Team should be designated to provide re-teaching as needed (e.g., student, teacher, and/or parent).
DIRECT OBSERVATION
A second method for verifying accuracy of intervention implementation is conducting observations of particular components. In this case, use of an observation checklist may be especially helpful both for documenting specific features that occurred and for providing feedback to implementers.
In particular, it is highly recommended that observations of the facilitator implementation and/or teacher feedback be scheduled, especially when a new student first enters the program.
Use of direct observation, feedback and re-teaching, as needed is a supportive response for facilitators and teachers who perhaps are being asked to use skills they have not previously developed. If an observation process is established and communicated, up front, as part of the intervention, facilitators/teachers will be more comfortable when the observations occur.
Intervention personnel, particularly those who are new to the program, that receive positive specific feedback and encouragement for correct implementation are more likely to have students that respond positively.
Each Tier 2 intervention identified in TFI 2.5 should have a checklist/observation form facilitators and teachers can use to assess their own implementation fidelity. This form can also be used by the intervention coordinator to observe and document fidelity to the intervention components. A CI/CO sample is located to the right.
Monitoring Social Validity of Interventions
Social validity, which is sometimes also referred to as treatment acceptability, focuses on whether the goals, the intervention elements, and the anticipated outcomes are acceptable, socially relevant, and useful to the individual and to those who care about the individual.
Social validity, which is sometimes also referred to as treatment acceptability, focuses on whether the goals, the intervention elements, and the anticipated outcomes are acceptable, socially relevant, and useful to the individual and to those who care about the individual.
WHAT DOES MONITORING SOCIAL VALIDITY MEAN?
WHAT DOES MONITORING SOCIAL VALIDITY MEAN?
Monitoring social validity means to regularly and systematically assess the social significance of intervention goals, the social acceptability of the proposed intervention procedures to attain the goals, and evaluation of the social importance of the effects resulting from an intervention (Kazdin, 1977; Wolf, 1978).
Monitoring social validity means to regularly and systematically assess the social significance of intervention goals, the social acceptability of the proposed intervention procedures to attain the goals, and evaluation of the social importance of the effects resulting from an intervention (Kazdin, 1977; Wolf, 1978).
Social validity data typically provides a picture of the extent to which particular stakeholder groups (i.e., students, families, and teachers) value an identified practice or program. Social validity data is commonly gathered through use of a survey or asking personnel to respond to items on a brief questionnaire.
If social validity results are low it may be difficult to continue implementation of the practice “as is.” Instead teams will want to investigate why the practice is perceived poorly and make adjustments either by providing additional information and technical assistance and/or by making changes to features that perhaps are not feasibly maintained.
WHY MONITOR SOCIAL VALIDITY?
Organizing efforts to strategically assess and review social validity data provides an opportunity for participating staff, students, and family members to identify and agree on target areas for intervention; appropriate, acceptable and attainable goals; and determine environmental supports that will reinforce use of new skills. Teams that incorporate opportunities for monitoring social validity generally have improved likelihood of commitment to implement and continue an intervention until the goal is met.
HOW TO MONITOR SOCIAL VALIDITY?
There are several options for on going monitoring of social validity. Simple procedures include use of a structured interview process and/or rating scales or questionnaires. One important aspect to consider is when social validity monitoring will occur. In most cases the simplest approach is to ask participants of an intervention to complete a survey after the intervention has concluded (e.g. upon graduation). However, a more thorough and likely more meaningful approach would be considering social validity throughout all phases of the intervention process; that is before the intervention begins, during the treatment, and after the intervention concludes. In addition, data that are collected from all relevant stakeholders (students, teachers, and family members) are most valuable for teams as they continue working to improve existing supports and sustain effective interventions.
For Social Validity sample surveys - Missouri, Ch. 4, pgs. 36-41
How to Score 2 Points on TFI 2.12
To receive a 2 Point Score Teams must be able to say yes to the following set of questions.
For each intervention selected in TFI 2.5:
2 Point:
2 Point:
Periodic, direct assessments of fidelity collected by Tier 2 team for all Tier 2 interventions.
Periodic, direct assessments of fidelity collected by Tier 2 team for all Tier 2 interventions.
1 Point:
1 Point:
Fidelity data (e.g., direct, self-report) collected for some but not all Tier 2 interventions.
Fidelity data (e.g., direct, self-report) collected for some but not all Tier 2 interventions.
0 Point:
0 Point:
Fidelity data are not collected for any practice.
Fidelity data are not collected for any practice.
Information adapted from: MO-SWPBIS Handbook/Tier Two Implementation Guide; Center on PBIS/PBIS.orgImages obtained from Google Images and/or created by J. Patrick
Page updated
Report abuse