Data Collaborative Portal

Is the adherence of what is done to the standard of the intended plan.

Tell me more

Fidelity is designed to assess the use and outcomes of the skills that are reflected in the selection criteria, taught in training, and reinforced and expanded in coaching processes. Assessments of practitioner performance (sometimes called measures of fidelity) also provide feedback useful to key implementation staff (interviewers, trainers, coaches, administrators) regarding the progress of implementation efforts and the usefulness of selection, training, and coaching methods. For example, organizations consistently monitor current performance assessments in search of common strengths and areas that need improvement to make adjustments in how selection, training, and coaching are conducted to help strengthen skills related to that area. The organization remains accountable for ensuring that current and future practitioners will achieve high levels of effective performance when working with children, families, and others. Organizations make use of data to continue to improve Fidelity Assessment methods. Fidelity assessment may include multiple measures to address context, content, and competency. Fidelity assessments may exist for different roles.

Driver Look Fors

When we are integrating the Implementation Drivers into our work we have specific look fors for each Driver that will help inform us whether or not we are removing as many potential barriers and seizing all the opportunities to improve our program. Below are the specific look fors for this driver. It includes the following information:

Look For Description: it will begin with a numbered look for the driver. It will include a description of a critical component that should always be seen in your organization.
Data Source(s): The Data Sources are a nudge to help us understand how we can prove that this is actually occurring throughout our organization, as well as a location to capture change ideas and improvements.
Fully In-Place: This represents the criteria that the Data Collaborative, and NIRN, considers that the look for is completely in-place. Or in other words, the organization is meeting our expectations for this component of the driver.
Partially In-Place: Similarly to Fully In-Place, this represents the criteria. The difference is that not all of the look for are represented in the organization.
Not Yet In-Place: This represents when the look for has not been reflected in your organization.

1. There is someone accountable for the fidelity assessments/observation of staff who will carry out explicit instruction in literacy.

Fully In-Place

A specific person is responsible for coordinating fidelity assessments processes for staff supporting explicit instruction in literacy

- AND -

This person is able to execute the responsibilities related to his/her role in fidelity assessment process

Partially In-Place

A specific person is responsible for coordinating fidelity assessments processes for staff supporting explicit instruction in literacy

Not Yet In-Place

There is not a specific person responsible for coordinating fidelity assessments processes for staff supporting explicit instruction in literacy

Data Source(s)

Job description of person accountable for fidelity assessments

2. Leadership Team (LT) supports the use of a consistent fidelity assessment/observation for explicit instruction in literacy

Fully In-Place

The fidelity assessment measures:

• Context - Measures the extent to which the prerequisites and conditions for explicit instruction in literacy to operate are met (e.g., staff-client ratio, location of delivery)

• Content - Measures whether the practitioner is following the

guidelines of explicit instruction in literacy (e.g., compliance to

model standards such as frequency, duration, curriculum)

• Competence - Measures the extent to which the practitioner demonstrates skill in the delivery of explicit instruction in literacy

Partially In-Place

The fidelity assessment measures:

• Context - Measures the extent to which the prerequisites and conditions for explicit instruction in literacy to operate are met

• Content - Measures whether the practitioner is following the guidelines of explicit instruction in literacy (e.g., compliance to model standards such as frequency, duration, curriculum)

Not Yet In-Place

LT does not support the use of a consistent fidelity assessment

Data Source(s)

Fidelity assessment may include multiple measures to address context, content, and competency

3. Fidelity assessment/observation is demonstrated to be correlated with outcomes

Fully In-Place

Research evidence demonstrates the fidelity measure to be correlated with outcomes

Partially In-Place

The fidelity measure Is currently in development (i.e., not yet correlated with outcomes or research validated)

Not Yet In-Place

There is no research evidence demonstrating a correlation between the fidelity measure and outcomes

-OR-

It is not known if the fidelity assessment is correlated with outcomes

Data Source(s)

Technical Manual
Research documents
Documentation of communication with fidelity measure developers

4. Leadership Team (LT) follows a protocol for fidelity assessments

Fully In-Place

LT follows a written protocol that includes all of the following:

• Staff are oriented to how fidelity is assessed

• Fidelity assessments use multiple sources of information (e.g., practitioners, supervisors, consumers)

• Intended use of fidelity assessment data to improve supports

• Communication protocol for sharing fidelity data

Partially In-Place

LT follows a written protocol that includes the following:

• Fidelity assessment data are used to improve supports for practitioners

Not Yet In-Place

LT does not follow a written protocol for fidelity assessments

Data Source(s)

Performance assessment (fidelity) Protocol
Documentation of staff performance (fidelity) assessments

5. Leadership Team (LT) uses fidelity assessment/observation data to improve explicit instruction in literacy outcomes and implementation supports.

Fully In-Place

LT uses fidelity assessment data at least four times per year

-AND-

LT use fidelity assessment data to:

• Assess and improve the effectiveness of selection, training and coaching processes for practitioners

• Assess and improve the effectiveness of data systems and organizational supports

Partially In-Place

LT use fidelity assessment data at least annually

-AND-

LT use fidelity assessment data to:

• Assess and improve the effectiveness of selection, training and coaching processes for practitioners

Not Yet In-Place

LT do not use fidelity assessment data

-OR-

Fidelity assessment data are used to penalize staff (e.g., used as sole source of data for annual staff evaluations or salary recommendations)

Data Source(s)

Documentation of action plans for improvement of selection, training, or coaching processes.
Documentation of feedback to coaches and/or trainers
Documentation of feedback provided to practitioners

Source:

Ward, C., Metz, A., Louison, L., Loper, A., & Cusumano, D. (2018). Drivers Best Practices Assessment. Chapel Hill, NC: National Implementation Research Network, University of North Carolina at Chapel Hill. Based on: Fixsen, D.L.,Blase, K., Naoom, S., Metz, A., Louison, L., & Ward, C. (2015). Implementation Drivers: Assessing Best Practices. Chapel Hill, NC: National Implementation Research Network, University of North Carolina at Chapel Hill.

Report abuse