Facilitator Competency Rubric

WHY the Facilitator Competency Rubric was Developed

We implemented a multi-site simulation education program for an undergraduate nursing program but realized that we had no way to evaluate whether our program was effective. Sure, we had the standard evaluation form that asks if the learning objectives were met and if the instructor was knowledgeable, but we did not have a way to observe for behavior and practice change. We searched the literature and asked our mentors and other experts, but a comprehensive facilitator evaluation tool did not exist. We wanted to look at the overall abilities of a facilitator, rather than selected parts, such as debriefing. Therefore, we set off to create the Facilitator Competency Rubric (FCR).

HOW the Facilitator Competency Rubric was Developed

The INACSL Standards of Best Practice: SimulationSM were used as the foundation for the FCR, which was built upon Benner's (1984) Novice-to-Expert theory, with the goal of creating a tool that could differentiate between varying levels of competency. We wanted to know who was competent, but also wanted to identify experts who we could enlist to mentor the identified novice facilitators.

Our five major constructs were identified, and sub-components to define each construct were selected. We presented the tool at six conferences over the course of one year to gain feedback as well as to socialize the concept of observational evaluation of facilitators. A workshop was also held for the purpose of developing the rubric concepts in alignment with Benner's theory. Our 4-member expert panel evaluated content validity and the content validity index (CVI) ranged from .75 to 1.0. Items with scores below .80 were revised for clarity.

Reliability and Validity of the FCR

The study was conducted at four sites, with a total of 7 raters and 18 facilitators volunteering participation in the study. Three sites were located in the US and one in the Middle East. Data analysis was completed with 81 observations.

Test-retest reliability, determined by the interclass correlation coefficient, ranged from .43 to .92 across institutions (good to excellent at 3 of 4 sites). Inter-rater reliability ranged from .62 to .90 (good in 3 of 4 sites). G coefficients ranged from .80 to .99 (good to excellent) for the four institutions. Variance attributable to the FCR items appeared to be at acceptable levels not exceeding 35% at any one institution.

Using the FCR

The FCR is designed for evaluation of simulation facilitators. While tested in the educational environment, the concepts are broad; feedback has indicated that the tool is appropriate for the clinical and academic simulation learning environments.

Inter-rater Reliability

It is vital that inter-rater reliability be established prior to using the FCR to evaluate facilitators to ensure that observers are interpreting the tool in the same way. You may also find that an item or two is not applicable for your learning environment. For example, scheduling (Preparation construct) is not the responsibility of the facilitator at many institutions. In that case, the item should be excluded from your evaluation.

Examples that can be used to help establish inter-rater reliability will be provided if you decide to consider use of the tool .

Evaluation

Gaining agreement from facilitators to be observed and evaluated may present difficulties. The FCR can also be used for self-evaluation by facilitators. The information gained from use of the FCR can be used to individually evaluate facilitators as well as to identify development needs and prioritize facilitator development activities.

Scoring

There is no established method to score the FCR. We suggest that you focus on the lowest scoring construct areas first, and prioritize the lowest item scores in that construct area. Use these low scoring items as a needs assessment for creating a facilitator development plan. An example is provided if you decide to consider use of the tool.

Click HERE to obtain your copy of the Facilitator Competency Rubric.


References:

For additional detail regarding the development and psychometric analysis of the FCR:

Leighton, K., Mudra, V., & Gilbert, G. E. (2018). Development and psychometric evaluation of the Facilitator Competency Rubric. Nursing Education Perspectives, 39(6), E3-E9. doi: 10.1097/01.NEP.0000000000000409 https://tinyurl.com/ya2tepcv

Webpage suggested citation:

Leighton, K, Mudra, V., & Gilbert, G. E. (2018). Facilitator Competency Rubric. Retrieved from https://sites.google.com/view/evaluatinghealthcaresimulation/fcr