AMC Evaluation Training Course  

Background 

Organization 

AMC (pseudonym) is a dialysis center with 2,600 outpatient clinics across North America. AMC’s mission is to deliver superior care to improve the quality of care for every patient and in doing so effectively set the standard by which the healthcare industry measures its performance. To help fulfill its mission, AMC developed an education department and more specifically a Learning Content & Solutions (LCNS) department. Both departments work together to design all education materials (e.g. training, performance aids, SOPs,and evaluations) to help nurses throughout AMC with increasing performance, knowledge, and skill. 

Problem & Instructional Design Request 

After a recent reorganization at AMC, the education and LCNS departments noticed a decrease in nurses completing learning evaluations after training. This change is significant because learning evaluations help to collect data-based feedback about the efficacy of training and can highlight areas for improvement at AMC that could impact the quality of care for patients and the success of the organization. After all, the relationship between AMC and their patients is that AMC employees, specifically nurses, will provide medically sound and professional ways to help them manage their life threatening medical hardship. Nate (pseudonym), a LCNS staff member at AMC and fellow student in Boise State University’s Organizational Performance and Workplace Learning master’s degree program, proposed the instructional design project to develop a training module to address the problem. I joined the ID project team along with two other graduate students. The project was sponsored by Nate’s director, Janet (pseudonym).

ID Project Stakeholders 

• Janet (pseudonym): the director of the Learning Content & Solutions (LCNS) department

• Nate (pseudonym): LCNS staff member and student in the Organizational Performance and Workplace Learning (OPWL) graduate studies program at Boise State University: a graduate studies program that prepares students to effectively apply systematic processes for analysis, evaluation, and design to new and/or infrequent or systemic organizational changes and problems.

• Upstream stakeholders: the entire LCNS department, including Janet and Nate, the education department, and AMCs leadership. 

• In-direct downstream stakeholders: nurse managers who work directly with nurses and sometimes directly with dialysis patients. They are responsible for managing the work process design and performer goals of their respective clinics, as well as assessing nurses’ performance. 

• Direct stakeholders: nurses who work directly with patients and whose behavior and performance directly impact the quality of services AMC’s dialysis patients receive, which is the primary factor of AMC’s mission.

Challenge(s) 

Interviews with the ID (instructional design) project sponsor, Janet, highlighted a number of challenges.  

1. Obtaining & Using Robust Evidence-Based Data: The presented challenge/problem of nurses not completing learning evaluations as much as they did before an organizational change was only supported by anecdotal evidence. In every interview with the client and the project liaison they both emphasized that learning evaluations were valuable to AMC for many reasons. Namely that the evaluations shed light on nurses’ current state of information, confidence, and comfortability applying what they learned from recent training to their work with patients, which if not expertly applied could lead to disastrous consequences for patients and negatively impact AMC’s ability to fulfill its mission. However, the client and the project liaison believed that a 10 week project timeline was not enough time to create a 60 minute training module and systematically review evidenced-based data or conduct a formative evaluation of the learning evaluation process to help the ID process. Evidence-based practices help to ensure that the design of interventions like instructional materials are not simply addressing the perceived problem, but are also addressing root causes (Rothwell, et al., 2016). Overall it was difficult for the team to advocate for and get approval from Janet, the project sponsor, to collect and analyze data aside from our interviews with her, so the project is solely based on one data source which is interviews with the project sponsor. 

2. Organizational Changes & Change Management: The decline in nurses completing learning evaluations after training was a new phenomenon that appeared to be the result of AMC’s recent organizational change. Organizational changes are disruptive. No matter how well planned they are they can lead to shifts in employee morale, behavior, performance, and retention (Watkins, 2006). Implementing an intervention such as the training course may be perceived by nurses as just one more change that could disrupt workplace performance even further. So we proposed, advocated for, and helped to build adult learning theories such as the influence of motivation on adult learners to engage and retain information as well as adjust to changes in the work environment into the design of our training course (Miriam & Bierema, 2014).

3. Nurse/Nurse Managers Schedules: With 2,600 clinics across North America, AMC’s nurses  and nurse managers worked in at least four different time zones with a variety of operational hours that were scheduled to align with the availability of the patients they served. For this reason, it was important to our team that we design a training course that was accessible (e.g. online) and maximized time and resources efficiently so that nurses and nurse managers received high quality, applicable, and time efficient training to improve their completion of learning evaluations. 

Goals 

The goals of our project were as follows: 

• Create a 1-hour long training course for a problem that could be addressed using instruction. 

• Use academic and industry vetted techniques and frameworks to design the instruction.

• Create a lesson plan and all training materials and present them in an executive report. 

Approach 

In order to achieve our project goals we used three separate systematic analyses to help to define AMC’s problem/challenge, analyze it, and design an instruction-based solution to address it. The three analyses we used are described in the following table and can be reviewed in Appendix A: 

We used a course templated lesson plan to outline the training course and designed a variety of instructional materials based on ARCS motivation theory and LeaPs instruction model such as:

1. A training course invitation email (Appendix B) 

2. Pre-training questions to orient nurse managers to the training (Appendix C) 

3. A JAM board activity, an online interactive poll, to assess and activate nurse managers prior knowledge and motivation in the training (Appendix D)

4. A scenario activity where learners apply what they have learned about Kirkpatrick’s evaluation to workplace scenarios (Appendix E)

5. A PowerPoint deck for the online training course (Appendix E) 

6. A “Commit to try” document for learners, nurse managers, to complete before ending training to catalyze accountability and potentially transfer of learning form the training to the workplace (Appendix F). 

Note, an example of the lesson plan template can be reviewed in Appendix A. 

Solutions 

Based on the TRA, LEA, and TA we determined that AMC’s problem of nurses not completing learning evaluations can be addressed through instruction. We also narrowed down the learners to be nurse managers because they are responsible for administering learning evaluations to nurses after training and knowledge and skills development. Based on the analysis of our learners and their varying schedules, we determined that it would be useful to make the training on evaluation online and live through multiple delivery dates so that it can be accessible to as many nurses as possible. The analysis of our learners and their learning environments inspired us to design the lesson plan according to ARCS motivation theory (Miriam & Bierema, 2014) because it is a useful tool for leveraging motivation to catalyze and sustain learning such as nurse managers' motivation to help AMC fulfill its mission. We also outlined the lesson plan according to LeaPs instruction model because it is a powerful tool for leveraging exploration and active participation to stimulate learning, such as having nurse managers from different locations work together to determine the type of evaluation level that could be used for the real life scenario presented. We knew from Janet and Nate that AMC typically uses Kirkpatrick’s 4-level Model of Evaluation to design most of their learning evaluations for the organization so nurse managers would be familiar with this type of evaluation. Therefore, we designed our training course to focus on the development of knowledge and skills, specifically how to effectively deploy Kirkpatrick’s 4-level Model of Evaluation. Additionally, the training module included interactive activities with real life examples to help increase nurse manager’s understanding of the value and impact that completing evaluations has on their work.

Areas for Improvement

learners who have a variety of schedules, the accessibility of the course could be improved in at least two other ways. 

1. Create an e-learning version of the course with software such as Articulate Rise 360 that includes interactive activities such as pre-recorded videos and engagement activities and assessments such as scenario-based learning quizzes to reflect the live online versions of the training. This type of training module could be sent to nurse managers who could not participate in the scheduled live online training courses. This would ensure that as many nurse managers receive the training. 

2. Create a Spanish translation of the training course. In interviews with Janet and Nate, both stakeholders mentioned that a large percentage of nurses and nursing managers identify as Latinx and speak Spanish. To help all learners feel comfortable confidently engaging with the material, it could be helpful to provide the training course in English and Spanish.

Results 

Nate (pseudonym), our client liaison, shared the executive instructional design (ID) report with Janet (pseudonym), the project sponsor. Implementing the lesson plan was not a goal of this project. However, Janet and Nate noted they would do their best to move forward with implementing the training course we created. 

References 

Merriam, S. B., & Bierema, L. L. (2014). Adult learning: Linking theory and practice. (1st ed.).

OPWL 537 Course Instructors (2021). Instructional design course handbook (4th ed.). Department of Organizational Performance and Workplace Learning, Boise State University.

Rothwell, W., Benscoter, B., King, M., & King, S. B. (2016). Mastering the instructional design process: A systematic approach (5th ed.). John Wiley and Sons.

Rothwell, W. J. (2020). Adult learning basics. (2nd ed.).

Watkins, R. (2006). Aligning Human Performance Technology Decisions with an Organization's Strategic Direction. In Pershing, J.A.  (Eds.) Handbook of Human Performance Technology : Principles, Practices, and Potential (3rd ed.). Pfeiffer.

https://ebookcentral.proquest.com/lib/boisestate/detail.action?docID=258899