(Kirkpatrick, 2006)
Before starting our evaluation plan, we referred to the Kirkpatrick Training Evaluation Model, including 4 levels of evaluation phases:
Reaction: Measure if the learners have found the training to be relevant to their role, engaging, and useful.
Learning: Measure if the learners have acquired the knowledge, skills, attitude, and confidence.
Behavior: Measure behavior changes after learning and see if they transfer what they learn.
Results: Measure if the targeted outcomes resulted from the training program.
Since the Kirkpatrick model is a globally recognized method of evaluating the results of training, we decided to use the evaluation method for our design.
How did learners react to the learning experience?
The questions are to find out the learners’ reactions toward our learning design.
Think Aloud Protocol
Did the learners increase their understanding of the course registration process after the learning journey?
Did the learners increase their skills in using Albert?
The questions are to find out what knowledge and skills learners learn after the learning journey.
Knowledge check quizzes after each module
Were learners able to apply the knowledge to create their course plans?
Were learners able to navigate and use Albert for registration?
How do students rate their confidence levels in registering for courses now?
The questions are to find out if the learners are taking what they learn and applying it to register for courses.
Interview
Survey
Do learners know the degree requirements?
Do learners know the resources they can resort to for the registration?
Do learners understand the registration process?
Can learners navigate and use the Albert registration tools proficiently?
The questions aim to find out if the learning outcomes match our learning objectives/ goals.
Interview
Survey
2 current ECT students
The entire process was conducted online and lasted for 30-40 minutes.
Pre-learning experience survey
Post-learning experience survey
Interview Questions
Think-Aloud Protocol