Embedded Files
Before the Evaluation
Before conduct our final evaluation, we decided to use Kirkpatrick Model for evaluation. The reason we chose the Kirkpatrick Model for evaluating training is that it is highly beneficial, offering a holistic approach to understanding the effectiveness of the training across multiple dimensions. We also involve rapid prototyping and user testing for testing the flow of the learning material.
Goal of the Evaluation: The primary goal is to assess the effectiveness of the training program. Also a internal check for whether the training website works well.
Requester of the Evaluation: This evaluation might be requested by the school district's administration, the educational board, or a funding agency interested in understanding the impact and effectiveness of their investment in teacher professional development.
Type of Evaluation: We will use Kirkpatrick Model which include both summative and formative assessment.
Types of Data Needed: Data include pre-test for checking prior knowledge about Gen-Ed teacher and a post-test after training. And a usability test for website testing.
Instruments for Data Collection: Google Form and google doc for survey. Record for usability test.
Data Analysis: Quantitative data (e.g., test scores) will be analyzed using statistical methods to identify significant changes or trends, while qualitative data (from surveys, interviews, and observations) will be analyzed thematically to extract common themes and insights.
Kirkpatrick Model
Level 1: Reaction —— How do learner react to the learning design?
Level 1: Reaction —— How do learner react to the learning design?
This formative evaluation question will help us improve the design based on findings.To anwser this question, we will conduct a usability test with 3-5 learners, using a combination of observation (with prototype and think aloud) and interview. Observations will help us identify issues that create frustration or confusion. Interviews will help us ask questions about things that aren’t represented in the prototypes or ask learner to elaborate on things we/they observed.
Level 2: Learning —— To what learner do teachers acquire knowledge, skills, and attitudes as a result of the training?
Level 2: Learning —— To what learner do teachers acquire knowledge, skills, and attitudes as a result of the training?
This formative evaluation question will help us measure whether learning materials and content need to be adjusted based on findings. To anwser this question, we will conduct an interview or survey before and after each session. This will help measure the level of learning and confidence in the subject matter.
Level 3: Behavior (transfer of training) —— Following the completion of the training, have general education teachers implemented new strategies in their classrooms to effectively support ELL students?
Level 3: Behavior (transfer of training) —— Following the completion of the training, have general education teachers implemented new strategies in their classrooms to effectively support ELL students?
This summative evaluation question will help us determine the long-term effectiveness of the training program designed for general education teachers to support ELL students without the need for specialized ELL instructors. To answer this question, We will conduct follow-ups several months after the training by using retrospective survey. The survey will involve supervisors, peers, and ELL students and will ask about changes after the training, including whether new strategies have been implemented in the class.
Level 4: Result —— How has the training of GenEd teachers impacted the academic outcomes and school-wide performance metrics of ELL students?
Level 4: Result —— How has the training of GenEd teachers impacted the academic outcomes and school-wide performance metrics of ELL students?
This summative evaluation question is designed to assess the long-term impact of training general education teachers to support ELL students without specialized ELL teacher assistance, focusing on academic outcomes and overall school performance metrics for ELL students. By answering this question, we will collect and compare pre- and post-training academic records and school performance indicators. We will also incorporate feedback from teachers, school administrators, students, and their parents to gauge the training's specific impacts on teaching practices and student learning experiences. Additionally, conducting long-term follow-up studies will help us assess the sustainability and broader effects of the training, enabling us to make informed adjustments and enhancements for future training programs.
Reference
Dirksen, J. (2016), Design for How People Learn. New Riders. Chapter 12: Designing Evaluation.
Determining the Success of an Instructional Design Product and Process, Chap 10, pgs 163-179. The Essentials of Instructional Design : Connecting Fundamental Principles with Process and Practice, Brown & Green (2015).
Kirkpatrick, D. L. (1959). Techniques for evaluating training programs. Journal of the American Society of Training Directors, 13(11), 3-9.
Page updated
Google Sites
Report abuse