Evaluation Report

Executive Summary

This report evaluates the effectiveness of a training program designed for general education teachers to support ELL students without specialized ELL teacher assistance. The findings reveal significant improvements in teachers' abilities to engage with and support ELL students, alongside identified areas for further content enhancement. Recommendations include expanding scenario-based learning and increasing follow-up support.

Evaluation Purpose

The two main objectives for the evaluation is: 

• Do teachers achieve the learning goals set by the program?

• Does the overflow and content of the website (learning material) work well?

Formative Evaluation Questions:

• How do students react to the learning design?

• To what extent do teachers acquire knowledge, skills, and attitudes as a result of the training?

• Is the overall website flow works well?

• What has the gen-ed teacher done well after each session? 

• Where are the gaps in their learning? 

Summative Evaluation Questions:

• Do teachers achieve the learning goals set by the program?

• What components of the training content are most and least effective?

• How has the training of GenEd teachers impacted the academic outcomes and school-wide performance metrics of ELL students?

• Following the completion of the training, have general education teachers implemented new strategies in their classrooms to effectively support ELL students?

• What should be the future steps to address any ongoing shortages of specialized ELL teachers?

Methodology

Participants: 

• 50 GenEd teachers across five schools participated in the training.

• 3 Expertise for usability test.

Instruments Used to Gather Data:

• Peer reviews and small tests (Formative Learner Assessment)

• Usability test (Formative Learner Assessment)

• Final test with a rubric (Summative Learner Assessment)

• Surveys, case studies, and structured interviews (Formative and Summative Program Evaluation)

Results

Learner Assessment (Formative):

• Peer reviews indicated that teachers felt more confident in their scenario-based activities by the third session.

• Small tests showed a 20% improvement in knowledge of ELL strategies.

Learner Assessment (Summative):

• 80% of the teachers met or exceeded the learning objectives according to the final tests.

Program Process Evaluation (Formative):

• The overall website flow is good and the content is making sense. But few problems we found through usability test below:

  1. The video should be paired with more content – activity after, knowledge check or text content.

  2. The scenario activity needs to be refined to account for data gathering / outcomes

  3. The self ranking of skills (knowledge check) at the beginning of the module should take some other format

Program Process Evaluation (Summative):

• Post-training interviews suggested that while teachers felt better equipped, a need for ongoing support and advanced training was highlighted to further reduce the dependency on specialized ELL teachers.

Conclusions and Recommendations

Conclusions:

• The training program has effectively enhanced GenEd teachers' capabilities in supporting ELL students. However, continuous improvement in training content, especially in cultural education, is necessary.

Recommendations:

• Introduce additional modules focusing on cultural sensitivity and diversity.

• Implement a quarterly follow-up session for the first year to reinforce learning and address new challenges.

• Explore partnerships with ELL specialists to provide periodic workshops.