The Purpose of Evaluation
Every program should have a solid evaluation plan in place in order to measure the effectiveness of the program as well as offer guidance to help us enhance the program while it is being implemented and developing.
The Developmental Evaluation Approach
We have decided to use the developmental approach to evaluate the CONNECT Program at TREC. This approach is favorable since CONNECT is a new program and Developmental Evaluation is particularly useful for "emergent and dynamic realities in complex environments."(Better Evaluation) It is important that measures can change during the evaluation as the process unfolds. It facilitates real-time feedback to program staff which will allow us to track developments, identify surfacing issues, and make appropriate changes to the program in its early stages. We will use evaluation forms that are user-friendly, often quick, diverse and easy to administer. We will also collect both quantitative and qualitative data through surveys to staff, students, and other community members through electronic formats like Google Survey and Poll Everywhere. Individual interviewing will also be utilized for the collection of qualitative data.
It is also important to choose an approach that allows for an internal evaluation team, rather than an outsider, since relationship and trust are foundational pieces in trauma informed practice and working with vulnerable, complex student populations. We know that if the "level of trust is low in a relationship or organization, people limit their involvement and what they are willing to do or share."(Jaffe, Dennis. 2018) So, the developmental approach is better suited to an internal evaluation team with whom students have already formed trust, thereby enabling students to be more open and honest when sharing feedback. Developmental Evaluation also suits the dynamic qualities of CONNECT because it, "Interprets development through a complexity lens, recognising that situations are often uncertain, emergent and dynamic, and evaluation is responsive to this reality."(Developmental Evaluation Institute) The leadership involved with CONNECT will require an evaluation strategy that is, "aligned with their values and far-reaching visions, the complexity of contexts in which they work, and the emergent and organic process of development that inevitably occurs. They need evaluation to be collaborative, build capacity and put a strong emphasis on trusting and the building of relationships." (Developmental Evaluation Institute) Working and evaluating in an alternate school program requires flexibility and innovation. Period.
Evaluation Questions
(P=Process evaluation I=Impact evaluation)
Attendance Questions:
(P) To what extent does CONNECT address Friday attendance problems? How many students are attending on Fridays? Did attendance increase?
(P) Are there any common characteristics between the students who are attending? (gender, age, culture, etc.)?
(P) Which options are best attended?
Belonging Questions:
(I) How much connection is happening between students in the activities? Are they working with others or feeling left out?
(I) What do students think about the CONNECT program? What do they like? What do they struggle with?
CORE Competency Questions
(P) How well does the CONNECT align with the three CORE competencies: Communication, Thinking, and Personal/Social competency?
(I) Is there any observable growth in their CORE competency skills?
Operational Questions
(P) How is CONNECT functioning from administrative, organizational, and/or personnel perspectives?
(I) Are staff feeling adequately supported to fulfil their CONNECT goals and pursuits?
(P) How were students made aware of the program?
(P) What resources does the project/ program need?
(I) How do the community partners feel about the program?