PME 802 | Program Inquiry & Evaluation

For this course, the key task is building a Program Evaluation Design plan (PED) for a social program we would like to evaluate. Please see the details below.

Context

I have the privilege of working at a school accredited by Canadian Accredited Independent Schools (CAIS). This national association of close to 100 schools represents the top independent schools in the country and to be formally accredited involves a rigorous process. As CAIS writes:

"In Canada, education is provincially mandated, and CAIS Schools are required to meet Ministry requirements. However, CAIS schools provide more. Our National Standards and Procedures are congruent with the internationally accepted criteria and model core standards adopted by the NAIS International Commission on Accreditation and updated annually based on current research and exceptional practices." (source) 

In brief, the process of accreditation takes the form of four components (see CAIS accreditation page for more information and the source of the quotes used below):

1. Internal Evaluation

The school seeking to be accredited or reaccredited drafts an extensive internal report on the its "strengths, weaknesses, future strategic plans, and examining how well its program fulfils its mission." The finished report can run to hundreds of pages and takes most schools over a year to complete. [this element will be the subject of this Program Evaluation Design task]

2. On-Site Review

A Visiting Committee of leaders from other CAIS schools then visits the school seeking accreditation/reaccreditation with the objective of validating "the school’s Internal Evaluation and challenge them to reflect on where there are opportunities for growth".

3. Visiting Committee Report

The Visiting Committee then produces an extensive report containing "an overview of where the school stands in terms of each Effective Practice and offers commendations (what the school is doing exceptionally well), suggestions (best professional advice) and recommendations (areas of growth that require action by CAIS)".

4. Response Reports

The school seeking accreditation/reaccreditation will then produce a response to the Visiting Committee's report, articulating how the School will respond to recommendations made.

Description of Program Content

I will be looking at developing a Program Evaluation Design for the Internal Evaluation stage of my school's CAIS accreditation process. Each school in good standing goes through the reaccreditation process once every 7 years. Our school's internal evaluation is to be completed before the 2020-2021 academic year.

Creating an internal evaluation is a significant undertaking, as the school needs to provide meaningful and insightful feedback on each of 12 standards:

In order to derive the most benefit from the Internal Evaluation process, schools should create a strategy that solicits feedback on school programming from as many constituencies as possible, including faculty, staff, students, parents, alum, and members of the governance team (board of directors, trustees, etc).

For our CAIS accreditation, we are looking at engaging over 200 employees (faculty and staff), 580 students, hundreds of parents, board members and thousands of alumni.

The timeline for this process is 16 months:

  • Summer 2019: plan the review process
  • Fall 2019: create teams and subcommittees to tackle the 12 differents standards
  • Winter 2020: gather data
  • Spring 2020: start the writing process
  • Summer 2020: edit and finalize Internal Evaluation
  • Fall 2020: submit Internal Evaluation to CAIS for distribution to Visiting Committee

My colleague and I have been charged with managing the internal review process.

Purpose for evaluation and specific evaluation questions

In brief, the purpose of the exercise is to evaluate how well the School is meeting the 12 CAIS accreditation standards and what are the School's strengths and areas for improvement in light of these standards.

At this stage, the specific evaluation questions that I have are as follows:

  • How will we systematically engage all the many stakeholders within the internal and external school community?
  • How will we create a framework to encourage fellow colleagues to approach the evaluation as a positive exercise in self-reflection, as opposed to a burden or 'test' of the a staff member's individual program?
  • What software can we utilize to manage the overall project and securely store the significant amount of data the project will generate?
  • In light of already being very busy, how we will collectively find dedicated time to complete the Internal Evaluation process?
  • What theories of evaluation would be most helpful to frame this project and how may those theories be shared with colleagues tasked with leading different components of the Internal Evaluation?

It is my hope that answers to these questions will be forthcoming in the next several weeks.