Evaluation

Collection and use of data is essential to the success of out-of-school programs, including afterschool and summer programs.

K-6 21st Century Programs Performance Objectives (PPOs)

PPO #1: 75% of 21st Century 3rd-6th-graders, attending 30 plus days, will improve their performance on Measures of Academic Progress (MAP), demonstrating above expected ELA growth. Staff will compare Fall, Winter and Spring benchmarks. The MAP uses national norms, and the growth score predicts Ohio ELA results with a rate of .70 to .85 (NWEA, 2018). (Interim Indicator)

PPO #2: 75% of students, attending 30 plus days, will move from Limited to Basic, and Basic to Proficient or above on the 3rd-6th-grade Ohio ELA tests. Since Spring ELA assessments are canceled due to the COVID 19, we are predicting student scores from MAP scores. 100% of 90 targeted students will be projected at BELOW Proficient. In setting academic goals, we expect students will likely return in 2020-21 even further behind due to difficulties in remote learning and family instability. We project higher rates of domestic violence, child abuse, poverty, and alcohol/drug abuse may be occurring (APA, 2020). COVID 19 surges in 2020-21 may result in further school interruptions. If our proposed goals do not match current realities, we will readjust targets in renewal grants with ODE’s permission. (Longer-Term Indicator)

PPO #3: 75% of 3rd-6th graders, attending 30 plus days, will show growth above the mean national norms, or 1.25-1.5 or higher, on the math MAP. MAP will be given in the Fall, Winter, and Spring, and is predictive of Ohio math tests with an accuracy rate of .77 to .89 (NWEA, 2018) (Interim Indicator)

PPO #4: 75% of students, attending 30 plus days, will move from Limited to Basic and Basic to Proficient or higher in Math over the grant period. Since 100% targeted 3^rd-6th graders are BELOW proficiency, we are tracking incremental change. However, 2019-20 results will not be available due to COVID 19, so we will use Fall MAP predictions. (Longer-Term Indicator)

PPO. #5: Classroom teachers report 80% of students, attending 30 plus days, are improving or maintaining high-level homework completion, class participation, and positive behavior, etc (Longer-Term Indicator) .

PPO #6: 82% of students report 0 to 1 discipline referrals. (Longer-Term Indicator)

PPO. #7: 80% of students attend 85% of afterschool days. (Longer-Term Indicator)

PPO #8: 85% of students (attending 30 plus days) maintain a high level of school attendance. (Longer-Term Indicator)

PPO #9: 80% of students increase scores or retain positive scores on the Devereux Student Strengths Assessment when pre-test is compared to post-test. DESSA is a norm-referenced scale with a social-emotional composite and meets reliability and validity standards (Nickerson & Fishman, 2009; LeBuffe et al., 2009). (Longer-Term Indicator)

PPO #10: 75% of the parents/caregivers, attending 3 plus activities, will show evidence of a moderate to high-level of engagement as measured by a modified version of the Parent Involvement instrument. This survey was used in the Fast Track project with 900 youth from multiple states and experts from Vanderbilt, Penn State, Duke, and Washington State (Malone, 2000). The survey asks questions about parent involvement in volunteering, school activities, reading to children, etc. (Longer-Term Indicator)

Intermediate and longer-term outcomes are the result of day-to-day progress that should be monitored weekly and monthly as noted below. Collection of data shows progress on these items and helps shape interventions.

Are students engage, attentive, and eager to attend?

Is progress being made in quizzes, online assessments, graded work, etc.

Are students missing days either in the program day or classroom day?

Have discipline referrals occurred?

Do parents attend family education events or followup with family support services?

Are staff engaged, excited, attending well, and showing a high-level of fidelity to the program?

Are we reaching the number and type of students (e.g. not proficient & economically disadvantaged) we proposed?

These answers help identify program strengths or weaknesses, which must be addressed.