Within the goals and objectives, two products are mentioned: a program description and a process evaluation plan. However, numerous other products were also produced including the creation of all survey tools used in collecting program data (e.g., pre-/post-term surveys for both staff and participants, weekly surveys to ask process evaluation questions to staff using quick feedback loops) (see Appendix A), surveys for activities used during WN live sessions (see Appendix B), creation of the program’s logic model (see Appendix A), analysis of the program data obtained through various data collection methods, use of the findings/results to produce a data report for the What’s Next Initiative (see Appendix A), two presentations given to the WN Team (see Appendix A), and the What’s Next Initiative itself. All products were created for the What’s Next Initiative and Boys and Girls Club of Hawaii. The What’s Next: Program Description and What’s Next: Process Evaluation Plan can both be found in the Appendices of the Agency Report (see Appendix A) and were both created following the Program Description and Process Evaluation rubrics from the MPH@Baylor University, PUBH 5360: Evaluation in Public and Community Health course (see Appendix C and D).
Each product was created through an iterative process with members of the What’s Next Team. All products were checked periodically with at least two WN staff members, one being either the WN Program Director Nadia Assaf or the WN Program Coordinator Bri Villarosa. These products are useful to the organization as they offer immediate, useable descriptions and evaluations for the What’s Next Initiative; meaning, the descriptions (e.g., goals and objectives, benchmarks) and evaluations (i.e., process evaluation components and measures) are all accurate, current, of APA quality, and follow an actual evaluation report template. If the What’s Next Initiative was to continue creation of an evaluation report for the brand new What’s Next Initiative, the first two sections would already be completed. That is a huge feat, especially when considering the process evaluation elements are largely finalized, with two Pilot cohorts already tested with little alteration required. It is recommended that the WN Initiative continue working towards completion of an evaluation report. The process evaluation measures are also recurring for each WN term, meaning the foundation for all survey elements are set in place, with little to no alterations needed. The program will use these methods to evaluate the program every term, consistently through the completion of the five-year Pilot.
The program description was completed over a timeframe of 23 weeks. This started with research being conducted on the areas involved, including the public health significance of the issue and evidence from academic literature to support it, the organization’s mission and social/political/economic climate, staffing structure, service population information and demographics, program goals and objectives, logic model, program components in terms of content and materials, summary of operational budget, and cost summary of program, among other topics. This required an extensive amount of research prior to beginning construction of the program description; once background research was completed, work on the program description document could commence. One section of importance was the creation of a brand-new logic model for the What/s Next Initiative. Before creation of the program’s logic model, a review of materials from the MPH@Baylor program curriculum and various external materials were conducted. Many program description aspects related to the organization and program were either non-existent or difficult to find, requiring there be time to navigate and locate or even create them entirely; one such section was the program budget and cost summary. As a budget summary was unable to be obtained from members of the WN Admin Team, a calculation of program budget had to be made using the available information (i.e., WN semester/cohort program plans). This required multiplication of the amount presented as the cost for various program materials by the amount of participants (25 estimated for each upcoming cohort) for each year of the pilot trail (5 years). Creation of the program’s logic model required a list of all program inputs, outputs, and outcomes (e.g., short, medium, long).
The process evaluation was completed over the course of 32 weeks. Lesser background research was required compared to the program description, but there were more materials using an iterative process that required approval with each step of their formation. Creation of the WN Teen and Staff Surveys (see Appendix A) begun with a list of potential questions, which were reviewed then selected upon by the greater WN Team. Drafts were corrected and molded until a final draft was agreed upon by all relevant party members. After a final draft was completed, external stakeholders were brought in to review and provide feedback on the draft. This new draft was administered to the WN participants and staff, and data was collected using a pre- and post-term survey. Once post-term survey data was collected, data was analyzed by Admin Intern Alexis Lam and included into a data report, which highlighted the progress towards the program’s measurable outcomes in addition to data for the main questions within each program Pillar. These steps and survey tools were all included into the What’s Next: Process Evaluation Plan to be presented as an internship/agency product at the end of the MPH@Baylor Graduate Project.
It is recommended that the WN Initiative continue use of the What’s Next: Process Evaluation through its five-year Pilot in quick feedback loop with both the teen and staff surveys, weekly staff survey, attendance records, and document review. Without this use of evaluation, it would be impossible to tell which components of the program are functioning efficiently and which aspects may not be working. This is how the program goals and objectives are measured, which are an indication of the positive or negative effect being had from the What’s Next Initiative on its service population. The What’s Next Data Report provides quantitative and qualitative data for the WN Teen Surveys, in addition to narrative details for every question and measurable outcome; these data reports will demonstrate trends in data between each term, across every question and subsequently, each program Pillar. Without the narrative data provided in the What’s Next Data Report, there would be no explanations as to the changes in WN survey data. This shows the progress of the WN Initiative on the service population and thus, the impact had by the BGCH on its service population. There is more information listed in the Agency Report (see Appendix A).
The five goals and objectives listed in the Goals and Objectives section of this report have all been completed within the 250-hour internship with the Boys and Girls Club of Hawaii. Goal #1 and its subsequent objectives was touched on during numerous meetings, the most prominent being the first data analysis meeting with external stakeholders Liz and Lala during Week 4 of the internship. Goal #2 and its associated objectives were touched on multiple times throughout the internship, with oral presentations being given at specified milestones (see Appendix A) and public health content being delivered in writing through the agency products continuously checked by the What’s Next Team. Goal #3 and its objectives were met among the program description research and creation, in addition to being touched on throughout the work required for development of the What’s Next Initiative. Completion of Goal #4 and its associated objectives was done during the weeks-long process of creating the survey tools needed to collect program data for the process evaluation. Finally, Goal #5 and its corresponding objectives was completed over various instances, but especially during the three data analysis meetings with external stakeholders Liz and Lala where they offered feedback on program aspects/components and offered advice on next steps with the program development process.