For the evaluation phase of the project, a three-tiered evaluation system was utilized: (1) an expert review (Evaluation Form A) in which an expert in urban gardening and instructional design reviewed the Seeds of Change booklet based on its content, instructional design, and technical design; (2) a learner pre-test (Evaluation Form B1) and post-test (Evaluation Form B2), which determines the knowledge of the selected learners before and after consuming the booklet to measure learning gains in the form of a 10-item multiple-choice quiz; and (3) learner perception (Evaluation Form C), which focuses on the clarity, visual appeal, and the overall effectiveness of the booklet.
For the expert review, CAFS associate professor Dr. Sheryl Iballa was tapped to evaluate the Seeds of Change booklet, as suggested by the author's external advisor or gatekeeper. Dr. Iballa noted a plethora of deficiences, detailed in her comments on the PDF file of the booklet. The booklet did not achieve two of the desired learning objectives: appraise methods of urban gardening based on viability in various settings and produce a small-scale urban garden. Morever, she noted several inconsistencies on the usage of the terms "urban gardening" and "urban farming", as well as subtopics in the booklet's table of contents and the subheadings of each topic. She also saw the need of captions for some images for added context, as well as the need additional explanations for common plant pests and diseases and detailed step-by-procedures of learning activities with appropriate images. Additionally, she suggested that some key words be highlighted whenever they are introduced and defined in the booklet.
On August 24, 2024, the "Evaluation Day for Learners" took place. Prior to this, an ocular inspection of the site where the evaluation day was held, took place in "Riverside", a residential area near Molawin Creek, which best represented the target learner demographics of the booklet: low- to middle-income (annual income below PHP50,000.00), has some interest in urban gardening, and at least a small space for planting a garden.
15 residents of "Riverside" by Molawin Creek took part in this evaluation day. Before and after the distribution of the booklets, which were printed in a copy shop in Los Banos, the residents were given pre-tests and post-tests, respectively, to measure their knowledge and understanding of the topics that were to be presented in the booklet. The results of the post-test compared to the pre-test were somewhat surprising, with some receiving lower scores, highlighting a limitation of the study, in which the instructional materials were published in English. Not all residents had a sufficient understanding in the English language; the media reviewer pointed that out when she evaluated the booklet with Evaluation Form A.
For Evaluation Form C, which was provided to the learners right after the post-test (Evaluation Form B), a form with eight questions related to instruction design and five questions on overall content evaluation was administered. The first part, which focused on the instructional design aspect, was a modified 4-point Likert scale--1 being the lowest and 4 being the highest--that analyzed the booklet based on the following areas: (ID1) clear, understandable language; (ID2) contribution to knowledge and skills in urban gardening; (ID3) relevance and applicability of topics discussed; (ID4) ease of use and navigation; (ID5) logic and organization of content; (ID6) visual design, i.e. typography, color, contrast; (ID7) alignment of graphics with the topics being discussed; (ID8) absence of factual and typographical errors. No "neutral" option was included in the Likert scale as it could mask the respondents' true opinions.
86.67% (13) of learners who read the booklet either agreed or strongly agreed that the language used there was clear and understandable. 93.33% (14), on the other hand, positively agreed that the booklet contributed to their knowledge and skills in urban gardening, discussed "relevant and applicable" topics and skills, was deemed easy to navigate and use, was organized in a logical manner. Moreover, 13 learners also had a positive perception with the appropriateness of the images and illustrations used in the booklet, as well as the absence of factual inaccuracies and typographical errors.
The second category of the learner perception evaluation, consisted of topics that the participants enjoyed the most, as well as their ratings of the booklet and their likelihood of recommending it to others. All 15 participants have stated that they are positively likely to recommend the booklet to other people, a stark contrast to the results of their post-tests.