Semillas to Sunflowers

Summer 2022 Program Evaluation

""Hummingbird. Huitzitzilin. Colibrí. Chuparosa. Now you know what to call it in English, Nahuatl, and two ways in Spanish."

- counselor to students during morning meeting

This summer, Life Lab hosted about 150 students from Pajaro Valley Unified School District (PVUSD) for immersive, week-long experiences that cultivated children’s love of learning, nourishing food, and nature in their special location at UC Santa Cruz. The Life Lab Garden Classroom at UCSC was a welcoming and inspiring place for students during this pilot program called Semillas to Sunflowers. The joyful, nature-based experiences on a university campus provided opportunities for kids from local communities who had been historically underrepresented in Life Lab programs to develop a sense of belonging in a college setting, to build community with one another, and to exercise their bodies and minds.

This report is the result of a study undertaken by the Educational Partnership Center at UCSC with the goal of learning more about Life Lab’s programming. As suggested by the quotation above, we observed an exceptional program that intentionally created spaces for PVUSD students, the majority of whom are multilingual learners, to feel seen, represented, and understood.

Methods

The Educational Partnership Center at UCSC collected two types of data for this evaluation: pre- and post-surveys of students’ attitudes and beliefs and ethnographic field notes. All data were collected during the final three weeks of the six-week Semillas to Sunflowers program because the first three weeks were half-day camps and the latter weeks featured all-day programming for students. The field notes were written by graduate student researchers after each of four 3-hour observations of mid-week activities in the garden. The pre-survey was given to students on their first day of the program at the meet up site, Mintie White Elementary School, while they ate breakfast and played. The post-survey was administered on the last afternoon at the Life Lab Garden Classroom. The survey (adapted from Cheng, & Monroe, 2012) consisted of nine statements to which students indicated their agreement (5-point Likert scale). The post-survey contained an additional item: “I feel comfortable being on the UCSC campus.” The students were given the option to complete the survey in English or Spanish.

Using the Wilcoxon signed-rank test, we see a significant difference between the pre- and post-surveys, Z=-3.1, p < .01. Furthermore, the sample size was sufficiently large to enable a paired samples t-test which compares the means of two measurements taken from the same individuals at two different times. The purpose of this test is to determine whether there is statistical evidence that the mean difference between paired observations is significantly different from zero. Based on paired t-test results, we can say that there is a significant mean difference between pretest and posttest scores (t₇₁ = -2.81, p < 0.01). In our sample, on average, posttest scores are 2.23 points higher than pretest scores. To put this number in context, because there were 9 items on both the pre- and post-survey, the highest possible score was 45 (strongly agree =5). So, on average, the post-survey scores were 2.23 points higher overall. We can go on to say we are 95% confident that this increase will be between .65 and 3.81 for any similar population of students (95% CI [.65, 3.81]. In sum, we can say that participation in the Semillas to Sunflowers week-long summer program is associated with a small increase in students’ agreement with statements measuring their attitudes toward nature, new foods, and being a scientist.

"I'm glad my mom made me do this program."

- overheard in the garden

Individual Item Results

In this section, we review the survey results for each item in the pre- and post-survey and supplement these with vignettes from the ethnographic field notes. As a reminder, there were nine matching statements in the surveys with five possible responses (1 = strongly disagree, 5 = strongly agree). For every statement, the mean score in the post-survey was higher than that in the pre-survey. Two items showed statistically significant differences: “I feel comfortable in nature,” (t65 = 3.48, p < 0.01) and “People are part of nature,” (t65 = 2.54, p < 0.05). We can therefore conclude that participation in the summer program positively affected students’ attitudes and beliefs about nature.

I feel comfortable in nature

One of the paramount goals of the Semillas to Sunflowers program is to provide children with positive experiences in nature. Feeling comfortable in nature is a good measure of this and is an important foundation for students’ garden-based learning experiences in their schools. This item showed the largest effect size (Cohen’s d = -.431) in our study. This suggests that participating in the program increased students’ feeling of comfort in nature to a moderate degree. The mean score in the pre-survey was 4.28 and the mean score in the post-survey was 4.71, a difference of .43 points.

Getting comfortable with chickens - an observation in the garden

"Deep breath! Are you ready?"

An instructor held a chicken in her arms and leaned in toward a student seated on the ground. Minutes earlier, the student had declared that she was afraid to touch the chickens, but watching her classmates interact with them had piqued her curiosity. Still, she let out a nervous shriek and pulled her knees toward her body.

“You’re going to give our chicken, Captain Hairdo, a big, gentle, loving hug,” the instructor said in a steady tone. The student eyed the animal cautiously, then took an audible breath in and slowly released it. Her classmate watched with rapt attention. The instructor passed the chicken into the student’s arms. After a moment, the student smiled and let out a squeal.

“Can you support Captain Hairdo by feeding him?” the instructor asked the student sitting next to her. He picked up a carrot and turned its leafy head to the mohawked chicken. “He just took a bite!” another classmate exclaimed.

As the scene unfolded, the once anxious student began to relax her body into her seat and pulled the chicken closer into her. “You see?” said her classmate beside her. “When you’re scared, the chickens are scared. When you’re fine, the chickens are fine.”

I like to care for animals

This item showed the smallest effect size (Cohen’s d = -.021) in our study. This very small effect size is possibly due, at least in part, to the very high scores on the pre-survey. The mean score in the pre-survey was 4.75, meaning most students agreed or strongly agreed with this statement at the start of the program. There was very little room for growth in the post-survey (mean = 4.77).

I like to care for plants

This item showed a very small effect size (Cohen’s d = -.161) between the pre-survey (mean = 4.48) and post-survey (mean = 4.63). Again, many students agreed with this statement at the start of the program. Because of the small effect size, we cannot say that the program affected students’ attitudes on this item.

People are part of nature

The Semillas to Sunflowers program sought to help children understand how people are a part of the interconnected web of life. Humans can have profound impacts on the natural world, for good or ill, and the program gently teaches students how to be respectful stewards through a variety of experiences. This item showed the second-largest, statistically significant effect size (Cohen’s d = -.314) between the pre-survey (mean = 4.12) and post-survey (mean = 4.52). We can say the program had a small to moderate effect on students’ agreement with this statement.

Composting - an observation in the garden classroom

A group is gathering under the trees. It's time for a midmorning snack of strawberries. The students can have as many as they'd like, and indeed many go back for seconds and thirds. A discussion starts about whether the leaves on strawberries are edible. Children and counselors alike share their knowledge and, together decide that, while you’d have to eat a lot of strawberry leaves to get sick, you probably shouldn’t. The lead counselor announces, “So we will eat the red, juicy part and share the leaves with the worms,” referencing the compost buckets. The students agree and spend the next fifteen minutes happily munching on fruit, chatting with friends, and playing in the dirt with sticks. Students pass the compost buckets and drop the leaves inside, looking into the ever-growing pile of red and green. This lesson, like so many we observed, gently enforces the message that humans are a part of the natural world. We share our leaves with the worms who will turn them into the rich soil which we will use to grow more strawberries.

The next four survey items all showed fairly small, non-significant effect sizes. This does not mean that the results are not interesting or important, only that we cannot make any claims about the Semillas to Sunflowers program affecting students’ attitudes.

People need plants and animals to live

I want to learn more about things I notice in nature

I like trying new foods

I like to learn about what I eat

I can be a scientist

One important goal of the Life Lab garden program is to provide students with experiences that are both educational and help to encourage in children a positive self-concept in science. Indeed, school-based science experiences and self-concept are strong predictors of students’ aspirations to pursue science in their future. This is especially vital for children from communities that are underrepresented in STEM fields - people of color, non-native English speakers, people from low socioeconomic backgrounds, women, etc. The belief that one can become a scientist is the essential first step.

Though the difference in means for this survey item was not statistically significant, it was notable because it had the lowest pre- and post-survey scores. As a group, the students who participated in the summer program did not see themselves as scientists before or after participating in the program.

I feel comfortable on the UCSC campus.

Just as seeing oneself as a scientist is an important predictor of future science course-taking and the pursuit of a career in STEM, feeling comfortable on a university campus is a strong indicator of a students’ likelihood of applying to and attending post-secondary school. This statement only appeared on the post-survey, and students overwhelmingly reported feeling comfortable being on the UC Santa Cruz campus. Indeed, the mean (4.82) is the highest of any item in this study; almost every student either agreed or strongly agreed with this statement. Notably, this item was also the only one that students spontaneously drew stars, flowers, and happy faces next to their response. This should be considered a great success. Most of the students had never been on the university campus before. In just one week, Semillas to Sunflowers created a positive impression of the university and fostered a feeling in students that they belong here.

Other Findings

There were some findings that emerged from the ethnographic observations that were not related to items in the surveys. In particular, all researchers documented observations of safe risk-taking and teamwork. A particular benefit of the Semillas to Sunflowers program is the small-group context that provides students opportunities to cultivate these life skills. These are especially crucial experiences for children in the context of re-opening after two years of emergency remote instruction during the Covid-19 pandemic.

"Only do it if you feel safe!"

- counselor advising a student contemplating a leap off a wall

Throughout our observations in the garden classroom, we noticed instances in which children were encouraged to take risks with their bodies. From climbing trees to using an unfamiliar tool to holding a chicken, the student participants in the program were encouraged and supported to try new things. The counselors consistently invoked a message of trust in the students to keep themselves and others safe. As a result, we observed many small instances of children engaging in safe risk-taking.

Cultivating safe risk-taking - an observation in the garden

As the students finish up their mid-morning snack of strawberries, I notice a student playing with a sturdy stick, drawing and digging in the dirt beneath their bench. I look around and see that sticks are quite popular today. Many students seem to have collected sturdy sticks and are playing with them in a similar way. A few students work together to build a small structure out of their sticks. As snack ends and the large group disperses back into small groups of four to six students and a counselor, the sticks go too. This is notable - playing with sticks is not typically "allowed" in school-like settings. The counselors in the garden welcome this exploratory behavior though, saying only "Abajo y suave. Low and slow."

The group moves on to the next activity which is creating nature art. Entering the outdoor classroom space, the students are given ten minutes to explore. There is a huge tree with wide, spreading limbs and the children want to climb it. Once again, the counselors allow the garden environment itself to teach and advise the students, "Only go as high as you feel comfortable. Listen to your body. If your body says stop, listen to it." Many happily climb and no one gets hurt.

"Teamwork!"

- student exclaiming while making popsicles with their group

We also observed many small instances in which the idea of teamwork was promoted and enacted. The students are organized into small groups of four to seven students and a counselor. The counselors spend time each day building community amongst the students through name games, group challenges, and intentionally-created tasks that require multiple hands and minds. Again, it is important that students have the opportunity to build social bonds and solve problems collaboratively

Cultivating teamwork - an observation in the garden

With the smell of fresh air of a late morning under cloudy weather, a group gathers around a picnic table. The project is making shortbread, and the counselor gives tasks to each student. Handing over a grater and a lemon, one student is offered the task of making lemon zest. He shakes his head in a doubtful no. The instructor asks another student to share the task. The uncertain student holds the grater while another student tries to rub the lemon. It turns into a team challenge to make zest. After so many trials and failures, switching roles, they figure out the best angle to hold the grater. It's a fun task to have more and more zest.

Conclusion

We saw evidence that the Semillas to Sunflowers program positively impacted the attitudes and beliefs of elementary-age children and provided them with exceptional opportunities for learning and growth in an environment that centered care. Throughout our observations, we noticed counselors skillfully facilitating groups of excitable youth in ways that affirmed the students’ experiences and emotions and promoted agency and teamwork. The small-group format created an environment in which all student voices were heard and authentically considered. In this environment, the students engaged in a plethora of activities that were place-based, exploratory, and often centered service to the community (e.g., making popsicles for everyone, creating “graduation” bouquets for the end of the week).

Analysis of the pre- and post-surveys showed a statistically significant overall increase in students’ positive attitudes and beliefs about nature. The largest and most significant gains were in response to the statements "I feel comfortable in nature," and "People are a part of nature." Though there were no statistically significant gains for the other survey items, the mean for each item was higher at the end of the week. There is strong evidence that participation in the program supported students’ sense of belonging in nature and at the UCSC campus. A potential area for growth suggested by the survey is around students’ self-concept around science.

We believe these gains in students’ positive attitudes about nature cannot be explained only by the garden classroom alone - though the space is mindfully created to spark curiosity and delight at every turn. It seems to us that the skillful guidance of the counselors in a small-group setting were likely crucial factors in promoting students’ sense of comfort and belonging.

"Things that might be distractions are actually instruction because the garden space itself provides the learning opportunities."

- one researcher to another after an afternoon of observations