This section presents an overview of the evaluation strategies of the four example projects discussed in these modules, and offers examples of the tools and criteria used in those projects. Reviewing these examples gives an idea of approaches to using evaluation to understand the impact of an intervention on a specific target behavior.
Evaluation Strategies for Campus Intervention Projects
This section presents an overview of the evaluation strategies of the four example projects discussed in the curriculum, and offers examples of the tools and criteria used in those projects. Reviewing these examples gives an idea of approaches to using evaluation to understand the impact of an intervention on a specific target behavior.
Hypothesis: Signage that encourages not taking more food than is needed will reduce the amount of food waste
Data Collection: (1) Food waste will be measured before and during the implementation of the intervention, (2) Diners will be interviewed to see if they noticed the signage and reflect on if it influenced their decision to reduce food waste or not
Data Analysis: (1) Data on the weight of food waste will be compared by taking averages of multiple trash bins and comparing each day of the intervention to see if it is decreasing, (2) Interviews will be judged to determine how relevant the signage was in food consumption and waste decision-making
Hypothesis: Learning about the health benefits and availability of plant-based meals, along with being nudged to take action, will increase consumption of plant-based meals
Data Collection: (1) After the intervention, the diners will be observed to see if they choose plant-based meals and other consumption choices, (2) These same diners will be interviewed to understand their decision-making processes and motivations related to their actions
Data Analysis: (1) The observation data will be quantitatively analyzed to see what portion of diners who experienced the intervention acted on the nudge, (2) Interview data will be reviewed to determine themes that suggest relevant insights about why diners acted as they did and if they were influenced by the intervention
Hypothesis: Posting clear signage that informs about the economic and environmental benefits of reusable cups, along with reusable cup giveaways, will decrease the use of single-serve coffee cups
Data Collection: (1) Categorized sale data, both prior to and throughout the intervention, will be collected, (2) Interviews will be conducted with a random sample of customers about their response to signage and decisions on beverage containers
Data Analysis: (1) Statistical tests will be run on the sales data to check for significant differences occurring in association with the intervention, (2) The team will look for themes in the interview data that suggest drivers, including the intervention, and barriers to increasing the use of reusable coffee cups
Hypothesis: Building awareness about composting options and stations through dining table-top signage will increase the amount of composted food
Data Collection: (1) Random-sampling of diners, participating in a survey, (2) The survey will ask about their motivations to compost and reasons why they did or didn’t, including reference to the intervention materials
Data Analysis: (1) Data will be organized into quantitative categories that will be compared with pre-intervention surveys, (2) Open-ended responses will be categorized into themes to add nuance to the quantitative data
Data Collection Tools
Observation may be an effective way of evaluating the behavior of your target audience in order to inform the design of your intervention. This example shows some key questions and potential categories of behaviors that might guide your observation efforts.
Being able to collect data after your intervention has been implemented is important to understand its impact and potential for improvement. This is an example of a post-intervention survey that could be compared with data collected before the intervention to understand what changes have occurred.
Approaches to Data Analysis
It is important to consider not just what data you want to collect but how you will interpret it in order to give you feedback on your project. This example shows some pre-determined indicators that could be applied in data analysis to understand the meaning of survey results.
This is an example of thematic analysis, a way in which qualitative data is reviewed to discover themes in the information that is gathered. Using an established method of qualitative data analysis can help to uncover key insights and examples of behaviors or perceptions of the target audience.