Data Collection & Analysis

Data collection, validation and analysis are important stages of the evaluation process. As Huey-Tsyh (2005) explains, "evaluators both build the capacity of the program and its stakeholders to collect evaluative data, and show them how to interpret and use their data correctly and meaningfully” (p. 181). For this reason, this step needs to be carefully thought-out and planned so that data is high-quality as it adheres to the following validity, reliability, completeness, precision, integrity, timeliness. (Unicef Innocenti, 2014)

Data Collection

As a starting-point, namely to ensure that nothing is overlooked, a data-collection matrix was created to organize the types and sources of data for the evaluation of The Grocery Foundation's School Breakfast Program. With the key evaluation questions in mind, the data is not only a mix of both qualitative and quantitative types, but its credibility is heightened by the variety of different sources from which it will be collected. Better Evaluation. (n.d). refers to this type of mixed data collection as "Parallel Data Gathering" and asserts that this method contributes to a well-rounded evaluation. With that said, this is the data-collection matrix for this project:

Data Validation

In terms of data validation, the types and sources of data will be combined with the goal of balancing the strengths and weaknesses of every individual aspect. Bamberger (2012) explains that the purpose of using mixed data types and sources is to "strengthen the reliability of data, validity of the findings and recommendations, and to broaden and deepen our understanding of the processes through which program outcomes and impacts are achieved" (p. 1).

Specifically, the data types and sources will be validated using triangulation. Triangulation will work well since it "facilitates validation of data through cross verification from more than two sources" (Better Evaluation, n.d). If we look at the matrix above, for example, many of the qualitative data sources will come from program participants and/or people involved in the operations of the program, which may present some inherent bias. Triangulation will help minimize bias that may result from measurement, sampling or procedure. Most importantly, it will help to enrich, refute, confirm or explain some of the findings.

For example, if we look at data sources for the sixth key evaluation question, "Does the program's effectiveness differ depending on the specific location it serves?", they are both quantitative and qualitative. The data for this question will be gathered from quantitative (demographic statistics, funding by program location, attendance records, etc.), as well as qualitative (observations as various program locations) information. Thus, when drawing conclusions in order to answer this key question, there will be multiple layers of information that can be used to enrich and explain the findings to the stakeholders.

Data Analysis

Patton (2002) says that the goal of data analysis is "to uncover emerging themes, patterns, concepts, insights, and understandings" (p. 37). Once the aforementioned types of data and its sources are validated using triangulation, the next step will be to not only summarize it, but to look for and, hopefully, identify patterns and themes.

Because the data collected for The Grocery Foundation's School Breakfast Program is both qualitative and quantitative, the data will be analyzed using the following methods, which are specific to the data category:

• For the analysis of quantitative data (e.g. financial statements), summary statistics will be used. Due to the very visual nature of summary statistics, they will be useful for quickly comparing change over time. This type of analysis will "provide information about how much variation there is in the data" (Better Evaluation, n.d).

• For the analysis of qualitative data (e.g. the interview responses), content analysis will be used. This will not only allow for the vast amount of qualitative data to be condensed into a more manageable amounts and formats, but it will once again link them back to the evaluation questions. This will be extremely useful in providing "insight into complex models of human thought and language use" (Busch et al., 2005).

Ultimately, regardless of the method(s) used for the analysis of data, all methods are conducted with the same end-goal: to find meaning. Suter (2012) affirms that "The analysis of rich descriptions occurring throughout the course of a project often provides new perspectives, and its analysis of interconnecting themes may provide useful insights. The depth afforded by qualitative analysis is believed by many to be the best method for understanding the complexity of educational practice. Qualitative analysis is also well suited for exploration of unanticipated results. Above all else, it is concerned with finding meaning embedded within rich sources of information" (p. 356).