Data Collection and Methodology

Connection to My Action Research Focus and Initial Wonderment

My proposed data collection process is directly related to my initial wonderment on this topic and and my guiding research questions. When I first began thinking of a topic for this project, I created several potential research questions. I ended up locating compelling previous studies that addressed the research questions that I refined and used to guide this action research. These questions centered on teachers' expectations and students' experiences with technology. When designing an intervention and deciding what to collect data on, I researched, adapted, and created methods that would allow me to collect data on teachers' reflections of their expectations, as well as students perceptions of technology and achievement when using technology. My first question will be addressed using qualitative analysis of voice thread responses in order to help me to gather data on how this intervention encourages teacher reflection. Addressing my second and third research questions (through the use of qualitative Flip grid responses and quantitative rubric scores) will help me to see whether or not my intervention changed students' perceptions of technology or achievement when working with technology.

Description of Methodology and Data Collection Phases

Data will be collected at several times throughout this action research process. Pre-action data will be collected first. Students will complete a project in which they integrate technology and create a video that describes and analyzes the term ‘volcanoes.’ I will collect data on students’ perceptions of this process by asking them to respond to a two-minute Flip grid video. I will utilize qualitative, inductive methods to code these statements in order to discover relevant themes or pertinent patterns. I will measure student achievement by assessing students’ videos with a scaled rubric. These scores will be reported as numerical values and quantitatively analyzed by finding the class' average score. It is also important that I observe students as they create their videos, taking notes so that I do not forget anything that I thought of. The contents of these notes will also be codified for themes and compared across the pre-test and post-test measures. I will also make note of focus students scores and comments in order to compare them to post-test data. Next, I will collect and analyze data from teachers’ responses to the ten reflective questions (that are recorded through voice thread). These responses will also be qualitatively analyzed and coded for themes. Although this data is not being tested twice and compared before and after an intervention, I will still be collecting an analyzing the data. In my experience, it would be a missed opportunity to ask teachers to reflect so deeply on their expectations and not explore the data. I will, at some point, build a separate, exploratory analysis or article in order to showcase these results. Finally, I will collect post- action data in order to measure any change that my intervention caused. All pre-action measures, including the Flip gird reflection and scaled rubric grading, will be repeated. The term volcano will be replaced with earthquake, so that students' prior instruction does not influence the results. This data will be analyzed through the same qualitative and quantitative measures in order to test the effectiveness of the intervention. For a specific discussion of the timeline with which this would be implemented, please visit the intervention page and read the timeline posted on the left-hand side.

Introduction to Data Collection Tools

In order to collect the data that I need to test the effectiveness of this proposed intervention, I will utilize a variety of tech tools.

Flip grid: This tool provides me with an opportunity to collect data on students' perceptions. I have designed Flip grid prompts for both the pre-test and post-test of students' perceptions. I have used this tool to create a reflection prompt to ask students what they thought about the process of using technology to facilitate the process of learning about a STEM concept. Important aspects of this tool include that it allows the teacher/researcher to set a time limit on responses and provides them with the opportunity to add a 'hint' to help facilitate answering of the question. In this case, I have provided students with a graphic organizer that explains the meaning of the word 'reflect.'

Scaled Rubric: Using the process endorsed by the California State University, Fullerton's credential program, I will create a scaled rubric to measure students' achievement, both on the pre-test video project and the post-test video project. Both the rubric and the Flip grid prompt will be the same at both times of measurement, to increase validity/reliability across tests. The only alteration on these tools will be the substitution of concepts from volcano (pre-test) to earthquake (post-test).

Voice Thread: Using voice thread, I will be able to capture teachers' responses to the the reflective questions, which represent my intervention. As I mentioned, this data is not being tested twice and compared before and after an intervention but, I will still be using the information to contextualize the pre-test and post-test data.. I do not want such deep reflective statements to be ignored, as they could yield information that may benefit the educational community. Later, I hope to complete an exploratory analysis of this information in order to understand, further contextualize, and publicize these results.

Informal Note taking: Phillips (2010) recommends taking notes while informally observing students as they complete tasks. I will follow this procedure and take informal notes as students compile both their pre-test and post-test videos.

Validity

The term validity refers to the fact that a tool measures what it has intended to measure (Fraenkel, Wallen, & Hyun, 2015). In other words, if a tool is valid, it measures or captures information on the factor in question, not a different factor. Fraenkel, Wallen, and Hyun (2015) recommend that classroom teachers improve the validity of their measures by discussing their data methods with a critical colleague. As you will hear in my reflection, I already have two critical colleagues who have helped guide me to this point. These two individuals, also classroom teachers and graduate students, will be available to serve as critical colleagues throughout the rest of this process. Having a critical colleague can help to increase the validity of data collection because it decreases author bias and calls on the researcher to incorporate more perspectives (Fraenkel et al., 2015). Because I want to make sure that my rubric is not measuring students' understanding of previously learned concepts, and is in fact measuring their ability to integrate content and technology, I have decided to use the term volcano for the pre-test and earth quake for the post-test. These terms are similar and are apart of the same fourth-grade Next Generation Science Standard:

ESS2.B: Plate Tectonics and Large-Scale System Interactions

§ The locations of mountain ranges, deep ocean trenches, ocean floor structures, earthquakes, and volcanoes occur in patterns. Most earthquakes and volcanoes occur in bands that are often along the boundaries between continents and oceans. Major mountain chains form inside continents or near their edges. Maps can help locate the different land and water features areas of Earth. (4-ESS2-2)

Reliability

Reliability is a measure of consistency; if results were to be collected by someone else or at a different time, they should be similar to the initial results that were collected (Fraenkel et al., 2015). The intervention that I designed is adapted from an instrument developed by Ramkellawan & Bell (2017). This instrument was based on the ladder of inference, which is a well-tested conceptual tool used to collect data. Both Ramkellawan and Bell's (2017) tool and the ladder of inference's reliability have been assessed and produced strong, consistent results. Although my intervention tool (reflective questions) are revised, they will have a stronger sense of reliability because of their basis on these established and reliable tools. In order to improve the reliability of my pre-action and post-action data collections, and thus my intervention, I will utilize the Kuder-Richardson Approach to test the reliability of the scores received on the video project. This method takes into account the mean and the standard deviation of scores(Fraenkel et al., 2015). Lastly, I have made sure that my tools are reliable across pre-tests and post-tests by keeping the format the same. The only alteration I have made, as mentioned above, was to the concept students used during the video process, which helps to improve the validity. Because these terms are apart of the same standard in the same grade level, they should present the students with similar challenges, increasing the reliability across pre-test and post-test measures.

Sources of Tools

I am collecting data on teachers' expectations and students' experiences through verbally recorded responses, similar to the approach suggested by previous sources, such as Bower, Cavanagh, Moloney, and Dao (2011). Their findings demonstrated that verbal responses reduced communication anxiety, elicited more complete answers from participants, and improved overall presentation quality (Bower et al., 2011). Voice thread was chosen because it follows this suggestion and allows the user to view questions, record their responses, and view a timer. Flip grid was chosen because it allows for recorded responses, the ability to add "hints," and was designed specifically for elementary-aged students. I will also use the process endorsed by the California State University, Fullerton's credential program, to create a scaled rubric to measure students' achievement, both on the pre-test video project and the post-test video project. Lastly, I am collecting data on my intervention, which was informed by the work of Ramkellawan and Bell's previous intervention (2017).

Sample Flip Grid Prompt (Pre-Test and Post-Test)

Sample Rubric (Pre-Test and Post-Test)

Page updated

Google Sites

Report abuse