Action Research Guidelines - Chapter 5: Data Collection Tools.

Chapter 5: Data Collection Tools

The selection of data collection tools and strategies derives from the nature of our research questions, rather than from theoretical orientations.
Classrooms and schools are rich with data that educators naturally collect through everyday activities like observing, listening, and reviewing. When these activities are purposefully organized and systematically approached as teacher action research, they become powerful tools for understanding educational practices. By using methods such as observation, interviews, surveys, and artifacts, educators can gather qualitative, quantitative, or mixed-methods data to gain deeper insights based on their research questions.

The chapter is organized around four types of teacher research instruments most appropriate to the classroom and school world:

observation
interview
survey
artifacts and documents

5.1 Observation

Observation involves purposefully looking at a setting to gain insights into the authentic life of schools and classrooms, allowing teachers to notice nonverbal behaviors and physical aspects that may be overlooked in the chaos of teaching, with various strategies ranging from qualitative to quantitative and open-ended to structured.

5.1.1 Qualitative Observation

Qualitative observation involves observing, listening, learning, asking, reflecting, and recording, with the approach being either semi-structured—focusing on specific issues—or unstructured, where the observer determines what is significant based on the classroom dynamics.
Tools used in qualitative observation include observation protocols, behavioral logs, and multimedia resources such as photographs, videotapes, and audiotapes

The Process of Conducting Qualitative Observation

Overall, the focus of your observation should emerge from the research question(s), but at the same time you should be cognizant of the general context of the setting.
Envision how the observation will help you in answering the research question.
You also need to make a decision about where, how often, and how long the observations will be.
Ask yourself questions such as What am I going to look for? and How will I describe the observation in words for future analysis and interpretation?
Determine what your role will be during the observation.
Will you observe from a distance with minimal involvement, fully engage as a participant/observer, or adopt a role in between, such as observing teacher-student interactions or your students in non-teaching situations?
Keep in mind that even if you do not purposely intervene, your presence in the classroom can affect the behavior of the people in the setting and change it.
Start by observing the entire classroom to understand the environment, then gradually focus on the key people and behaviors relevant to your study, occasionally broadening your view to maintain awareness of the overall context.

Observation Protocol

The observation protocol includes detailed field notes that capture what is seen, heard, and felt during the observation, as well as reflective thoughts and feelings, with rich descriptions that bring the setting to life.
Observation protocol includes two kinds of field notes: descriptive and reflective.

Descriptive Notes

Descriptive notes aim to record what happened during the observation without inferring feelings or responses to what is happening.
In the descriptive notes, describe the physical setting; provide a verbal portrait of the participants; record the acts, activities, and events that take place; and document conversations verbatim.
Draw a classroom seating chart of the physical layout of the setting including desks, tables, other furniture, computers, and other teaching tools, such as a smartboard.
Before you start, you should concentrate on the aspects that are directly connected to your own topic. Consider the following general questions:

Who is in the setting? For example, indicate the number of people present, and their gender and age group.
When does the group meet together? For example, the time of the school day and the length of the session.
Where is the meeting taking place? For example, the physical setting, the ways the group uses the space, and the ambience of the room.
What is happening here? For example, what are the people in the setting doing or saying to one another? In what events, activities, or routines are the students engaged?
How is the situation organized? For example, what rules or norms govern the situation?

As you focus your observation on the issues directly related to your own investigation, direct your attention to the following aspects:

Portray the individuals or groups that you chose to follow: for example, their physical appearance, mannerisms, and gestures.
Describe activities: for example, provide a detailed depiction of activities related to the focus of your study.
Note particular interactions: for example, who was involved and what occurred?
Record dialogues: for example, direct quotes, unique phrases, and revealing statements.

Reflective Notes

Reflective notes are used to record reflections and insights about what is happening in the setting. These notes may include:

Reflections on the meaning of what is observed: for example, your insights and emerging interpretations.
Reflections on the observation methods: for example, reflect on procedures and materials used in the process, and think of some suggestions for future observations.
Reflections on problems encountered: for example, ethical dilemmas and conflicts.
Reflections on your own frame of mind: for example, attitudes, expectations, and biases.

The reflective field notes are written in a way that clearly sets them off from the descriptive notes. Some researchers use brackets or parentheses to indicate their reflective notes, whereas others draw a line down the center of the observation protocol form to divide the descriptive notes from the reflective notes.

Figure 5.1 for a sample observation protocol form

Observation Steps

The following are some guidelines and suggestions to help you plan and carry out your classroom observations:

Find a place that provides a good view of what is going on, without being conspicuous.
Spend some time at the beginning of the observation without taking notes and then confine your attention to the general characteristics of the setting.
Identify the people, activities, and interactions that will be the focus of your attention.
Draw a diagram of the classroom.
Record your descriptive and reflective field notes on the observation protocol form.
Start with a broad sweep and gradually narrow your focus to particular participants or interactions that are most valuable for your research.
Record what happens in the class. It would be helpful to note the progression of the classroom activities and topics discussed.
Write down behavioral descriptions in the descriptive field notes section as they occur; be sure to set them off from the reflective field notes.
After completing the observation, take some time to review your notes and add any descriptions and reflections you may have missed. Respond to questions such as What is important here? and What would I want to focus on more closely if I return to this class?

Behavior Log

The behavior log is a useful observational tool for a more focused observation.
This type of log is a running record used to keep track of the behaviors exhibited by an individual student or a small group of students during a given situation (e.g., student group work, playground).
The running record enables the teacher to identify patterns of behavior.
It provides an examination of the dynamic between what preceded and followed the behaviors and allows for a comparison between an individual’s conduct at different times or in different situations.
In the behavior log, record the sequence of events and describe what preceded and what followed the studied behavior. You may record a student’s inappropriate behavior every time it occurs (see Figure 5.3).
Another option is to keep an interval behavior log where you record what is happening at regular intervals, such as every 5 minutes (see Figure 5.4). For example, let’s say you observe a child, who is diagnosed as having attention deficit disorder, while he or she is working on math exercises. Every 5 minutes record his or her behavior; note also the reaction of the people in the context (e.g., peers and the teacher) to his or her actions during that time. In a separate column add interpretive comments.

Photographs, Videotapes, and Audiotapes

Observational tools like photographs, videotapes, and audiotapes can enhance your capacity to capture student behaviors, attitudes, and social interaction through images and sounds.

Photographs

Photographs provide a useful means of representing a setting and its members. They offer powerful visual images that complement the observation protocol and reinforce the verbal descriptions. For example, if you investigate students’ interactions during lunch, you may take a set of photographs of the lunchroom at 5-minute intervals over a 30-minute time span for several days. Together the pictures depict patterns of seating arrangements, behaviors, and interactions in that setting on a typical day.

Videotapes and Audiotapes

Videotapes and audiotapes are useful when you want to observe your own teaching and you lack the time or the ability to take observational notes while you teach the class. The advantages of video- and audiotapes is that they provide a permanent record you can review to notice, with greater accuracy, students’ behavior, attitudes, and social interactions, and other aspects of the classroom dynamics that are not easily captured or remembered.

As with every research activity that involves participants, ask the permission of the people in the setting, and in the case of young children, their parents’ permission, before you tape them.

Mark all tapes with dates, time, and participants’ identities and other pertinent information. Record the observational notes from videotapes and transcription from audiotapes following the observation protocol process described above.

5.1.2 Quantitative Observation

Quantitative observation involves systematically recording specific, measurable behaviors or activities in pre-determined categories to produce numerical data for analysis and comparison.

Tally Sheets

A tally sheet is used to track the frequency of a target behavior or event at a specific point in time. Figure 5.5 is an example of a tally sheet showing how many times during class Johnny raises his hand and asks for help.

Checklists

A checklist is made up of a predetermined list of behaviors or activities that are the focus of the observer’s attention.
In order to develop a useful checklist, the observed behaviors should be defined and be separated into their components (e.g., “The students concentrate on their work”; “The student sways on his chair.”).
Checklists allow you to track the development of behaviors over time or across settings (see Figure 5.6 for a sample checklist).

Rating Scales

A rating scale is used to record the extent to which a specific behavior or situation exists.
The response choices range on a continuum from one end of the scale to the other (e.g., strongly agree to strongly disagree).
Rating scales are also called Likert scales (named after Rensis Likert, the developer of the scale) or Likert-type scales.
Figure 5.7 shows a sample of a rating scale that is focused on a specific student’s behavior.

Figure 5.8 presents a list of suggestions and considerations to keep in mind as you plan your qualitative or quantitative classroom observations.

5.2 Interview

The interview, a key data collection strategy in teacher action research, facilitates in-depth conversations and provides insights into participants’ experiences, complementing observation data and offering multiple perspectives on the investigation. Interview data can be collected in an unstructured, semistructured, or highly structured process.

Unstructured Interviews

Are informal yet purposeful conversations where broad questions are casually presented, allowing the discussion to flow naturally, with key insights recorded afterward.

A Semistructured Interview

Uses pre-prepared open-ended questions while allowing the interviewer to explore unexpected topics and seek clarification based on the participant's responses.

Structured Interviews

Involve asking all participants the same set of prepared questions in the exact wording and order, ensuring consistent and comparable data collection across different individuals and settings.

The Interview Process

The interview process involves developing effective questions, following structured steps for conducting individual, focus group, or online interviews, and drawing insights from experienced sources

Conducting the Interview

During the interview, actively listen to the responses, take detailed notes, and ask follow-up questions to deepen the discussion.

Before the Interview: Carefully select a knowledgeable interviewee, arrange a convenient and quiet setting, and test all recording equipment beforehand.

During the Interview: Build rapport, explain the purpose of the interview, ensure confidentiality, and remain attentive to the interviewee’s responses to foster trust and openness.

After the Interview: Reflect on the process, transcribe the conversation accurately, and evaluate whether the interview provided the necessary insights for your study.

5.2.1 Focus Group Interview

A focus group is a valuable interview technique where participants, such as students, parents, or teachers, interact and share diverse ideas and experiences, fostering a dynamic exchange of perspectives while the facilitator guides the discussion.

Here are some suggestions to ensure a productive focus group interview:

Form a focus group of four to seven participants.
Create a seating arrangement so that everyone can see and hear each other.
Present clear and open-ended questions to the group.
Provide an appropriate time limit for each question.
Facilitate an open and free discussion; however, keep the conversation on track.
Ensure that all participants have an opportunity to express their views.
Make sure that all opinions are heard respectfully.
Prevent one or two participants from dominating the conversation.
Avoid a situation in which the majority opinion silences a contradicting point of view.

5.2.2 Online interviews

Is conducted through synchronous chat rooms or asynchronous email exchanges, offer the advantage of time-saving and accessibility, but may lack the depth of immediate verbal and nonverbal communication, potentially limiting deeper insights into participants' feelings and thoughts.

5.3 Surveys

Surveys are a widely used, efficient method for gathering large-scale information quickly and at minimal cost, allowing for the assessment of opinions, perceptions, and behaviors, but they can face challenges such as low response rates, potential bias in responses, and limitations in capturing the depth of participant views, particularly when administered electronically.

Remember that most surveys measure respondents’ perceptions or attitudes, not what they actually do or how they behave.

The Process of Conducting Surveys

Clearly articulate to yourself the purpose of the survey and the research questions you hope to answer.
Identify the type of information you want to collect and how you would use that information to answer the research questions.
Decide what demographic information (e.g., gender and age) you need to gather in your study.
Develop a plan to administer the survey (e.g., online, hard copies, or face to-face) and to gather the surveys.
Prepare a timetable for constructing, distributing, and collecting the surveys, and for coding, analyzing, and interpreting the responses.
Determine the sample needed for the study. If you plan to survey students, teachers, or parents, make sure that you have permission to administer the survey. Have viable and realistic plans to ensure a satisfactory response rate.
Give a title to your survey (e.g., “Students’ Attitudes toward Extracurricular Activities,” as opposed to simply calling it “A Survey”).
Write the first draft of your survey items (questions, statements, and response choices).
Pilot test the survey, if possible, or consult with colleagues and classmates and ask them to review the items before you administer the survey to your sample. (This is a very important step that is often overlooked!)
Prepare a cover letter or a brief explanation of the survey.
Provide clear directions to the survey respondents at the beginning of the survey on how to complete the survey items.
Administer the surveys; collect the completed forms.
Code the data; enter the numerical data into a spreadsheet; or type the open-ended responses.
Analyze and interpret the results.

5.3.1 Structured Responses

The response choices to structured items are divided into several categories:

Rating responses

Rating responses are designed to capture opinions, attitudes, or beliefs by providing graded options, commonly using formats like Likert scales, which range from "strongly agree" to "strongly disagree," or Likert-type scales that measure quality, frequency, or self-perception, with considerations for clarity, directionality, and scoring to ensure meaningful data collection.

Yes/no responses (Dichotomous questions )

Respondents in this category are asked to select from two options, "yes" or "no,", "True" or "False", which are ideal for collecting factual information, though additional choices like "don't know" or "not applicable" may be included to accommodate nuanced responses, such as providing frequency options instead of a binary choice for statements like "Student X plays with other students during recess."

Ordered responses

Responses can be presented in a logical order, such as arranging education levels from the lowest to the highest, or grouping grade levels into categories like primary, intermediate, middle, and high school to simplify choices.

Unordered responses

This category is used when the responses cannot be presented in any particular, predetermined order.

Checklist

Checklists are tools used to record or assess items, behaviors, or skills, often employed in surveys where respondents select from a dichotomous list of choices, marking items they choose, with variations allowing for ranking or selecting a limited number of options.

Rank-ordered responses.

Survey respondents use the rank-order format to organize a list of items by assigning rankings, with 1 representing their favorite choice and higher numbers indicating less preferred options.

5.3.2 Unstructured (Open-Ended)Responses

There are two major types of responses in this category:

Short Write‑In Responses

An item with a short write-in response in which the survey takers are asked to respond to a question with limited response options, usually one or a few words, or a number.

Responses with One or More Sentences

In surveys, items requiring responses in one or more complete sentences typically ask for detailed information, comments, or feedback. While these questions can provide valuable insights, the response rate may be lower as participants often prefer simpler tasks, like circling options. To improve engagement, it's advisable to position such open-ended questions at the end of the survey.

5.4 Artifacts and Documents

5.4.1 Official and Personal Artifacts and Documents

Teacher researchers utilize official and personal artifacts and documents—such as student work, school reports, meeting minutes, and personal reflections—to construct a comprehensive understanding of educational processes, incorporating both quantitative data like test scores and qualitative insights from journals and student projects.

5.4.2 Teacher Journals

Teacher journals serve as a valuable research method for documenting critical incidents, classroom interactions, and personal reflections, helping educators gain insights, monitor subjectivity, and address ethical considerations in their practice and research.

5.5 Final Comments about Data Collection Tools

When developing your own data collection strategies, it is important to consider the following points regardless of the tools you select:

set specific times for data collection
avoid relying on a single source of data by triangulating with multiple methods
prioritize the quality of the data over the quantity of methods used
remain flexible with your research design to adapt to new insights gained during the process.

Key Points

1. Data Collection Strategy Selection Based on Research Questions
Data collection tools and strategies should be chosen based on the specific research questions rather than theoretical orientations.

2. Complexity of Classroom Life and Categorizing Data Collection Methods
The complexity of classroom settings makes it difficult to strictly divide research tools into qualitative or quantitative categories, as a single tool can generate both types of data depending on how it is applied.

3. Key Considerations in Developing Data Collection Strategies
When planning data collection, researchers should set specific times, collect data from multiple sources, and remain flexible in modifying the research design as needed.

4. The Role of Observation in Teacher Action Research
Observation involves purposefully looking at a setting to systematically examine activities, people, and physical aspects of the educational environment.

5. Continuum of Observation Strategies
Teacher researchers can use various observation strategies that lie on a continuum from qualitative to quantitative, open-ended to closed-ended, unstructured to structured, and holistic to focused.

6. Defining the Role During Classroom Observation
Researchers should define their role when observing their own classrooms and aim to adopt an outsider's perspective to gain objective insights.

7. Observation Protocol: Descriptive and Reflective Field Notes
An observation protocol includes both descriptive field notes, which detail what is observed, and reflective field notes, which capture the researcher’s thoughts and insights.

8. Descriptive Field Notes for Documenting Classroom Activities
Descriptive field notes record observable events, participants, activities, and conversations in the classroom to create a detailed account of the observation.

9. Reflective Field Notes for Gaining Insights
Reflective field notes provide a space for researchers to record their personal reflections and insights gained during the observation process.

10. Behavior Logs as a Focused Observation Tool
Behavior logs help researchers track specific behaviors of students or groups during particular situations, with interval behavior logs used to record actions at regular time intervals.

11. Enhancing Observations with Media Tools
Tools like photographs, audiotapes, and videotapes can enrich the observational process, offering a more comprehensive record of student behaviors, attitudes, and social interactions.

12. Structured Closed-Ended Observations for Quantifiable Data
Structured closed-ended observations focus on predefined categories, gathering specific, measurable behaviors that produce numerical data.

13. Quantitative Observations for Analyzing Patterns and Trends
Quantitative observations enable comparisons across different settings, individuals, or situations, highlighting frequencies, patterns, and trends.

14. Using Tally Sheets to Track Behavior Frequencies
Tally sheets are an efficient observational tool used to track the frequency of a specific behavior or event at particular moments during the study.

15. Checklists for Focusing Observation on Key Behaviors
Checklists are used by observers to focus on specific behaviors or activities, with predetermined lists guiding the data collection process.

16. Rating Scales for Measuring Behavior Intensity
Rating scales allow observers to assess and record the intensity or extent to which specific behaviors or situations occur during the observation.

17. The Role of Interviews in Understanding Participants’ Perspectives
Interviews provide an in-depth understanding of participants’ experiences, allowing them to share their ideas, opinions, and knowledge on the research topic.

18. Unstructured Interviews for Informal Conversations
An unstructured interview involves broad, open-ended questions in a conversational style, allowing the discussion to unfold naturally.

19. Semi-Structured Interviews for Co-Constructing Narratives
Semi-structured interviews are based on open-ended questions prepared in advance, with participants encouraged to share their insights and explore additional issues related to the study.

20. Structured Interviews for Consistent Data Collection
Structured interviews follow a strict format, ensuring that the same questions are asked in the same order to each participant, providing consistent data.

21. Using an Interview Guide for Organizing and Reflecting
An interview guide is divided into two sections: Section I, which includes research questions and interview details, and Section II, which records reflections and insights gained after the interview.

22. Focus Groups for Group-Based Data Collection
Focus groups involve gathering a small group of participants to discuss their thoughts, ideas, and experiences related to the study topic, fostering collaborative insights.

23. Internet-Based Interviews for Flexibility and Efficiency
Internet-based interviews, including synchronous chat and asynchronous email exchanges, offer time-saving and flexible options for conducting interviews with participants remotely.

24. The Use of Surveys in Action Research
Surveys are a widely used and efficient method for gathering large-scale responses quickly, providing valuable data on people’s opinions, attitudes, and factual information.

25. Challenges of Survey Data in Action Research
One challenge with using surveys in action research is the potential for low response rates and the possibility that participants may provide answers they think are expected, rather than their true opinions.

Questions

List the types of data collection tools (observation, interview, survey, or artifacts and documents) that you plan to use in your research project and the reasons for using them. Explain how the different research tools will allow triangulation and offer different perspectives.
Develop a 5- to 10-item survey that can be used to collect data related to your research topic.
Make a list of documents, records, or artifacts that you plan to use for your study.