Conducting video-based research

This is the research process for video-based research that I have my students go through.  It's pretty flexible to a wide variety of research questions, and puts equal weight on your professional development and your scientific results.  This post follows Writing as a Generative Process, and takes as given that you already write copiously and generatively.

Broadly speaking, this approach grew out of Dr. Rachel Scherr's brilliant work in SPU's iRISE program which promoted video-based education research and professional development around the Energy Project program.  I formalized this approach in conversation with Rachel, informed by foundational papers in research design for video-supported research. 

My students' first projects involve plundering the video library, not generating more data.  There are two reasons for this:

• Generating new data is annoyingly time-intensive and fraught with technical problems.  You have to wait for research subjects to appear or classes to happen, the camera might be turned off or the ambient noise too much, etc.  For someone who's only working on one project at a time -- or someone working on their first project -- I don't want you to waste time collecting slow and potentially useless data.  Later, when you have analysis to occupy yourself during data collection, you can generate new data.  Because you'll already know something about analysis and research design, you'll also be able to collect higher quality data that's better tuned to your research questions.

• I already have a lot of video that needs more analysis.  Quantity video is cheap to acquire and store; the difficult and important part is knowing what's available and what's in it. PER should spend more time with archival video instead of rushing out to collect more data every time someone thinks of a new project, so I try to instill a sense of bricolage into my students.

Because you won't be collecting data, this research process is aimed at helping you with analysis.  It won't tell you what papers to read or research questions to ask; rather, it treats research questions as emergent from initial looks at the data and your research interests.

Notation: PI goals are my goals as the supervisor.  They help me figure out what's going on.  PD goals are my goals for your professional development.  They help you become a better researcher.

A statement of research interests is usually a 1-2 page document about what kinds of populations, methods, and questions you are interested in.  Graduate and job applications usually require them, so it's a good idea to get started early on writing one.  If you've never written one before, it can be difficult to get started.  There are lots of examples on the internet (google "statement of research interests physics education"), and you can use the writing prompts from Writing as a Generative Process to get you started with words on a page.  If you already have a statement of research interests from another kind of research, it's a good idea to start fresh with this one.  

Sometimes I skip this step for people who won't be applying to graduate school or jobs soon, or for people who already have a statement of research interests.

• PI goal: I assess your writing. 

• PD goal: they consider their research interests. 

• Product: Statement of research interests, such as might be used for a personal statement or job application.

• Next: I figure out what kind of research data would be interesting or useful for them.

Researchers should be able look at any repository of data and find something of interest.  It's easier to find interesting data if you know something about your interests and the repository is well-cataloged.  This step of the process helps you figure out what's in the repository, and it helps future researchers comb through the repository more effectively.

The collections in my library are generally organized by which course, interview project, or experience generated them. For your first project, you'll probably look within one collection rather than between several.  This is a practical constraint: I don't want you to split your attention between multiple collections when you're just starting to learn how to do this kind of research.  

Watch video in the repository and write a catalog of events and participants.  Start to think about what might be interesting to research using this collection.  

• PI goal: Video library curation; more extensive catalog. 

• PD goal: practice in finding interest in data; practice cataloging data; familiarity with general shape of data. 

• Product: Observational data is cheap to acquire and store, but expensive to catalog.  So making a catalog (or enriching an existing catalog) makes it easier for future researchers working with the corpus.

Engaging with the literature related to your topic is a great way to refine your ideas. When you're just getting started in a field, it can be hard to figure out the most fruitful keywords to search for or the most impactful or relevant papers.  On top of those difficulties, methodological or theoretical papers -- which are fantastic at the early stages of research -- often use a slightly different topic or population than you, so they are even harder to find.  

For my students' initial papers, I usually recommend selections from my Mendeley library or my Essential reading list.  Sometimes I also recommend papers based on whatever I've been paying attention to lately.  Very rarely, my students find early papers on their own.  I try to keep the number of papers smallish here: a handful of papers rather than a complete reference list.

If your writing is bad or there are too many different projects going on for me to remember the details, you will also write one-page summaries of each article.  Each summary answers three questions: what happened in the paper? what was interesting? what was confusing?  The summaries are useful later when you're writing a literature synthesis, and they're useful right now as you're trying to figure out what's important and how it relates to your project.

• PI goal: Acquaint students with general issues around the use of video for research; nudge them into thinking about topics that are likely to be fruitful.

• PD goal: generally learn about methods and theories.

• Product: Perhaps some article summaries.

A prospectus is a 1-2 page document which lays out a plan for conducting research.  It's a living document: the initial prospectus focuses more heavily on what's interesting and why, and later prospectuses include more details about the research design. Some graduate programs require that their students write a longer document laying out their research plans, rationale, and literature review.  This prospectus is not that; this one is short and focused.

In your prospectus, you will talk about what your research problem is, why it is problematic, what theory you're using to understand it, and what you're going to do to investigate it.  To help you write your prospectus, you should do the exercises in Research Design.

• PI goal: I assess their writing; first formal declaration of research interests as tied to data; generate suggestions of papers to read.  Research becomes more scientifically rigorous.

• PD goal: Decide and commit to some ideas for what to research; initial induction into writing as a generative process.

• Product: The prospectus document 

• Next: We have a conversation in which we refine their interests into something that is likely to be productive given the data available, their expertise, and the time we have for analysis. 

After the prospectus-refining conversation, I will suggest papers to read that will help you further refine their ideas.  Perhaps you will also look for some papers, depending on how extensive my library is on your topic and your library skills.   Depending on your writing skill and confidence, your will either make article summaries (as above) or write a synthesis of several papers.  

• PI goal: Make you less dependent on me for ideas to shape your research. 

• PD goal: Understand the research literature around your topic.  Practice writing. 

• Product: Some paper summaries or a synthesis, which might be useful for an eventual paper or dissertation chapter(s).

Possibly with help, select 1-3 interesting episodes to help with your emerging research questions.  For each episode, write a narrative as if you were telling the story of what happened to an interested third party.  Narrative lengths vary, but generally I aim for 3 pages.  Students sometimes get nervous when writing their first narratives, and tend to go in one of two directions: either too "transcript-like" with not enough interpretation, or too brief with not enough detail of who does what and why.  If you're nervous, stick to a story-telling style.  You'll spend a lot of time updating and amending your narrative with feedback from me and other researchers, and we will treat it as a living document.

• PI goal:  You find interesting data; initial data reduction; possible corrective steps if your interests are too divergent.

• PD goal: Sharpen focus on what's interesting about these episodes.  Improved skill at finding interesting episodes. 

• Product: Narrative will probably be included in a paper, possibly in condensed form. 

At this point, you should know enough about your research -- what you're looking for, the literature around it, and how you're going to generate new knowledge -- that you can write a stronger research design statement.  

This grows from your initial prospectus to include more details of how you're going to conduct the work, how the work relates to others' work on similar topics, and why this work is interesting or valuable.  If the initial prospectus included references and design elements, this step clarifies them.  If it was more vague, this step makes them appear.  You're already familiar with the research design post; now is the time to more carefully consider the questions in it.

• PI goal: internally allocate time to helping project along and decide on project timeline

• PD goal: Research design document and ensuing conversation with me are major PD opportunities to figure out the shape of this project and think rigorously about design. Also, this is your first opportunity to engage with the "living" nature of the prospectus.

• Product: plan of attack for how to conduct this project.

Here the research process forks.  Case-based research examines a small number of episodes in deeper detail.  Pattern-based research uses the initial episodes to build a codebook and dictionary, with the goal of finding the prevalence of these patterns across many episodes.  Both paradigms become a lot more specific to the particular research questions, available data, and expertise of the researchers. 

Case-based research:

• Figure out what these episodes are cases of, probably in consultation with diverse literature.

• Search for disconfirming episodes and perhaps one or two pithy episodes for the paper.

• Transcribe all episodes for IRR.

• Build overt interpretations into the narrative document and enrich it with transcript snippets.  This becomes the central part of the paper.

• IRR, focusing on strength and believability of interpretation; looking for ideas we may have missed or overfit.

• Repeat this process until it feels saturated.

Pattern-based research:

• Develop an initial codebook and dictionary based on original episodes.

• Make predictions about where to find more data

• Find more data (~5-10 more episodes) and code it, updating the codebook and dictionary iteratively.

• When the codebook and dictionary seem stable, find more data (~10-20 more episodes, depending on design) and code it. 

• IRR on about 30% of data, focusing on replicability of coding and making the codebook explicit and complete.

• Repeat IRR until coders can be >90% in agreement on any subset of data before discussion.

• Condense and reduce data, possibly involving graphs and/or statistics, refining the central message of the paper.  Develop compelling representations for your data.

• Select some data representations for inclusion into paper.

• Write up central argument of paper using those representations.

Here the process merges again.

• Write wrapper for paper: intro, theory, methods, discussion, conclusions.   Lots of this writing comes from other sources: the prospectus&design, article summaries, etc.

• Do the flow handout in Writing better papers.  This clarifies the argument and cleans up the prose. 

• Find a journal home and update the paper to fit the journal's mission and audience.

• Send paper to beta readers and revise based on their ideas.

• Submit!