Summary: Empirical research tests a hypothesis based on new information and is defined by the methodology it uses to gather information and test it. This video covers the following points: the distinction between empirical and normative questions and between a hypothesis and a thesis (or what I call research vs theory).
I’ve said several times now that the focus of this class is empirical research on political science. So what exactly is empirical research? In this video, I try to answer that question – it’s harder to identify than you’d think.
Empirical vs normative
The first key distinction you need to be aware of is the difference between empirical and normative questions. The definitions of these terms are fairly straightforward – empirical research is based on observation, normative questions are based on norms, or values. So empirical questions focus on what something is, how it happened, or why it happened (and in particular we are going to focus on the how and why questions). Normative questions focus on what should be. They are questions about right and wrong, about the moral dimension to a question. We aren’t going to ignore ethics and morals here, but you evaluate normative questions based on your personal values. You evaluate empirical questions based on observable evidence. We might still disagree about the interpretation of the evidence, but the process is different that disagreeing about values.
An example of the distinction is in legal research. Legal research has empirical and normative elements. You need to establish the facts of an individual case – what actually happened. But interpreting the law and deciding what applies in a given case is different. This is based on deductive reasoning and interpretation of precedent. Both parts are important, but we evaluate data collection and interpretation in a different way than we evaluate legal analysis.
So the first step of is it empirical is are we dealing with observable evidence.
Research vs Analysis OR Hypothesis vs Thesis
The second step in figuring out whether something is empirical research is how we interpret and analyze that observable evidence. The key distinction is whether you are testing whether you are wrong or trying to prove you are right.
One way to think about this is the distinction between a hypothesis and a thesis statement. You all should be familiar with thesis statements – they are the central points of essays that you work to support with pieces of evidence that you gather through library research. A hypothesis is a statement that you can test and prove false. It is going to be more specific than a thesis in describing the cause and effect you are studying and what you think the relationship between that cause and effect are. You will evaluate it with research, but that research will look at original data, not secondary sources, and you will be looking at evidence that could prove your hypothesis true or false.
Before I provide examples of what I mean by that, let me be clear on what I don’t mean. When I say empirical research, I’m using research as a noun, not a verb. Looking up some sources to cite on an essay is “doing research” because you are looking for information, but a published piece of empirical research is the result of a specific kind of “doing research.”
To show these distinctions, I’d like to compare a few pieces of writing. The Tragedy of Great Power Politics is an influential book written by John Mearsheimer in which he describes his theory of offensive realism. He uses examples related to China, the United States, and the former Soviet Union to support his central argument that relative military power drives relations between states. This is a good, well-written book, but the evidence he cites is all used to build and support his thesis – he doesn’t look at the whole universe of all relations between all countries to test whether he is generally right. He looks at the countries that support his theory – countries with strong militaries. That doesn’t mean you don’t have to read about realism in Intro to IR – understanding this theory and other theories is an important first step to thinking about international relations. But it’s not empirical research.
By contrast, this article by several authors, including VCU’s Michael Paarlberg, tests two different theories of why U.S. cities might declare themselves sanctuary cities. It looks at data from both sanctuary and non-sanctuary cities and compares and contrasts different explanations (like political party registration of citizens or percent foreign born population) to determine what causes the outcome (sanctuary status). That’s what I mean by empirical research testing a hypothesis. They specifically state their hypotheses and do show support for them, but there was a chance going in that they were wrong – they didn’t know the outcome in advance.
That is the core, but there are several other indicators that can help you figure out quickly what you are looking at:
1) Did it gather original data in a systematic way? Empirical research has a data gathering process (DGP) and a methodology.
For example, academic journals aren’t the only place you can find empirical research. Think tanks and government organizations also produce it, and it tends to be this type of research – presenting new information about something we didn’t know before. For example, this report published by the Brookings Institute is a case study of a specific education program in Bangladesh. It reviews in detail how well it worked when a small program scaled up to be used in other places. What makes it empirical research is the description of their methodology – how they gathered information in a systematic way about this topic. They didn’t just talk to people they agreed with – they came up with a plan on how to get a good sample of information from a variety of sources. This is also a good example of how not all research needs to be based in numerical data. Interview data and participatory experience are equally valid ways to learn about a topic, and ones we will learn about.
Another example of the distinction between a methodology and just gathering information is online polling. There are many standards that polling companies use to try and get an accurate representation of public opinion, including random sampling (meaning you ask a group of people that could actually represent the population you are trying to study). Polling counts as research when the authors of a study are clear about how they selected who they would survey and what they did to make sure that group represented the group they were interested in. This research might be good quality (a true random sample) or it might be biased (underrepresenting Republican voters, for example), but since it is clear about the standards you use to evaluate it, it’s research and you can then analyze it to decide whether it is good or if it is biased. Polling is just entertainment when there is no effort to track or control who answers questions. Most internet polls fall into this category, which is why you get outcomes like Boaty McBoatface.
2) A second question you can ask about an article to see if it is empirical is whether it was published in a peer reviewed journal. This is not the best way to identify empirical research, since peer reviewed journals publish things aside from research articles and empirical research can be found in other places, but it can be a straightforward first cut. Research articles tend to look a certain way (for example, they have a data or methodology section) and it can be a fast way to evaluate what you are looking at. I cover what peer review means and the types of documents you’ll read this semester in the next video.
This all matters because how you assess a theory or opinion piece is different from how you evaluate empirical research. Theories have lots of purposes, including clarifying or shaping debate on an issue or presenting new perspectives on an issue. The academic standard for assessing theory is both its logical consistency and whether you test a theory on different evidence than you built it on. What separates opinion (in this case, a theory) from empirical research is not whether they present facts to support it (they both do), but how those facts are presented. It’s relatively easy to find evidence that supports a specific argument. Empirical research is valuable because it can be hard for arguments to hold up under different conditions. And that is what we will focus on this semester.