The Art of Being Choosy: Finding & Evaluating Sources

Finding a source for your paper can prove harder and more heartbreaking than finding a date for the dance this weekend. In both cases, you will probably wait until the last minute, you'll have no idea where to begin looking, you'll turn to weird websites at two in the morning, you'll string together a few incoherent phrases, you'll pick the first thing that looks halfway normal, and if anyone questions you on your decision, you'll defend it with your life. "How dare you question my taste in articles!"

If you're like me, you'll start with Google, then you'll timidly wander over to your school-library's website, you'll randomly select one of the hundreds of databases (probably Academic Search Premier, because it's usually listed first), you'll type in a few keywords, you'll peruse dense articles that appear to be written in German, and then you'll go back to Google, typing in phrases like, "Bush Iraq Baby Formula Scud Missile," or, "How do I write a research paper?"

It's okay. You've got to start somewhere. But you need to know that while there is a lot of great stuff on the Internet, there's also a lot of junk. It might seem like your job is to root through all of it to find that one perfect source, but it's not. Research writing is not intended to be a needle-in-the-haystack exercise. It's unlikely your professor knows of some obscure newspaper article out there that you'll have to drive to North Dakota to find on a micro-fiche machine in the basement of the Fargo Library. Give up the myth of the perfect source and embrace, instead, the comforting reality of the adequate source.

Don't be mistaken, though, by the word adequate. It certainly has the negative connotation of less than ideal, as in, "Meh, it's not quite what I wanted, but it'll do." Adequate means that while it may not be perfect, it is certainly sufficient, appropriate, and suitable to your purposes. So that weird article you found on Ask.com that really has nothing to do with your topic is not adequate. Drop it and move on.

To be fair to that weird article, though, there actually is a time and a place when just about anything you find on the Internet could be considered an adequate source. In this chapter, we'll discuss some of the different types of sources out there, and we'll look at how you can judge for yourself if they are adequate. That way, at two in the morning when you're left to your own devices, you won't be duped into selecting a source that has no business being in your paper.

Primary, Secondary, and Tertiary Sources

All sources can be categorized in one of three ways: as primary, secondary, and tertiary sources.

Any information that comes from the actual thing you are studying is considered a primary source. So if you are researching the life of Lincoln, anything he wrote would be a primary source. If you are researching String Theory, conducting an experiment is a primary source. If you are analyzing The Scarlet Letter, then The Scarlet Letter would be a primary source. Your high-school journal entry on the book would also be a primary source.

Any field research you conduct is a primary source. Interviewing someone, surveying a group of people, observing a situation, analyzing an artifact, and conducting an experiment are all examples of primary sources.

Primary sources are a great way to put your original stamp on your research. In the academic world, we call it "creating knowledge." Whenever you collect data or analyze something, you are adding knowledge to the field. Professors love that kind of thing.

Any information that comes from what other people have said about the thing you are studying is considered a secondary source. So if you are researching the life of Lincoln, an article someone else wrote about his life is a secondary source. If you are researching String Theory, discussing an experiment conducted by scientists at Caltech is a secondary source. If you are analyzing The Scarlet Letter, then anything anyone else has written about the book is a secondary source.

Get it?

Secondary sources are a great way to show what we already know about a topic—what's already been said and done. Entire papers can be composed of secondary sources. Bibliographic essays, for example, are research papers that are intended to simply show what's already been written about a topic. Similarly, literature reviews examine the range of sources available on a topic.

Occasionally, a professor will ask you to write a research paper as a means of demonstrating that you understand something, so secondary sources will be all that you need. Usually, though, secondary sources are used in conjunction with primary sources. A common organization strategy for research papers is to include a short literature review at the beginning (secondary sources)—to show what's already been said on the topic and what still needs to be said—and then to include your own field research (primary sources).

Tertiary sources are just reference material, like dictionaries, encyclopedias, maps, data charts, and instruction manuals. There's nothing wrong with them, but usually when a professor asks you to use a minimum number of sources, they mean primary and secondary. You still need to list your tertiary sources on your works-cited or references page, but your instructor likely won't count them towards the minimum-source count.

Types of Primary Sources

Interviews

Here's a secret: the best way to blow your professor's mind is to seek out someone who knows your topic well and interview them. If you're writing a paper for criminal justice, find a cop or a lawyer or a judge and ask them some questions. If you're writing a paper on whale migration, find a marine biologist or a shrimp-boat captain and interview them. Take advantage of the cool people around you and pick their brains. It makes for interesting, original, and insightful material for your paper that can't be found anywhere else.

There are a million ways to conduct an interview. It could be a casual conversation on a subway ride. It could be a formal conversation in someone's office. It all depends on who you are interviewing, how much time they have, and the types of questions you're asking. There are a few pieces of advice I have for ensuring that interviews are quick and easy and useful in your paper.

Before the Interview:

Schedule the interview. Don't just show up unannounced Michael Moore style and hope the person will answer your questions.

Let them know why you are interviewing them. Tell them as much as you can about the assignment and how you intend to use what they say in the paper.

Let them know how long the interview will last. Don't promise them it'll only take a minute if you have thirty questions you want to ask.

Let them know who will be reading the paper (classmates, your professor, readers of your blog, your agent, etc.).

Let them know it is completely optional and that they are under no obligation to participate. Don't ever coerce someone into being interviewed.

Learn everything you can about the person before the interview. Visit their Twitter and Facebook and LinkedIn. Read their blogs, articles, and books. And then don't ask them questions that you can find on your own. It's a waste of their time.

Write down your interview questions (believe me, you will forget some of them if you don't, and if you plan to just "wing it," your brain will certainly let you down in that critical moment and you'll ask them something completely lame).

Check your recording equipment. If you plan on using an audio or video recorder (and you should), make sure the batteries aren't dead, that you know how to operate it, and that you have enough storage space.

During the Interview:

Make sure the person is okay with you recording them. In most states, it is illegal to record someone without their permission. Also, it's rude. If they refuse, then you'll have to do it old-school with a pen and paper. But if they allow it, I highly recommend recording it.

Start with small talk. You're not an FBI interrogator. You don't need to launch in right away with your first question.

Don't interrupt an interesting story just to get to your next question. You can steer the conversation, but let them talk.

After the interview, ask the person if it's okay to contact them with follow-up questions.

Thank the person for their time. Follow-up emails or thank-you notes go a long way, especially if you ever want to ask them for another favor.

After the Interview:

Write up your notes of the interview as soon as possible. Even if you recorded it, write down your thoughts and impressions.

Transcribe (type up) the interview. Keep in mind that one hour of recorded interview will take you three or four hours to type!

Mark up the transcript. Highlight the quotes and comments you find interesting. Make notes to yourself for how you might use it in the paper.

If it’s feasible, offer to let the person read the paper or the transcript before submitting it to your professor (or your publisher), especially if they discussed any sensitive information. The person should always be allowed to rescind anything they say.

Surveys

Surveying is a great option when you want to get a lot of opinions on a matter. The nice thing about surveying is that a small group of people can actually be quite representative of a much larger group. A survey of thirty first-year students on your campus can give you a good indication of the prevailing opinions and attitudes of the first-year student population. Likewise, if you live in a town of fifty-thousand, you don’t need to survey every resident to get an accurate read on them: a survey of, say, fifty people might get the job done.

When you survey people, there a few tricks to help guarantee you are getting accurate information that will be useful in your paper.

Don't Manipulate The Respondents

You might be tempted to word your survey questions in a way that leads to you getting exactly the answers you want—but why? That doesn't "create knowledge." That's not what professors are looking for. Instead, word your questions in a way that elicits honest, useful information.

Take for example this question: "How nasty is the food at the university's dining hall: nasty or really nasty?" This doesn't leave the respondent with much choice but to indicate that the food is nasty, even if they think it tastes good. And then you, as a researcher, are left with unreliable information. Of course your paper is going to overwhelmingly show that the students think the dining hall's food is terrible.

Consider this revision of the question: "On a scale of 1 to 5, 1 being completely disgusting and 5 being completely delicious, how would you rate the food at the university's dining hall?"

Now, you really don't know what you'll get until the survey results come in. And hopefully, the results will surprise you. Researchers rarely conduct field research to prove what people already know. They do it to find answers to questions.

Make Sure Your Survey Group is Representative of the General Group

If you are trying to find out how students on campus feel about changes in graduation requirements, don't just survey your first-year writing class. Try to include students of all grade levels and in a variety of majors. Likewise, if you are trying to determine if residents of your town are in favor of a new community center, don't simply survey the people in your neighborhood; survey people from across the town. While a relatively small amount of surveys can be conducted—say, fifty to represent five-thousand people—the target group should have a similar make-up and ratio as the general group. So if twenty percent of the student body are seniors, then make sure twenty percent of the surveys come from seniors. If fifty percent of the town's population live downtown, then make sure fifty percent of your surveys come from downtown residents.

Inform Respondents of Their Rights and Your Intentions

If you are planning on publishing your research in any way—posting it online, presenting your findings at a research conference, revising it as a letter to the editor—you will need to get approval from your school's IRB (Institutional Research Board). The IRB is set up to make sure all research involving human subjects is conducted legally, ethically, and safely. It may seem odd to guarantee the safety of survey respondents, but that's just the way it is. IRB approval is needed for any field research: surveys, interviews, observation, and experiments. Basically, if you are using a human being as a source of data, you need approval.

If you're not sure if you should seek out IRB approval, ask your professor.

Even if you don't need to seek IRB approval, you should still let your survey respondents know why you are conducting the research, how it will be used in your paper, who will be reading it, and that they are under no obligation to take the survey. Also, make sure they know they can skip questions or withdraw from the survey at anytime without repercussions.

For more information, visit your school's IRB page (I guarantee you that your school has one). They have all sorts of useful templates and advice on conducting effective and ethical surveys.

Select a Distribution Medium

You can go as high-tech or as old-school as you want with surveys. You can stand on the corner with a clipboard and ask people to answer a few questions. You can print out your survey and pass it out to your class or your dorm or your work. You can create a Word document and email it to people. You can create a survey widget on Facebook or your website or blog. You can use an online survey organization, such as Survey Monkey or Google Forms.

I recommend using Google Forms. It's free and easy to use. Once you create your survey, you can share the link for the survey through email or your blog/website. Google Forms will also help you make sense of the survey data you receive by putting the information in pretty charts and graphs. Try doing that with a clipboard and a piece of paper!

Observation

You may not realize it, but you are already a field researcher. Most everything you have written has come through personal observation. From the moment you were born, you've been gazing upon this big, weird world, trying to make sense of it all. And when you have written about it—in journals, in text messages, in emails, in "My Summer Vacation" essays—you've relied on your sense of observation.

Using observation intentionally as a form of research is no different. The only difference is that you observe with a goal in mind. If you want to learn about how children play, sit on the couch and watch your siblings. If you want to know more about people's shopping habits, hang out at the mall and pay attention to what people buy. You're an insightful, observant person—take advantage of it. Chances are, the answers to a lot of your questions are right in front of you if you open your eyes and pay attention. You don't have to rely on Google for everything.

Experimentation

Experimentation is a lot like observation, but with the purposeful manipulation of variables. If you decide to sit in a doctor's office waiting room for two hours on a Monday afternoon observing how many people play on their phone and how many pick up a magazine, then that is observation. If you come back the following Monday and disconnect the WiFi to see if it makes a difference, that's an experiment.

If you are conducting an experiment of any kind that uses people or animals, always get IRB approval. Even if it seems harmless. If the experiment is on inanimate objects and poses no threat to people or animals, I still recommend you ask a professor who is skilled in that field for assistance with designing and implementing the experiment (chemists, physicists, biologists, education specialists, sociologists, and so on).

Whether or not you need to seek IRB approval, always approach experiments with the mindset of being safe, respectful, and in doing no harm. No good comes from experiments that hurt people, animals, communities, or the environment.

Artifacts

An artifact is anything that is made by humans. You are surrounded by endless artifacts: clothes, office equipment, phones, computers, purses, video games, website, magazines, books, movies, tattoos, art, candy bars, junk mail, guitars, cars--you name it. And all of these things are worthy of analysis. Anytime you take something manmade and analyze it, that artifact becomes a primary source. You can analyze artifacts from many angles: historically, culturally, rhetorically, and so on. Maybe you're analyzing how it was made, or how it could be made more efficiently, or what it says about the user, or what it's deeper meaning is, or why it's so important to society, or why it's misunderstood. There really is no end to how you can analyze artifacts. In college, I was a literature major, which meant that I spent a great deal of time analyzing poems and short stories and novels. Each was an artifact.

The research paper you are about to write is an artifact. Even the articles you use in that paper are artifacts. That doesn't mean, however, that all of those articles are primary sources. They are only a primary source if you analyze them as an artifact. For example, if I'm writing about gun control and I have a great quote by Bob Smith from his article about AK47s, then that would be a secondary source. But if I use my paper to analyze Bob's argument, then I am treating his article as an artifact, and it would be considered a primary source.

Types of Secondary Sources

I'm sure there are many ways to categorize secondary sources, but for the sake of ease, I like to think of them in terms of popular sources and scholarly sources.

It might seem like a simple distinction, but at times it can actually be difficult to identify which is which. An article, for example, from The Chronicle of Higher Education might seem like a scholarly source (it has the word higher education in it, after all, and it sounds terribly boring), but it's actually a popular source.

To understand the difference between the two, you have to understand the process of how things get published. If anyone can publish in that venue, then it's a popular source. If it's up to a single editor or publisher who gets to publish in that venue, then it's a popular source. Popular sources include newspapers, magazines, blogs, websites, and books.

Scholarly sources, on the other hand, have to go through a vetting process before they are published. The most common venues for scholarly sources are academic journals and trade journals. When you write an article and try to publish it in one of these, it will be sent off to several "readers" who are experts in the field. Each reader will give it a thumbs up or a thumbs down, and if all of them agree that the article is well written with good sources and actually adds knowledge to the field, then it will be published. Compare that to popular sources, which are all about publishing articles that make money or increase readership. If you write an article and send it to a popular source, like Rolling Stone, then the editor isn't worried about "knowledge creation"--instead, he or she will be worried about how interesting the article is and if it will increase magazine sales.

Popular vs. Scholarly Sources

It would be a gross injustice to advise you to only use scholarly sources and avoid popular sources in your research writing. Both have a time and a place.

Scholarly sources are things such as academic-journal articles, conference papers, and monographs that are written by experts for fellow experts. They are peer reviewed, which means several other experts in that particular field read and validated the source before publishing it, so by the time you read it, you know it is reliable. This can go a long way in helping you write a solid, credible paper. It's hard to blow off something written by an expert in the field, even when you disagree with it. They are also great when you want to use a lot of solid facts and data.

There are a couple downsides, though, to scholarly sources. First, the process of writing and publishing an article in an academic journal is time consuming, so by the time the article is available for you to read in a database, its findings might already be several years old. This can be a problem if you're writing about a time-sensitive topic, like a current event. And second, scholarly sources are usually written by experts for fellow experts, so the language can be dense, boring, and esoteric. You'll have to spend some real time wrestling with the article so that you fully understand it, and then you'll need to summarize and paraphrase it using simpler language for your readers.

Popular sources--such as blogs, websites, magazines, newspapers, and books--are great when you want something fresh and interesting. Popular sources don't go through the time-consuming peer-review process. Instead, an editor or team of editors will choose what to publish--and that choice is usually based on whether or not they think people will read it. It's all about the Benjamins. While some publishers might have a high ethical standard and aim to only publish reliable and accurate information, that is all in the service of getting people to buy, click, or subscribe. The plus side of this is that popular articles are published quickly (time is money, and old news doesn't sell), and they are written more with readers in mind, so you'll get a lot more flavorful quotes and opinions.

The downside to popular sources is that credibility, reliability, and accuracy aren't always guaranteed. They can be biased, misleading, incomplete, inaccurate, and unreliable. That's why you, as a researcher, need to know how to judge if a source is worth using. Just because an article is entertaining or confirms everything you already believe does not mean it's a good fit for your paper. But there are many wonderful popular sources out there that will really help you make your argument or answer your driving question. You just need to know how to identify them.

Judging the Adequacy of a Source

I was given some terrible advice as a college student. The Internet was still in its infancy, and one of my professors told me that .com sites were not to be trusted, .org sites were relatively reliable, and .edu sites were always reliable.

I was writing a paper on American Naturalism (think Jack London and Call of the Wild), and I found a great .edu website that had a whole bunch of articles that covered the same topics I was writing about. I cited four of five them, and I used them as the key evidence to support my claims I was making.

I turned in my paper, and a couple weeks later I got it back with a D-. I stopped by the professor's office after class and asked her about it. She pulled up the website I had used, and I quickly realized that the .edu site was a high-school classroom page and the so-called articles I had used were actually term papers written by students. Fortunately, she let me rewrite my paper, and I learned a valuable lesson about determining a source's reliability.

Just to clarify--I wasn't a complete rube in college. In those early days of the internet, websites all relied on simple HTML coding. You couldn't upload a PDF or anything, and there weren't many styles, colors, fonts, or templates to choose from. Websites were ugly, incomplete, hard to navigate, and confusing. So a teacher's classroom website wouldn't look much different than a major magazine's.

It's up to you to be a detective when it comes to your research. Probe around the website or magazine or book or wherever it is that houses your source and find out if it's reliable. Who wrote it? Are they an expert in the field? Are they a bored high-school student writing for their blog? Who paid for the article to be written? Who has access to it and can therefore edit it?

Wikipedia is probably the most infamous unreliable source out there. Many teachers ban it since anyone can log onto a page and change anything they want. Don't believe me? Try it. Pick a random page and change anything you want. The site won't even ask your name. Some pages, like global warming, are changed up to a thousand times a day. Depending on whether you access it at 8:05 am or 8:06 am, you're going to get a radically different viewpoint about whether or not global warming really exists. I like Wikipedia. I use it all the time. It's a great place to get started and learn about your topic. You can also follow the links on the page to see where the author got his or her information, and that will often lead you to reliable sources that you can actually use and cite in your paper. So don't automatically dismiss Wikipedia as worthless; just accept that it's not always reliable and you need to approach it with a detective mindset.

The CRAAP Test

A simple way to evaluate whether or not a source is worth using in your paper is to perform the CRAAP test. Familiarize yourself with the source and then answer the following questions:

C – Is the source current?

• • • If you're writing about a current event, pop culture, or technology, then the newer the better.

    • Using "old" sources is often a red flag for plagiarism.

R – Is the source relevant?

• • • If the source doesn't do much to support your argument or answer your driving question, then ditch it.

A – Is the source written by an authority on the subject?

• • • Or does the author at least least cite people who are authorities?

A – Is the source accurate?

• • • Don't fall for fake news. Websites like Snopes or Politifact employ highly educated and dedicated fact checkers. Visit them to verify if the information in your source is accurate. If it's not, ditch it.

P – What is the purpose of the source?

• • • If the source was written for political, marketing, or financial reasons, you might want to ditch it. For example, an article written by Ford about the virtues of the F150 (to get people to buy more of their trucks) probably won't be very reliable. Likewise, an article by Rush Limbaugh (a conservative pundit who writes to fellow conservatives) about the failings of Hillary Clinton probably won't be reliable either.

Your source doesn't necessarily have to pass every single question of the CRAAP test, but if it fails one or more, you should proceed with caution and divulge any discrepancies to your readers. For example, if you wanted to use the article I mentioned by Rush Limbaugh, then you might make it clear in your research paper that the article is politically motivated and biased, but--for whatever reason--you are still choosing to use it.