Evaluating Information and Media Bias

Section 1: Evaluating Information Sources with the CRAAP Test

How important is your ability to identify information that is untrustworthy? Do you want to be an adult that makes good life decisions based on thinking critically about the available information? Yes is the answer to both of these surely, so here is how you build the habit of thinking critically and evaluating information effectively. 

There are a variety of good quality tools like the CRAAP Test, if your teacher has instructed you to use an alternative because it better suits that subject, go with it. Otherwise, let's learn about CRAAP Testing. 

CRAAP stands for Currency, Relevance, Authority, Accuracy and Purpose. Use the CRAAP Test to evaluate your sources of information such as websites, books, newspaper articles etc. The CRAAP Test provides a list of questions that prompts you to think about the source of information.  Not all information is good quality information, especially online.

Explanation of CRAAP Testing for Websites

Use this guide to help you when you use the CRAAP test table to rate a resource:

Currency (ask yourself):

When was the information written and last updated?

• How important is it that your information is up-to-date? Is it a science, technology, current affairs or health-related topic?

• When was the Web page originally published, last updated or revised? Are all of the links on the page functional? 

• Remember, if you are searching for primary sources of historical information, or if you want to compare something like current attitudes vs 1950's attitudes, then it would be relevant to refer to older information sources. 

Relevance (ask yourself):

Is this the information you need for your topic? 

• Does this information help you answer your question? 

• Is it written at the right level (not too simple or too complicated)? 

• Is there a better source you could use with similar information? 

• Is it the right type of information (e.g. have you been asked to find data, primary sources, expert opinions…)?

Authority (ask yourself):

What qualifies this author to provide information on this topic? How can you tell whether they know what they are talking about? 

• Who is responsible for the information on the page (the author, creator, sponsor?) This might be a ‘corporate author’ (a group or organisation rather than an individual).

• Are there any credentials given – academic qualifications, job title, employment history? Use a search engine to do some quick background research on the author (or corporate author) – what qualifies them to write about this topic? Being an expert in an entirely different field doesn’t count! Do they have a good reputation?

• Learn about the organisation on its home page. Look for an “About Us” link – but don’t rely on this. Check what others are saying about it too.

Accuracy (ask yourself):

Is the information backed up by evidence? Are there any sources cited?

• Where did the information on the page come from? Did the sponsor/author of the page create the information?

• Are there cited sources or links to the original source of the information? Do these sources look trustworthy? Watch out for articles with long lists of sources that are just articles from the same publication or websites with no authority. Can you find the original sources? 

• Can you check any of the information in other independent sources or from your own knowledge? 

• Are there obvious spelling, grammar or layout errors? 

• This site from the University of Georgia (http://guides.libs.uga.edu/factchecknews) has an excellent list of fact checking organisations including Fullfact.org and Media Bias Fact Check.

Purpose (ask yourself):

What was this source written for? Is there any obvious bias?

• Why has the page been created? Is it to inform, teach, sell, entertain or persuade? Is there obvious bias or does the author seem fair and objective? 

• Is the information based on facts or is it opinion? Is the author using good evidence for their opinion? 

• If the site is linked to an organisation, does that group have a particular agenda? 

• Use a site like https://www.computerhope.com/jargon/num/domains.htm to check the meaning of the domain suffix (e.g. .edu is a US education site) 

• If it is a news source try a site like https://mediabiasfactcheck.com/ to assess accuracy and bias.

Source: (CRAAP Testing: Websites by Oakham School is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License)

Evaluating Information from Generative Artificial Intelligence Websites

You may use generative artificial intelligence (generative ai) websites like ChatGPT as a source of information. When used in an ethical and informed way, this can be an excellent strategy. It is also highly risky in relation to how you use that information; the risk of academic malpractice (cheating) is high without careful use and following school procedures of citing and referencing generative ai sourced information. 

In the context of learning about evaluating information sources, it is vitally important that you understand generative AI does not by default refer to the most authoritative and reliable source of information. AI data is based on numerous sources including sites like Wikipedia and Reddit that `ing ad revenue, the information may not be accurate, verified, unbiased or even slightly reliable. ChatGPT can also generate entirely new text, that is convincing, irrelevant, nonsensical, and wrong.

Bias is defined by Smith and Rastagi (2020) as “A tendency, inclination, or prejudice toward or against something or someone. Biases are often based on stereotypes, rather than actual knowledge of an individual or circumstance” (p. 54). Smith and Rastagi (2020) describe Biased AI (or Algorithmic Bias) as an AI system that produces information that discriminates against certain populations, these populations can be grouped by factors such as ethnicity, age, gender, sexuality etc.

Bias in AI can occur because it is created by the dominant culture, the values of the business leaders impact policy and developments, and technology companies tend to be staffed by white male people (Smith & Rastagi, 2020, p.21-22). The large data sets that AI are trained on are generally large sections of the internet which contain numerous biases, so tools like ChatGPT can at times replicate bias and prejudiced ideas -even when these tools have been designed to not be biased.

To learn more about effective and ethical strategies for using generative AI tools like ChatGPT, go to the Ethical Use of Artificial Intelligence page.

Section 2: Assessing For Political Media Bias in a News Source

The following information attempts to provide you the reader with language and tools to evaluate media bias critically. In writing this information we have attempted to be neutral without bias toward any particular political ideology. The language and subject matter of the information on this webpage have a focus on political bias in media news reporting. Other types of bias not referred to here include cultural bias (racism), gender bias, sexuality bias, religious bias, wealth/business bias and more. Thinking critically about bias will help you spot these other types of bias as they occur.  

Media Bias in Political Reporting

We expect a news journalist to present an objective, neutral or impartial way that tells both sides of a story in a balanced and truthful way. But news reporters can at times present a biased version of a news event; this can be a small bias, or it can be significant to the point of false information. As we can see from the glossary description of media bias, the driver of the bias may be in cultural perspectives, it may be a result of a personal political preference, or even an institution-wide bias where the owner of a media organisation has a strong political preference and directs the newspaper staff to report news stories in a way that positively values one political party over another -this is political bias.

How is Your Bias Detector?

Election time for politicians is a great opportunity for you to fine tune your bias detector. When big news stories happen, they are typically reported across multiple news platforms including different newspapers, free to air news television and web-based news services.

Watching how different news services report the same story gives you an opportunity to use your critical thinking skills and look for signs of bias in how the story is reported differently. It can even be significant when a news story is, or is not, reported by a news service. Ignoring a significant news story can advantage or disadvantage a political party. Unfortunately, it is not unusual for one news service to ‘spin’ an event as a positive event, and another news service reports the same event as negative.

According to media expert Bill Press, “there is no good definition of spin. It’s easier to say what it’s not than what it is: It’s not the truth. Neither is it a lie. Spin lies somewhere in between: almost telling the truth, but not quite; bending the truth to make things look as good — or as bad — as possible; painting things in the best possible — or worst possible — light” (NBC News, 2002). Politicians can be seen at times attempting to spin information to their advantage, or to the disadvantage or their political opponents.  

But What is Left Wing & Right Wing Bias?

What is left and right wing bias? This is arguable… the following could/should be argued over; it is not suggested as absolute truth, but rather as a starting point for you to learn about this concept. 

According to the University of Sunshine Coast:

“Left wing are considered members of a socialist, progressive, or radical political party or section of a party.

Right wing are considered members of a conservative or reactionary political party or section of a party, generally those opposing extensive political reform.

Nonpartisan are considered as not supporting any of the established or regular parties.

Bipartisan are considered as representing, supported, or characterised by two parties, especially political parties”  (USC, 2022).

Media broadcasters at times support and promote a particular side of politics, while others tend to be more neutral.

Tools for Evaluating Media Bias

Media Bias Fact Check (https://mediabiasfactcheck.com/) (MBFC)

Media Bias/Fact Check (MBFC), founded in 2015, is an independent online media outlet. MBFC is dedicated to educating the public on media bias and deceptive news practices. MBFC’s aim is to inspire action and a rejection of overtly biased media. You can enter pretty much any news organisation into their MBFC website search bar and see a report about their left-right bias and their level of accuracy.

Where do you typically get your news from? Click on the link and see what sort of level of bias and accuracy it has: https://mediabiasfactcheck.com/. Note: MBFC is one opinion, not an absolute truth, once you find the results for your news source, try some internet searches for “Newspaper Name bias”, see what other opinions or evaluations are out there of your news source -but don’t forget to think critically about who is voicing that opinion and are they a reliable source.

Here is one example of an Australian media outlet and the MBFC evaluation:

Here is another evaluation of the media bias of news organisations in Australia:

More Tools for Evaluating Media Bias and Fact Checking

AllSides has a specific page that presents a current news topic and then provides you with three different articles representing that news story, one right bias, one centre, and one right. This is an invaluable resource that allows an easy comparison of how differently one story can be told depending on an organisation's bias. View at: https://www.allsides.com/unbiased-balanced-news

AllSides also features a page that has a variety of tools that you can explore on topics including Media Bias Chart, Misinformation & Disinformation explainer, and the Rate Your (own) Bias page. View at: https://www.allsides.com/schools/resources

FactCheck.Org (use the search button, enter a subject or fact that you are investigating)

• We are a nonpartisan, nonprofit “consumer advocate” for voters that aims to reduce the level of deception and confusion in U.S. politics. We monitor the factual accuracy of what is said by major U.S. political players in the form of TV ads, debates, speeches, interviews and news releases. Our goal is to apply the best practices of both journalism and scholarship, and to increase public knowledge and understanding. 

How to Spot Media Bias

"When trying to spot bias, ask yourself these questions: 

1. What kind of information is it?

News? Opinion? Ad? Does it appeal to your emotions or does it make you think?

2. Who and what are the sources cited and why should you believe them?

 Is the source given? Is the source associated with a political party or special interest group?

3. What’s the evidence and how was it vetted?

What’s the evidence and how was it vetted? Is the source a document? Witness? Or is it hearsay/speculation?

4: Is the main point of the piece proven by the evidence?

Did the sources provided justify the conclusion or main point of the story?

5. What’s missing?

Was there an aspect or point that was not covered or unclear that you are left wondering about?"  

(Source: https://leeuniversity.libguides.com/c.php?g=1178892&p=8619066 / via https://www.americanpressinstitute.org/publications/six-critical-questions-can-use-evaluate-media-content/)

For an expanded version of these 5 dot points, go to the American Press Institute.

What is the Problem With Media Bias?

Media bias can give you a false perception of reality. This can be in small and subtle ways or in significant ways. 

Bias can cause divisions in society

Bias in reporting can often lead to negative stereotypes being used; this can include unfair representation of people and cultural groups in society. This can lead to unnecessary divisions within our society where differences are emphasised or exaggerated.  

It can lead to censorship

Journalists can choose to emit, or leave out, certain facts. Whole stories can be not reported on, or smaller parts of a story can be left out. This can lead to the audience either not knowing about an important issue, or having a biased view that lacks a full understanding.

Politically motivated bias

Journalists and media organisations can work in collusion with government officials or political parties to present a version of the news that is favourable to them. This can be to a small or large degree. It can alter the audience's perception or understanding of events. This can result in individuals having a particular opinion, this can lead to individuals also voting in particular ways which can have a significant impact on the direction that a whole country moves towards. 

Extreme bias can lead to propaganda

When a media organisation is extremely biased towards favouring a governing body, this can lead to completely false and one-sided news reports happening. This can brainwash people to blindly accept what a government wants them to believe. This of course is dangerous and can lead to tragedies like countries going to war for false reasons, politicians stealing large amounts of money etc.

(The section "What is the Problem With Media Bias?" is adapted from the Future Learn website: https://www.futurelearn.com/info/blog/understanding-media-bias-how-credible-are-your-sources).

Looking Ahead

As you read news stories, look for bias. Try to find the same news story reported by different news organisations and compare the articles. Is there overt bias, or are there subtle differences of language that shape the readers perception, are the conclusions different, is there a different emphasis of what is described as significant? If you can spot differences, does this tell you more about the reporter or the news organisation? If people were reading only one version of the news article, would they have an accurate view?

Note: articles labelled “Opinion Piece” are not meant to be neutral, they are personal opinion pieces and are likely to have a bias.

Further Reading

Having a better understanding of different types of cognitive bias that happen in our minds makes for fascinating reading and can contribute to your critical thinking skills development. The article “Cognitive Bias: How to Make Objective Decisions” expertly describes ten different types of bias and provides different strategies to avoid bias in your own thinking. (Being able to identify and describe different types of cognitive bias can be helpful in many subjects -it's a high-level critical thinking skill!)

Learn more about the history of journalistic objectivity from the Wikipedia page: https://en.wikipedia.org/wiki/Journalistic_objectivity

Further Watching

Media Watch is a long-running ABC News critique of Australian media. Each episode is a quick six-minute dissection of the Australian news industry's performance including scathing critiques, at times jaw-dropping bloopers and occasional praise. You might consider cross-referencing Media Watch criticisms of news organisations (including the ABC) with one of the media bias references like Figure 2 above; can errors or bias in reporting be linked to overall bias within an organisation?

Media Watch can be seen via the ABC app or online at: https://www.abc.net.au/mediawatch/

References

Smith, G., & Rastagi, I. (2020). Mitigating Bias in Artificial Intelligence: An Equity Fluent Leadership Playbook. The Center for Equity, Gender and Leadership, Haas School of Business, University of California/Berkeley. https://haas.berkeley.edu/wp-content/uploads/UCB_Playbook_R10_V2_spreads2.pdf Retrieved on 02/08/2023.

Other references have been identified in text.