Biases, Fallacies, and Errors in Reasoning I

A useful way to understand the host of thinking mistakes we make is in terms of

1) failures to conduct an adequate search for evidence, and once that search is done,

2) failures to adequately assess that evidence and draw conclusions from it.

When we are confronted with some claim that is alleged to be true like, “Eating sugar makes children more energetic,” or "Strange things happen on a full moon," or "Women are worse drivers than men," first, it will strike us initially without much critical reflection as either correct or incorrect. The inclination to believe or disbelieve is immediate and largely unconscious.

Then, in conformity with that initial impulse to believe or disbelieve, we often conduct a cursory search of our memory for relevant information. We remember cases of children we have seen becoming hyperactive after eating too much candy, we recall gorging ourselves furiously on Halloween candy, we recall hearing many people make the claim. The full moon claim is often remembered when there is a full moon and something strange appears to happen. Emergency room nurses and police, already having the expectation that strange things happen on a full moon

That is, we often search for and find evidence that is merely consistent with the allegation. Unfortunately, for many of our conclusions, this is as far as any serious critical thinking about the claim goes. We are satisfied as a result of this process that the claim is justified and we conclude that it is true.

This procedure is inadequate for several reasons. First, we should resist the temptation to accept the deliverances of common sense or unconscious intuition in these cases and we should resist the influence that that initial inclination has on our subsequent search for relevant evidence.

Error: Believe first, then go find information to support second.

Second, in general, having some evidence that is merely consistent with or seems to support a claim is not enough to render it justified. It turns out that humans are not very good at searching for a completely different class of information that is crucial to the question—what we need to consider before deciding is counter evidence. Until I have done that, I’m not entitled to draw a conclusion about the overall reasonableness of the claim. Being able to search for counter evidence requires that we get good at asking and answering this question: what sort of information, if I did find it, would amount to disproof of the claim in question? That is, what evidence would be inconsistent with it, and how would I find that evidence if it were out there? Take the example of Smith who thinks that Asians are bad drivers. Smith can give you several cases of an Asian he saw driving poorly. The problem, of course, is that some cases from memory that are consistent aren’t sufficient to establish the truth of the claim. One question we should have is, what does he mean? Does he mean that all Asians are bad drivers? That must be false—disproof of this claim would require only a single instance of an Asian who is not a bad driver. Does he mean most? This is much harder to prove—Smith’s recollection of a few cases doesn’t establish the greater-than-50% claim. Nor will finding some cases of Asians who are not bad drivers. Does Smith mean some? That’s surely true, but not very interesting. Some drivers of all racial groups must be bad at it. Does Smith mean that on the whole, the bad driving rates for Asians are higher than the bad driving rates for other racial groups? Perhaps, but gathering the evidence and adequately considering the counter evidence for this claim is a substantial task, and it would be fair to ask if Smith has that body of evidence to justify his claim.

Suppose Smith, the gambler, has devised a method for winning at the slot machines that involves his standing up twice from his chair, turning around once, and tapping his right foot three times. If he plays a slot machine 100 times using this method and wins 4 times, he can’t reasonably conclude that his method works by only citing those 4 wins. So the commonly omitted question is:

Have I adequately considered counter evidence?

Error: consider only the information that I possess that seems to be consistent with the claim.

So when we are confronted with a claim, before we can decide about its reasonableness, we should wonder about the overall adequacy of the information that we have readily available to us. Our memories are unreliable, we tend to look for confirmation instead of disconfirmation, and we tend to belief first and construct justifications second. One way to understand the justification problem is this:

Question: If there were counter evidence to the claim, would it be available in my search?

So I should strive to not overlook information that I have that might disconfirm it. And I should also wonder if the information that I have available to me is adequate with regard to disconfirmation. I might be selectively finding information in my experience that supports a favored hypothesis, or there might be some external factors that are biasing the sorts of information that I am getting. If we ask a cop about the severity of the drug problem in the United States, we will get a different answer than if we ask an Amish preacher.

In fact, the tendencies to believe first and justify later, and to find only confirmation for beliefs are so strong that we all need to work hard to compensate. We need to actively and aggressively seek out possibly disconfirming evidence whenever possible. We need to cultivate this intellectual procedure:

Search Phase:

1) doubt first—don’t lend your assent prematurely.

2) Consider the general adequacy of the information available to you—does it include possibly disconfirming evidence?

3) If it is not adequate, don’t decide, investigate.

4) Ask yourself this question: What would disconfirm the hypothesis? That’s where we should be looking.

Evaluation Phase:

5) Once we are satisfied that the body of information we have is adequate with respect to its potential for confirmation and disconfirmation, then we can decide what the evidence suggests overall.

6) There are several mistakes that we habitually make in the evaluation of a body of evidence:

Confirmation Bias: ignoring or neglecting evidence that would disprove a favored hypothesis, while giving undue weight to evidence or selecting for evidence that supports it.

Confusing Possibility with Probable: Many events are remotely possible like being struck by lightening on a sunny day, or being the victim of a terrorist attack, but it takes much more than their mere possibility to elevate them to the range of probable, or a greater than 50% likelihood.

Finding Evidence that is Merely Consistent with the hypothesis instead of considering evidence that could potentially disprove it.

7) Once we have evaluated the body of evidence while minimizing biases as much as possible, we can draw a conclusion: Gather adequate evidence first, evaluate it fairly, and then believe last.

Assessing the Adequacy of My Own Belief Forming Procedures

We have a more comprehensive empirical picture of how humans form beliefs now than humans have ever had in history. There are countless pitfalls and errors that we fall into, and detecting them can be very difficult, particularly since we are using our cognitive faculties to evaluate the reliability of our faculties. Despite the difficulties, there are a number of procedural questions that we can ask about a particular case where we search for evidence, evaluate it, and draw a conclusion about it. Considering these questions as part of the evidence gathering and evaluation procedure can dramatically improve the accuracy of the resulting conclusion. Habitually considering these issues can develop the epistemic virtues that will make a person a far better thinker and decision maker. Here are the questions, in no particular order.

Is there any data?

What exactly is the data?

Have I conducted an exhaustive search?

If there were significant counter evidence, would my search have found it?

What else could it be?

What would disprove the hypothesis?

Has my enthusiasm for any particular hypothesis affected the evidence I have searched for or emphasized?

Have I adequately considered other alternatives?

Has search satisfaction led me to stop looking prematurely?

Have I thought about it long enough?

Has my enthusiasm for a hypothesis led me to relax evidential standards for it or increase them for competing hypotheses?

Am I prepared to change my mind in light of new or different evidence?

If there are personal, psychological, or social factors that tilt my evaluation of the evidence, would I be aware of it?

Have I given more or less important pieces of information their appropriate amount of weight?

Has the order of my consideration of the evidence affected my evaluation when it shouldn’t have?

Has the recentness or remoteness of some evidence in time affected my evaluation when it shouldn’t have?

Is my memory supplying me with a representative picture of the relevant experiences?

Are there external factors that may be giving me a tilted picture of the facts?

Am I applying principles of justification here that are consistent with the ones I use normally?

Did I sustain a high level of open-mindedness during the search and evaluation phase?

Are the estimates of likelihoods or probabilities that I am employing accurate or realistic?

Would the conclusion drawn withstand a reasonable level of skepticism?