EBM vs. SBM
Basic Science and Applied Science
A distinction is made between the basic sciences and the applied sciences.
The basic sciences are the accumulated body of scientific knowledge that describes the basic principles of nature. Physics, Chemistry and Biology lie at the base of our scientific understanding of the world. The basic sciences describe the basic facts, laws and theories that provide the backbone of our understanding of how the world works. These facts, laws and theories are so well established through proper scientific methodology that it would take extraordinary evidence to overturn them.
The basic principle of science assumes that nature behaves in a certain way all of the time. We may never be able to know the actual state of affairs of nature per se, but in science, we take it for granted that nature is behaving as it has in the past and as it will in the future. Indeed, we learned in You Can't Have it Both Ways that science can only have any value if and only if this basic principle holds true all of the time.
Applied sciences are concerned with real-world applications of the facts, laws and theories from the basic sciences. Applied sciences generally do not discover new laws and theories. Instead, they apply the knowledge of basic science and use goal-directed ingenuity to achieve practical results. Scientists in the applied sciences assume the principle of reductionism (which basically means that observations in the applied sciences can be explained in terms of basic science's fundamental facts, laws and theories).
Two broad categories of applied science are engineering and medicine.
Engineering applies basic physics and chemistry in a practical way to achieve goals. Through this we get the various disciplines of engineering such as mechanical, civil, electrical, and nuclear, and related fields such as automotive design and aviation. It should go without saying that research in these fields can only happen with the assumption that there are basic laws of physics that will always be valid. A designer of airplanes can only be successful if the basic Newtonian laws of gravity and force are reliable.
Medicine is similar to engineering in that it is an applied science. Medicine applies the scientific knowledge of physics, chemistry and the many subcategories of biology to achieve the goal of promoting health. When we talk about "medical science" in this respect, we are really talking about the science of clinical medicine which proceeds through the collection of evidence through clinical trials. Just as aviation design depends on Newtonian physics, research of the clinical effect of an antibiotic depends on basic scientific concepts like chemistry, physiology, microbiology and (of course) Germ Theory itself.
** It is important to understand that the methodology for acquiring knowledge in the basic sciences proceeds very differently than in the applied sciences. Basic science is explored in the laboratory and usually under tightly controlled conditions. Applied science is explored in the real world of people and their biases.
However, the applied sciences should always proceed under the assumption that they can ultimately be reduced and explained through the basic sciences.
** It is also important to understand that, although basic science is necessary for applied science, it is not sufficient. One should not make practice-level claims in medicine without doing the appropriate clinical research.
This Yogi Berra quote was used in the Scientific Studies in Medicine section and bears repeating here:
"In theory, there is no difference between theory and practice. But, in practice, there is."
Evidence Based Medicine (EBM)
"Evidence based medicine is the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients. The practice of evidence based medicine means integrating individual clinical expertise with the best available external clinical evidence from systematic research." (Sackett et.al.)
The term "evidence based medicine" was coined in 1990. It became increasingly realized that many of our medical treatments were based only on plausibility from the basic sciences, anecdotal evidence and individual experience. Many treatments were assumed to work because they made sense. The drugs worked in test tubes. Doctors relied on past experiences to guide their practices.
It became recognized that practicing simply by experience and opinion is unreliable. Such practice is extremely prone to human biases. This table demonstrates that practice from experience (as inferred by the age of the practitioner) correlates with poor outcomes.
In the 1990's, experience-based treatments were called into question. Physicians, healthcare systems and third party payers began demanding evidence. This lead to an increase in value for treatments that were tested properly. Levels of evidence were developed as we discussed in the Scientific Studies in Medicine section. This was a much needed paradigm shift. The practice of medicine has become more scientific and less prone to errors due to biases and "experience based medicine". Also, the "makes sense" approach to practice (extrapolated from basic science) without proper clinical trials was relegated to the bottom.
The Center for Evidence Based Medicine ranks the reliability of evidence accordingly:
1a. Systematic review of randomized controlled trials
1b. Individual randomized controlled trials
2a. Systematic review of cohort studies
2b. Individual cohort studies
3a. Systematic review of case control studies
3b. Individual case control studies
4. Case series
5. Expert opinion without explicit critical appraisal,
or based on physiology, bench research or "first principles"
Stasis Points Revisited
We now need to briefly revisit the idea of 'Stasis Points' (pronounced stah-sis) in argumentation. Recall from the Argumentation and Logic section that in order for an argument to be successful, the arguer(s) and the listeners should be very clear about the subtle gist of the argument. 'Stasis points' mark the gist, or the angle of the argument.
Changing stasis points in the course of an argument may be referred to as "Moving the Goalpost" (see Logical Fallacies) and is a hallmark of a bogus argument.
1. Conjecture - concerns whether an act actually occurred. There is little point in arguing over the effects of a phenomenon without establishing if the phenomenon actually exists.
2. Definition - concerns what an act actually is. An argument cannot proceed if the topic is not clearly defined.
3. Quality - concerns if an act is justified. For instance, a disease may be treated, but the treatment may not have an acceptable risk/benefit ratio. To argue from Quality, one assumes that the disease exists, has a treatment and both are well-defined.
4. Place - concerns if the argument is occurring in an appropriate forum ("This is not the time, nor the place to discuss this.")
Note - An argument from "place" concedes that the argument may have "quality"; an argument from "quality" concedes "definition"; and "definition" concedes "conjecture".
If one starts an argument from Definition or Quality, the arguer assumes that the thing actually exists (assumes Conjecture). For unlikely claims, such a strategy can be deceiving. This is an important point as we shall see below.
The Face on Mars Effect
What does Mars have to do with medicine? The answer will become clear, but for now, consider the controversial 'face on Mars'. In 1976, the Viking 1 spacecraft was orbiting Mars and photographing its surface. One particular photo showed what appeared to be a clear-cut, sculpted monument of a humanoid face! Many took this as evidence of an advanced civilization that existed on Mars. Conspiracy theories cropped up claiming that the government knows about these civilizations and is covering it up. Even some NASA scientists began to believe that the Martian face was evidence of intelligent life.
The claim that intelligent life exists on Mars is an extraordinary claim. Was this extraordinary evidence?
Years later, another spacecraft surveyed the same area with better photography equipment and technique. The result was less than extraordinary. The legendary face on Mars was actually a mundane mountain range. Recall that we are prone to seek patterns that are relevant to us in randomness.
Low quality studies may yield positive results for unlikely claims. When better studies are done, the 'effect' goes away. In other words, higher quality studies may prove the results of low quality studies to be illusions. If so, the only sensible thing to do is to throw away the 'results' from the low quality studies. It would make no sense to combine the results of the illusory low-quality studies with the high-quality results. The new Mars photos trump the old ones. We do not declare the results to be mixed and inconclusive. We do not call for further photos of ordinary mountain ranges to investigate further.
The Problem with Evidence Based Medicine
Remember, EBM arose out of a need for eliminating ineffective treatments that were merely thought to be plausible and replace them with effective ones that stand up to scientific scrutiny. EBM was a wonderful paradigm shift that was sorely needed.
However, EBM appears to place its emphasis on clinical science (controlled trials) and relegates basic science to the bottom (Level 5 evidence includes claims based on physiology, bench research or "first principles". It should be noted that practice should never be based solely on basic science because such evidence is insufficient in clinical practice.
But, ...
** it has been pointed out that basic science should be considered a necessary starting point for clinical research because basic science provides plausibility.
The argument to justify EBM has inadvertently been made by starting at the 'stasis points' of Definition and Quality. It appears to have skipped the stasis point of Conjecture. In other words, EBM is concerned with evaluating medical claims by defining the claim and evaluating the quality of the claim as defined by its rank on the pyramid of evidence (see above). The burden of proof is much smaller if one does not have to start the argument from the point of conjecture. It would take a greater amount of work to first show that a claim is plausible in the first place.
Shouldn't we consider the first stasis point (Conjecture) when evaluating a claim? Shouldn't we first ask ourselves, based on established scientific knowledge, does the claim make any sense? Does the phenomenon in question exist? Is it even plausible? These are critical questions that one should ask before worrying about randomized controlled trials. Conjecture cannot be assumed, especially when it comes to claims with low levels of plausibility. To argue from the stasis points of definition and quality is to concede conjecture. This may not be warranted. Inadvertently conceding conjecture can provide a wedge for pseudoscience to wiggle its way into the world of legitimate science.
By placing most of the emphasis on clinical trials without first establishing the prior probability of claim, EBM provides such a wedge. Remember in Bayesian terms, the weight of new evidence from clinical trials is factored in with an idea's prior probability when determining the probability of the idea. A claim with very low prior probability is, by our definition, an extraordinary claim. The 'Extraordinary Claims' axiom states that extraordinary claims require extraordinary evidence. Skeptical doctors must think in such Bayesian terms to keep things in perspective. EBM, as it stands today, seems to forget to proportion the probability of a claim to both new evidence and prior probability.
At any level of the EBM pyramid, the significance of a clinical trial's conclusion is determined by the "p value". Remember, the p value really represents the likelihood of wrongly rejecting the null hypothesis. It does not take into account the prior probability of the claim. Proponents of SBM have suggested that the scientific community use the Baye's Factor (or Likelihood Ratio) in lieu of the p value to better account for prior probability.
Supporters of low-plausibility claims can (and do) perform clinical trials. The data that comes out of a clinical trial is only as good as the quality of the trial itself. The Center for Evidence Based Medicine understands that pretty well, hence the pyramid / hierarchy of evidence defined above. But even lower-level evidence (such as case controlled studies and poorly done randomized studies) is still evidence. Poor studies often show small effects. They often show somewhat positive results. These results get published and held up as support for the claim. Systematic reviews (such as those found at Cochrane.org) of poor studies of improbable claims often conclude that, although the claim may have positive effects, the weakness of the studies reviewed prevent definite conclusions. These reviews typically then call for more trials to study the claim.
There are many examples of this. The Cochrane Review of Homeopathic 'Oscillococcinum' (for treatment of influenza) illustrates this point very well. We will learn more about the near-zero plausibility of Homeopathy in the Pseudoscience in Healthcare section. It basically involves diluting a substance in water until that substance is no longer in the water, then using the water's 'memory' of the substance to treat disease. Not only does this idea have very low plausibility (some would say "zero" plausibility), Oscillococcinum is not even a real substance. With such low levels of prior probability / plausibility, how can logical thinkers look at minimal evidence from poor studies and decide that the topic deserves more studies? For an excellent review of these concepts, please read Should Cochrane Call for More Studies Into Homeopathy?
In response, more studies are then done, and, as the quality of the studies increases, the measured effect of the unlikely claim diminishes or vanishes. It is analogous to the Face on Mars issue above. However, unlike in space exploration or other sciences, the issue is usually considered inconclusive, thereby still allowing a sense of legitimacy.
Sometimes the authors conclude that science is not equipped to evaluate their particular claim. However, as we have concluded in the last section, you can't have it both ways. Bizarrely, authors may conclude that not only does their treatment work, but so does the placebo! By not first considering basic science and prior probability, EBM as it stands today can lead to a rabbit hole filled with unlikely and implausible claims.
This is not to say that we need a plausible, basic science explanation for all claims. We can (and do) rely on evidence only from clinical trials to support the use of many treatments for which we do not have a basic science explanation. Although science tends to fill in the missing mechanisms as we discover new aspects of basic science, we have used many treatments with only clinical evidence. The value of hand washing was discovered as a result of the clinical studies done by Ignaz Semmelweis. There was no basic science of microbiology at the time. The basic neurophysiology of many psychotropic medications was unknown (and still is in some cases). They have been used because clinical trials confirmed that they work.
So, a known basic science mechanism of action is not really needed to study a claim. However, ** a claim should not contradict established basic science knowledge**. The above examples did not contradict any scientifically established facts, laws or theories of physics, chemistry or biology. In fact, the clinical knowledge that came from these examples sparked the search for new basic science knowledge.
A claim that would require established basic science to be false in order for the claim to be true, would be an extraordinary claim indeed. It may be possible that a law of physics is incorrect, but it is more likely that the claim is incorrect (remember Occam's Razor?).
** Prior probability and plausibility should matter. They do matter.
Are RCT's Always Appropriate?
The majority of medical claims should be studied with RCTs. Many plausible ideas have been overturned by good clinical science (see Medical Practices Not Supported by Science). It seems that there are practices that carry a near 100% prior probability of being true, and that randomized controlled trials would be unethical. For example, one would not wish to test the use of insulin in diabetic ketoacidosis against placebo. Few would consider testing the concept of setting a complex long bone fracture, or surgically repairing a perforated appendix. A tongue-in-cheek article was published in the British Medical Journal looking at the efficacy of parachute use to prevent death and major trauma related to gravitational challenge!
Similarly, we could ask if RCTs are appropriate for claims with extremely low scientific prior probability (claims that would require established basic science to be incorrect). We have seen above that RCTs in such situations usually produce equivocal results which are deemed 'inconclusive', followed by calls for more studies. The systems of practice in the Pseudoscience in Health Care section exemplify this. If RCTs are done on low probability claims, the results should be unequivocally positive to overcome this hurdle. This has hardly ever happened.
Science Based Medicine
In 2008, a group of doctors/ science writers coined the term "Science Based Medicine" (SBM) as an answer to the prior probability 'blind spot' of EBM. Doctors Steven Novella, David Gorski, Harriet Hall, Wallace Sampson, and Kimball Atwood started The Science Based Medicine blog. It is the current resource for SBM's pertinent issues. To borrow a phrase from Rationally Speaking author Massimo Pigliucci, SBM "explores the borderland between science and nonsense" in the field of medical science. SBM takes a Bayesian approach to evaluating claims by combining a claim's prior probability and new evidence from clinical research to determine the claim's current probability. This approach places value on both basic science and applied science.
"The idea of SBM is not to compete with EBM, but a call to enhance it with a broader view: to answer the question “what works?” we must give more importance to our cumulative scientific knowledge from all relevant disciplines." Paul Ingraham
SBM recognizes full well that basic science is not sufficient to justify practices. Indeed, much of pseudoscience stems from this mistake. SBM asks us to consider the basic science plausibility of a claim before committing to a randomized controlled trial. It asks us to consider both basic science and clinical research. There really should be no conflict. Some avid proponents of EBM appear to think that SBM proponents value basic science over clinical trials. This is not the case. SBM supporters simply want to consider the basic science first, or at least alongside clinical trials. Claims that contradict basic science should require far more evidence from clinical trials than should plausible claims. The more extraordinary the claim, the more extraordinary must be the clinical evidence.
Many avid supporters of EBM don't seem to see bizarre, unscientific claims as being a real problem. Such claims really aren't on their radar. As a result, pseudoscience has already wedged itself into prominent and local centers of academic medicine under the guise of 'Integrative Medicine". Dr.'s Gorski and Novella published an article explaining the blind spot of EBM with respect to "Complementary and Alternative Medicine" (CAM). They state:
"Evidence-based medicine (EBM) assumes that treatments do not reach the stage of RCTs without having amassed sufficient preclinical evidence to justify the effort, time, and expense of RCTs, as well as the use of human subjects. The EBM paradigm resembles the illustration [**Note -- see the above pyramid for a representative illustration**], with observations and discoveries in basic science leading to in vitro work in cell culture, which leads to in vivo experiments and observations in animal models, which, if promising, ultimately lead to clinical trials. Of course, this is a grossly simplified model. Observations from each step frequently cross-pollinate other steps, and the progression is rarely as neat as illustrated. Even so, the major assumption underlying EBM is that by the time an investigational treatment is ready for RCTs it has passed all preclinical tests and has thus demonstrated biological plausibility. Before, CAM or IM, treatments without biological plausibility and compelling evidence from preclinical studies and pilot clinical trials usually did not reach the stage of RCTs. Indeed, so integral to this process is biological plausibility based on preclinical data that the Declaration of Helsinki states, ‘medical research involving human subjects must conform to generally accepted scientific principles, be based on a thorough knowledge of the scientific literature, other relevant sources of information, and adequate laboratory and, as appropriate, animal experimentation’."
SBM considers concepts that we have learned in earlier sections of this site. Remember, skeptical doctors utilize basic skeptical tools such as 'Hume's Fork' / 'Falsifiability' (Is the idea even science to begin with?), 'Occam's Razor' (Is there a more plausible explanation?), 'Extraordinary Claims Require Extraordinary Evidence' (the basis for Bayesian methodology), and 'Correlation is not Necessarily Causation' (the fundamental logical fallacy that fuels many implausible claims).
It seems that the idea of Science Based Medicine should be obvious. It is possible to think that we are studying the 'effects' of something that isn't actually there. If we do not start from the stasis point of conjecture, and instead begin with the assumption that an idea is real, then clinical studies can be designed to examine the idea's 'effects'. At best, poor studies may show some positive results due to the nature of 'poor' studies. At worst, no effects will be shown and the idea will be declared 'inconclusive' and no recommendations will be made. Inevitably, more studies will be called for.
Contributor Harriet Hall, MD illustrates this with her idea of "Tooth Fairy Science". Tooth Fairy science seeks explanations for things before establishing that those things actually exist. For example:
"You could measure how much money the Tooth Fairy leaves under the pillow, whether she leaves more cash for the first or last tooth, whether the payoff is greater if you leave the tooth in a plastic baggie versus wrapped in Kleenex. You can get all kinds of good data that is reproducible and statistically significant. Yes, you have learned something. But you haven’t learned what you think you’ve learned, because you haven’t bothered to establish whether the Tooth Fairy really exists."
Conclusion
SBM seems to encompass the spirit of what EBM started out to achieve. EBM came about as a tool to establish the scientific value of medical claims. The proponents of EBM had the right idea. As touched on in the Promoting Critical Thinking in Medicine page of this site, the practice of medicine is an art, but the practice of medicine must be informed by the science of medicine. Medical science deals with scientific knowledge (facts, laws and theories) as it pertains to medicine. EBM is a way to help doctors sort out the value of this knowledge.
Because EBM places all of the emphasis on clinical trials, it forgets to ask the first, most basic question of all: does the idea make sense? Remember, science helps us understand how things are. Philosophy helps us figure out what we ought to do. SBM's philosophy is that we ought to consider prior probability and plausibility from basic science when determining if a claim is real enough to study. If the claim passes this test, then it should be studied with proper, well-executed clinical research. Science Based Medicine adds an extra layer of the Philosophy of Science to medicine.
The call for SBM is not a call to end EBM. It is a call to advance the evidence-driven paradigm; to fill in the crack that lets in the wedge of pseudoscience. They should be one and the same.
If we recognize this, we just might get it right.
John Byrne, M.D.
Further Reading
(For in-depth discussion of these issues, please read these key articles from the Science Based Medicine blog.)
Prior Probability: The Dirty Little Secret of “Evidence-Based Alternative Medicine”
Prior Probability: the Dirty Little Secret of “Evidence-Based Alternative Medicine”—Continued
Prior Probability: the Dirty Little Secret of “Evidence-Based Alternative Medicine”—Continued Again
Of SBM and EBM Redux. Part I: Does EBM Undervalue Basic Science and Overvalue RCTs?
Of SBM and EBM Redux. Part II: Is it a Good Idea to test Highly Implausible Health Claims?
Of SBM and EBM Redux. Part III: Parapsychology is the Role Model for “CAM” Research
Of SBM and EBM Redux. Part IV: More Cochrane and a little Bayes
References and Links
"Why Science-Based Instead of Evidence-Based? - Save Yourself." 2009.
<http://saveyourself.ca/articles/ebm-vs-sbm.php>
"First principle - Wikipedia, the free encyclopedia." 2005.
<http://en.wikipedia.org/wiki/First_principle>
"Fundamental science - Wikipedia, the free encyclopedia." 2004.
<http://en.wikipedia.org/wiki/Fundamental_science>
"CEBM > Resources > Glossary." 2009.
Jackson, Rodney, and Gene Feder. "Guidelines for clinical guidelines: a simple, pragmatic strategy for guideline development." BMJ: British Medical Journal 317.7156 (1998): 427.
"Science-Based Medicine » Ososillyococcinum and other Flu bits." 2011.
<http://www.sciencebasedmedicine.org/index.php/ososillyococcinum-and-other-flu-bits/>
Smith, Gordon CS, and Jill P Pell. "Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials." BMJ: British Medical Journal 327.7429 (2003): 1459.
<http://www.bmj.com/content/327/7429/1459>
Timmermans, Stefan, and Aaron Mauck. "The promises and pitfalls of evidence-based medicine." Health Affairs 24.1 (2005): 18-28.
<http://content.healthaffairs.org/content/24/1/18.full>
"Clinical trials of integrative medicine: testing whether magic ..." 2014. 24 Aug. 2014
<http://www.cell.com/trends/molecular-medicine/fulltext/S1471-4914(14)00103-8>
"NeuroLogica Blog » Andrew Weil Attacks EBM." 2011.
<http://theness.com/neurologicablog/index.php/andrew-weil-attacks-ebm/>
"McTimoney Chiropractors told to take down their ... - The Quackometer." 2009.
<http://www.quackometer.net/blog/2009/06/chiropractors-told-to-take-down-their.html>
"Tooth Fairy science - The Skeptic's Dictionary - Skepdic.com." 2010.
<http://skepdic.com/toothfairyscience.html>
Madsen, Matias Vested et al. "330 acupuncture treatment for pain: systematic review of randomised clinical trials with acupuncture, placebo acupuncture, and no acupuncture groups." Bmj 338 (2009).