“Once more for slow learners, epidemiologists often try to baffle you by describing their work as ‘stochastic’ . Sounds serious, huh? But as the Oxford Shorter English Dictionary shows here, it means ‘guess’ . Pseudoscience works by dressing guesswork in a crisp white lab coat.” Peter Hitchens @ClarkeMicah, 8 May 2020
Let us be clear: distrust in science is good. Good scientists distrust science because they know it is tentative and might change in the future. That is really the whole point of science: with new evidence we adjust the signposts to point us in a slightly new direction with greater knowledge that guides us towards better conclusions. Science doesn't know all, but it knows a lot. Experts don't know everything, but they know some important things. The current discontent seems eager to throw the baby out with the bathwater, misrepresenting scientific discourse in ways that critically undermine trust in researchers and mix fact with opinion. For example, acknowledgements of statistical uncertainties that are used to describe the limits of our understanding and potential for variation have been painted as markers of ineptitude or statistical obfuscation. It is disingenuous and dangerous to pretend there is no distinction between healthy scientific scepticism and wholesale repudiation of expertise.
Recent anti-science rhetoric has railed against modellers attempting to predict the spread of Covid-19. Amongst them was a tweet by journalist Christopher Hitchens suggesting the word ‘stochastic’ means ‘guess’. Hitchens’ tweet adds fuel for popular anger accusing scientists of hiding shaky theory behind a wall of academic language and even outright fraud. Certainly, academia has problems when complex language can be found off-putting, but a stochastic model does not mean it is merely guesswork. Many people have already responded to Hitchens to explain the word's actual meaning in the context of scientific models.
Hitchens’ other tweets make it clear that his comments are intended merely as provocation, which the Internet calls trolling, so they would not necessarily merit any response. But, at the Institute for Risk and Uncertainty, stochastic modelling happens to be our stock-in-trade. We feel it is important to point out the importance of stochastic modelling to science and to mention the contribution of British scientists to its development and adoption across all quantitative sciences.
The (unabridged) Oxford English Dictionary quotes a scholarly article from 1972 which offers the sentence “Refinement in modelling eventuates a requirement for stochasticity” (1972 Journal of Statistical Computation & Simulation 1:42) as attestation for the modern meaning of the word. Stochasticity is eventually needed in any serious scientific modelling simply because the world itself is stochastic. That is, it has elements of randomness in it that preclude our making perfect precise predictions about every detail of a real-world process. For instance, we may be able to predict that a biological population is likely to increase or decrease in abundance in coming years, without being able to say by exactly how much, or even whether its growth will be consistently positive or negative each year. Such populations are affected by conditions that might be known or under our control, but they are usually also affected by things like weather that we neither control nor can predict precisely. Insofar as the population is controlled by weather and other happenstance effects, its change will be stochastic.
If scientists can estimate how variable the future weather is likely to be, perhaps by examining the variation seen in past weather, they can make a better model of population growth. The better model incorporates a probability distribution to model the variable weather. We don't know which weather pattern the population will actually experience in the future, but a collection of possible future weather patterns imply a distribution of possible population growth outcomes. This stochastic approach to modelling populations has been used in biology at least since the 1950s, which, interestingly, is attested to in the Oxford English Dictionary with this sentence:
“A new approach to population dynamics was needed, and quite recently this has been provided by J.G. Skellam in the form of a stochastic model which allows the experimentalist to regard his population as a random variable at each instant in time, and is much more flexible than the earlier deterministic equations.” New Scientist 17:3 (20 June 1957)
Skellam was an important British scientist whose work is famous in both biology and statistics. In this context, a ‘deterministic’ model attempts to make predictions by using precise numbers to unrealistically characterise quantities whose values are changing or unknowable. Maturing from deterministic models to stochastic models is one of science's important accomplishments of the last century.
So what are these models? Models are just assumption analysers. They tease out the logical consequences of the assumptions (which, yes, are presumptions or guesses) that scientists enter into the models for this very purpose: they are looking for which assumptions seem to work and conform with reality, and which don't. This modelling isn't just an unfettered process of throwing in anything that piques our fancy. Artists may enjoy that kind of freedom, but scientists and engineers don't. They must restrict themselves to a method which is reasoned and corresponds to the world that we actually see. But what is that method? There is in fact a special sauce that holds together models and distinguishes good ones from bad ones. The special sauce has four ingredients:
1. Balance units: Do the calculations and equations even make sense? Are you sure you're not trying to add apples and oranges? This may seem obvious but it can trip up even the most intelligent: in 1999, NASA's Mars Climate Orbiter was lost after the maneuver to enter orbit failed because quantities were computed using formulas with incompatible units mixing metric newton-seconds and imperial pound-force seconds. The simple miscalculation scuttled the multiyear $330 million mission. Unit agreement is a fundamental aspect of sound modelling. You don't hear it often enough.
2. Acknowledge uncertainty: Are you accounting for what you don't know? Given what you know you're not sure about, what can you say about what you're trying to estimate? This is where the stochasticity often comes in. We know that people are different and that disease exposure events are varied. We know that we cannot predict everything about about the future, even if we could have perfect knowledge about the present, which we are far from having. A scientist has some level of confidence in real-world observations and knowledge of physical systems. But this confidence has limits, and acknowledging this uncertainty helps describe the boundaries of confidence about a model's predictions. Estimates based on studies with small sample sizes, for instance, will usually be less reliable than those from larger studies. Scientists use strict methods to estimate the confidence of model predictions that depends on the precision and sample sizes of empirical observations used in the model.
3. Match observations: Does the model's output match what we see? Validation is the process of checking how well a model's output matches corresponding observations from the real world. If there is an acceptable match it may be assumed that the model is reflecting reality reasonably well. Of course this favourable match may also be due to pure chance, so further checks should be conducted. Each successful check helps to show that the model is working well.
How models are used depends on the researcher: scientists testing a hypothesis seek to falsify their results, whilst engineers concerned with practical applications need to verify that their model reflects real-life phenomena. Both verification and falsification ask a second question: is the model doing what it is expected to do? Verification is concerned with finding whether the result is true and is informed by building verifying cases, as described above, but on its own it is not enough to trust a model; scientists must beware of being too credulous as their may always be unaccounted errors. Falsification is the complement to this where tests are designed to falsify, or refute, the result and is a fundamental to modern science when investigating a hypothesis.
4. Predict new stuff: What surprises does the model offer? Even more interesting than getting outputs that match what we see is getting outputs that we did not expect. The predictions of unexpected outcomes or estimating the values of quantities that couldn't otherwise be estimated are the main reasons that scientists and engineers use models. We are trying to figure out what is implied but not yet obvious in the combination of observations and data from the real world and the underlying physical or biological processes that are at play.
Public discussions of science often only focus on the validation and falsification aspects of science. Philosophers of science sometimes take this overly limited perspective too, but they understate the importance of the other ingredients. The progress of science depends on falsifying theories, but the application of science to practical problems—such as predicting a pandemic in real time—depends largely on these other aspects of science, which philosophers sometimes call normal science, but which practitioners often just call modelling.
Scientific models are representations of reality and therefore are necessarily simplifications. The famous British statistician George Box famously said, "All models are wrong, but some are useful." In fact, almost all scientific models fail. Sometimes they fail because the simplification left out critical processes that govern a phenomenon, so the model gives wrong predictions. Models can also fail because the modeller neglected to check the units, or misstated the uncertainties, or couldn't match what we see in the data, or failed to predicted anything interesting that wasn't already known. Even the models you hear about, which are the ones that survived their infancy of jealous discussions with colleagues and maybe murderous formal peer review, may still be fundamentally wrong. Plenty of scientific papers have been retracted because of mistakes. Vastly more should be discounted because they studied small or unrepresentative populations, or just turned out to be wrong. Through this process of culling, science hopes to weed out the bad models from the good.
Using as yardsticks our amazing technological advances and our ability to predict some—if not all—things pretty well, it seems to be working. That science thing is working. You don't hear it enough. This means that, to paraphrase Box, all models are wrong, many are not very good, and some are even stupid. But not all models are stupid. Many are clearly very helpful, acting as signposts which guide decision-making under uncertainty; anticipating how structures may fail, rivers flood, or markets fluctuate is crucial when building informed hazard-mitigation policy. Many of these models, including stochastic models, have been woven into the modern systems that underpin our society, including medicine, communication, agriculture, commerce, transportation (remember that?), and entertainment, and we are absolutely dependent on them. Hitchen's suggestion that a stochastic model is just a guess is true, but it is true only in a very limited sense. This sense is so limited that practioners understand it is essentially a lie.
As it so often is, our civilisation seems to be at a crossroads. Will we embrace science and face the future together, or deny science and retreat from modernity? Given the ongoing surge of anti-intellectualism and the increasing political irrelevance of evidence-based policy, it's easy to see us treading the latter path. Existential uncertainties have been stoked by political turmoil, financial crises, climate doomism, and now global pandemic. Their effects are hurting our wallets and our health, but the hazards are abstract and without a common enemy to focus the human impulse to lash out blindly whenever injured or offended, so there seems to be no way to channel the concern and rage. Giles Coren, a restaurant critic and columnist, wrote in The Times, “OK, coneheads, enough of your weird science: The government’s next virus adviser should be an arts grad as the Stem nerds have done nothing but wreck the planet”. This opinion seems to be an extension of Michael Gove's “I think the people in this country have had enough of experts with organisations from acronyms saying that they know what is best and getting it consistently wrong.”
Anti-science hyperbole is nothing new in public and political discourse. An always easy target is science, led by distant experts in their ivory towers who are seemingly forever changing their minds about what the fundamentals are. Who better to blame in an age of complexity and uncertainty than an elite who claim ownership over such terms? Post-crisis deconstruction may again reveal that the unforgivable crime of scientists will have been not being believed. <<call to arms for better scientists to improve their communication skills; sci comm and outreach should be considered a major part of science training and practice>>
-----------------
The video collective Wisecrack has an accessible explanation of how falsification arose from the prior verificationism in the context of the fascinating and somewhat terrifying growth of the conspiracy theory that the earth is flat. Joel Achenbach's article “Why do many reasonable people doubt science?” in National Geographic gives the broader if dispiriting context for the mistrust of science we see lately, contextualising it with various fringe beliefs of flat earthers, anti-vaccination activists, climate deniers, creationists (anti-evolutionists), and others with sundry conspiracy theories about water flouridation or the moon landing, which collectively embody a fringe that threatens to subsume the whole of society. Daniel Rozell recently gave a fascinating talk in our Risk Institute Online series on how attitudes about technological risk play in science policy and public reception of science, and how trust and tribalism have historically played a significant role and continue to do so even today. Researchers in Canada have studied possible effects of anti-science attitudes on behaviour during the current COVID-19 pandemic.
Here at the Institute for Risk and Uncertainty, we used stochastic models to discover the effects of test inaccuracy during widespread testing and its possible consequences for future infection rates in the COVID-19 pandemic. See the resulting paper on MedRXiv. See a journal club meeting at the University of Manchester for an idea of what people in the medical community think about that paper.
IMAGE OF MEDIA/TWITTER ANTI-INTELLECTUALISM HERE
Simon said:
I've had an initial read through. I'll go in and start editing later today, but some initial thoughts:
- are you able to define "stochastic" in a way easily digestible to the lay person (or even academics unfamiliar with it!) Hitchens' argument is definitional so it might be helpful to start there. Also, talking to academics who find themselves in stochastic modelling but without the formal background often see stochastic as a vague term.
- Images! Images to make in interesting. Images to hit a point home. Do you know of any useful and/or pretty ones? Quoting the tweet, alongside a few other ignorant/uncharitable interpretations would be a good way to enrage & engage the audience at the beginning. I'll start collecting a few.
Scott replied:
Thanks, excellent. Very good points! Good editing. <<Let's put these email exchanges on the website.>>
Yeah, the easy definition is easy. "Containing elements of randomness" is the briefest true explanation. Just like evolution contains random mutations, but that doesn't means that evolution is random! A couple of the Tweets responding Hitches are spot on, and provide a better definition for your purpose. Should look at them again.
It's vague because it's technical for sure. And, no doubt, some people may somewhere be using that to protect a priesthood. But some abuse in that vein doesn't invalidate its legitimate use.
In population biology, I started using the word "babies" to talk about what the profession was calling "reproductives" (which was ambiguous as hell) or "recruits" (which was a bit militaristic). I am pleased that people in my local community started using my word, which I think helped to de-mystify the field for the better. My first taste of 'influencer' power! And scientists are almost always really trying to de-mystify their work. We need to do this to convince the funders and each other that it's making sense. Since my run in with the Sigma Xi guy, I've been looking for a de-mystifying word or short explanation for 'stochastic'. Sometimes there is no easy shorthand for a thought (like for 'schadenfreude'), and we have to learn a new word for the concept (like we had to for 'schadenfreude'). A technical term builds a wall against understanding by outsiders, but it can also really be communicative for people doing work in the field. If you tell me you made a stochastic model of population growth, or virus spread, I know what to expect. You can do it different ways, but I can tell if your model is not stochastic, and I will be on the lookout for ways you might have done it wrong.
May I suggest the "tangle" of trajectories as the main image? Maybe a single trajectory as a start, then a couple, then a tangle. We've been successful with that sequence among people who care to learn.
I mean, yes, of course. Everybody knew, or should have known, that it would be scientists who would be the fall guys for punishment-of-the-innocent phase of the covid crisis, just as they were for Ebola hysteria and the financial meltdown of 2007-2008. Nevertheless it is still galling when it lands in your input stream from sophomores as Giles Coren, a restaurant critic (!) and columnist for The Times, who wrote “OK, coneheads, enough of your weird science: The government’s next virus adviser should be an arts grad as the Stem nerds have done nothing but wreck the planet”.
I mean, I get. What's a restaurant critic to do when restaurants are closed during the lockdown? Sure, fan the anti-science flames. What else? As always, the unforgivable crime of scientists was not being believed. <<Cassandra>>
See the <<Red Dawn emails>> for context in the current crisis.
On Sat, May 9, 2020 at 11:44 AM Simon Clark <simon.clark.90@gmail.com> wrote:
Peter Hitchens hates us
https://twitter.com/ClarkeMicah/status/1258679760902934528?s=19
I am so sorry to read his tweet. I had heard his name a long time ago. I hadn't realised he was Christopher's crazy brother. But the Twittersphere's responses are mostly encouraging I think.
One of my very first scientific talks was at a Sigma Xi meeting in my uni final year about a stochastic model of a forest growth model. One of the professors sitting on the front row asked me what "stochastic" meant. I had used the word in the title and about a hundred times during the talk. As the time for questions had already run out, I answered "contains randomness", which I thought was helpfully brief. But he very snidely responded, "Why didn't you just say that?" Apparently I was too pretentious for Sigma Xi.
Maybe we should write a blog post about the tweet. I'll start; you edit:
We are at a crossroads again in our civilisation. Are we going to embrace science and face the future together, deny science and retreat from modernity? Probably the latter I suppose. Without a common enemy to organise the human impulse to lash out blindly whenever injured or offended, there seems to be no mechanism to channel the concern and rage.
Of course distrust in science is good. Good scientists distrust science, and they know it is tentative and might change in the future. That is sort of the whole point of science. But only stupid people throw out the baby with the bathwater. And only disingenuous people pretend they don't understand this distinction.
You want an open mind, but don't open it so much that your brain falls out.
Yes, observing delicate things may necessarily alter them, and more deeply, the Heisenberg uncertainty principle means you can't even theoretically measure position and momentum perfectly, but that doesn't mean you shouldn't seek to improve measurement precision in physics broadly. And it certainly doesn't mean you can't trust measurements.
Suggesting that a `stochastic` model is just a `guess` is true, but it is true only in a very limited sense. This sense so limited that educated people must surely understand it is essentially a lie. Yes, all models are wrong as George Box famously said, and lots are not very good, and some are even stupid. But not all models are stupid. Many are clearly very helpful, and we are absolutely dependent on a lot of these models, including stochastic models, that have been woven into the modern systems that underpin our society, including in medicine, communication, agriculture, commerce, transportation (remember that?), and entertainment.
Models are just assumption analysers. They tease out the logical consequences of the assumptions (which, yes, are presumptions or guesses) that scientists enter into the models for this very purpose. They are looking for which assumptions seem to work and conform with reality, and which don't. This modelling isn't just an unfettered process of throwing anything that piques our fancy. Artists enjoy that kind of freedom, but scientists and engineers don't. They restrict themselves to things that make some sense and correspond to the world that we actually see. There is in fact a special sauce that holds together models and distinguishes good ones from bad ones. The special sauce has four ingredients:
Units agreement Do the calculations and equations even make sense? Are you sure you're not trying to add apples and oranges, or meters and seconds, like NASA inadvertently did when designed the Mars Climate Orbiter that crashed when it tried to enter orbit around Mars? Unit agreement, or dimensional soundness, is a fundamental aspect of modelling. You don't hear it often enough.
Uncertainty acknowledgment Are you accounting for what you don't know? Given what you know you're not sure about, what can you say about what you're trying to estimate? This is where the stochasticity often comes in. We know that people are different and that disease exposure events are varied. We know that we cannot predict every thing about about the future, even if we could have perfect knowledge about the present, which we are far from having. <<>> This is also were questions about sample size often pop up. <<>>
Match observations Does the model's output match what we see? <<yada yada validation yada yada>>
Predicting new stuff Even more interesting than getting outputs that match what we see is getting outputs that we did not expect. The predictions of unexpected outcomes or estimating the values of quantities that couldn't otherwise be estimated are the main reasons that scientists and engineers use models. We are trying to figure out what is implied but not yet obvious in the combination of what we see to be true in observations and data from the real world and what we're pretty sure about the underlying physical or biological processes that are at play.
Public discussions of science often only focus on the validation aspect of science. Philosophers of science sometimes take this overly limited perspective too, but they understate the importance of the other elements.
With failure rates worse than that for small businesses, and even worse than that for marriages, almost all scientific models fail, either because the modeler neglected to check the units, or misstated the uncertainties, or couldn't match what we see in the data, or failed to predicted anything interesting that wasn't already known. Even the models you hear about, which are the ones that survived their infancy of jealous discussions with colleagues and maybe murderous formal peer review, may still be fundamentally wrong. Plenty of scientific papers have been retracted because of mistakes. Vastly more should be discounted because they studied small or unrepresentative populations, or just turned out to be wrong. Through this process of culling, science hopes to weed out the bad models from the good. And, if we use as the yardsticks our amazing technological advances and our ability to predict some--if not all--things pretty well, it seems to be working. That science thing is working. You don't hear it enough.
Simon - first edit:
Once again we are at a crossroads, determining the path of our civilisation: will we embrace science and face the future together, or deny science and retreat from modernity? Given the ongoing surge in anti-intellectualism and the increased political irrelevancy of evidence-based policy, it's easy to see us treading the latter path. Existential uncertainties have been stoked by decadal financial crises, climate doomism, and global pandemic - to name a few. Their effects are hurting our health and our wallets, but the hazards are abstract and without a common enemy to focus the human impulse to lash out blindly whenever injured or offended, there seems to be no mechanism to channel the concern and rage. Science, lead by experts distanced in their ivory towers and whom seemingly are always changing their mind about something or another, is an attractive target. Who better to blame in an age of complexity and uncertainty than an elite who claim ownership over such terms.
Let me be clear: distrust in science is good. Good scientists distrust science because they know it is tentative and might change in the future. That is the whole point of science: with new evidence we adjust the sign-posts to points us in a slightly newer direction and better guide us towards knowledge and understanding. But the current discontent seems eager to throw the baby out with the bathwater, taking and transforming scientific rhetoric in ways that critically undermine trust in researchers and mixes fact with opinion. For example, statistical uncertainties used to report on the limits of our understanding and potential for variation have become examples of ineptitude or statistical obfuscation. It is disingenuous to pretend there is no distinction between healthy scientific scepticism and disowning expertise entirely.
You want an open mind, but don't open it so much that your brain falls out.
Recent anti-science rhetoric was railed against modellers attempting to predict and communicate the spread of the virus SARS-CoV-2, colloquially named the coronavirus. Voices in the media and political establishment have sought to confuse the public and further stoke distrust in scientific expertise with claims of ???. Amongst them was a tweet by journalist Christopher Hitchens quoting a basic-language dictionary of "stochastic" to mean "a guess", adding fuel to the angry rhetoric regarding researchers as intellectual frauds hiding shaky theory behind an obfusctaing wall of academic language. Certainly, academia has problems where complex language can be found off-putting however "stochastic" communicates something more than guesswork: it is the introduction of randomness within a model. This is a pehonomenon reflected in nature: evolution is a process driven by random mutation, but it doesn't mean evolution is random. For example, natural selection ensures that the mutations that are passed down generationally are ones which better adapt the given species to their environment. Where mutations may be random their contribution to the long-term development of a species is determined by the environment within the species must survive.
IMAGE OF MEDIA/TWITTER ANTI-INTELLECTUALISM HERE
Suggesting that a `stochastic` model is just a `guess` is true, but it is true only in a very limited sense. This sense so limited that practioners understand it is essentially a lie. Models are a representation of reality and necessarily a simplification. This means that, to paraphrase George Bof, all models are wrong, many are not very good, and some are even stupid. But not all models are stupid. Many are clearly very helpful, acting as signposts which guide decision-making under uncertainty; anticipating how structures may fail, rivers flood, or markets fluctuate is crucial when building informed hazard-mitigation policy. Many of these models, including stochastic models, that have been woven into the modern systems that underpin our society, including medicine, communication, agriculture, commerce, transportation (remember that?), and entertainment, and we are absolutely dependent on them.
Models are just assumption analysers. They tease out the logical consequences of the assumptions (which, yes, are presumptions or guesses) that scientists enter into the models for this very purpose: they are looking for which assumptions seem to work and conform with reality, and which don't. This modelling isn't just an unfettered process of throwing anything that piques our fancy. Artists may enjoy that kind of freedom, but scientists and engineers don't. They must restrict themselves to a method which is reasoned and corresponds to the world that we actually see. But what is that method? There is in fact a special sauce that holds together models and distinguishes good ones from bad ones. The special sauce has four ingredients:
1. Units agreement: Do the calculations and equations even make sense? Are you sure you're not trying to add apples and oranges? This may seem obvious to many but it's a something that may hound even the most intelligent: in 1999, NASA's Mars Climate Orbiter crashed into the red planet after its trajectory was calculated using both newton-seconds (the standard unit used internationally in science for impulse) and pound-force seconds (common in the Unite States). A 327.6 million dollar mission decimated due to simple miscalculation. Unit agreement, also known as dimensional soundness, is a fundamental aspect of modelling. You don't hear it often enough.
2. Uncertainty acknowledgment: Are you accounting for what you don't know? Given what you know about what you're not sure about, what can you say about what you're trying to estimate? This is where the stochasticity often comes in. We know that people are different and that disease exposure events are varied. We know that we cannot predict every thing about about the future, even if we could have perfect knowledge about the present, which we are far from having. When modelling scientist have a certain level of confidence in real-world observations and knowledge of physical systems. But this confidence has limits and acknowledging this uncertainty helps describes the boundariess within which researchers know a result may exist. <<>> This is also were questions about sample size often pop up. <<>>
3. Match observations: Does the model's output match what we see? <<yada yada validation yada yada>>
4. Predicting new stuff: Even more interesting than getting outputs that match what we see is getting outputs that we did not expect. The predictions of unexpected outcomes or estimating the values of quantities that couldn't otherwise be estimated are the main reasons that scientists and engineers use models. We are trying to figure out what is implied but not yet obvious in the combination of observations and data from the real world and the underlying physical or biological processes that are at play.
Public discussions of science often only focus on the validation aspect of science. Philosophers of science sometimes take this overly limited perspective too, but they understate the importance of the other elements.
With failure rates worse than that for small businesses, and even worse than that for marriages, almost all scientific models fail, either because the modeler neglected to check the units, or misstated the uncertainties, or couldn't match what we see in the data, or failed to predicted anything interesting that wasn't already known. Even the models you hear about, which are the ones that survived their infancy of jealous discussions with colleagues and maybe murderous formal peer review, may still be fundamentally wrong. Plenty of scientific papers have been retracted because of mistakes. Vastly more should be discounted because they studied small or unrepresentative populations, or just turned out to be wrong. Through this process of culling, science hopes to weed out the bad models from the good. And, using as yardsticks our amazing technological advances and our ability to predict some--if not all--things pretty well, it seems to be working. That science thing is working. You don't hear it enough.
THIS PARAGRAPH DOESN'T SEEM TO FIT ANYWHERE??
Yes, observing delicate things may necessarily alter them. More deeply, the Heisenberg uncertainty principle means you can't even theoretically measure position and momentum perfectly, but that doesn't mean you shouldn't seek to improve measurement precision in physics broadly. And it certainly doesn't mean you can't trust measurements.