Combating Indubiety

We are susceptible to fake news, dubious science and clever advertisers. Our votes can be swayed by duplicitous politicians and terrorist shocks. More and more adults subscribe to irrational conspiracy theories. People who believe the Earth is flat are increasing exponentially via the web. Indubiety (poorly calibrated skepticism) is an impediment to our culture and social progress as serious as illiteracy was in the past, and for the same reasons. There are ways that people can learn to be less gullible and more discerning. We need to teach these ways in schools and colleges.

Indubiety is badly calibrated skepticism. It includes believing too readily on little evidence and thus being open to error or deception, or having unjustified certainty about a belief or being innocent in a way that can be dangerous. It also includes stubbornly or inexplicably not believing well justified inferences despite abundant evidence. Dubiety, in contrast, is a healthy, well calibrated feeling of skepticism that people need to navigate the murky waters of advertising, propaganda, half truths, casual lies, and viral misunderstandings.

Historians talk about the development of western culture as a series of epochs: stone age, followed by the bronze and iron ages, and then the industrial and post-industrial ages, and, most recently, the information age. Each age is heralded by a handful of technological advances that reverberate throughout society, revolutionizing the means of production, transforming social organization, and generating new ideologies. Transition from one age to the next requires large-scale adoption of new skills and ways of thinking. The industrial revolution of the 18th and 19th centuries, for example, resulted in the need for an increasingly skilled workforce and greatly expanded managerial class. Widespread illiteracy presented a formidable obstacle to cities, states, and nations as they attempted to meet the demands of new industry. Forward-thinking nations mounted literacy campaigns to enhance their productivity and international competitiveness. To this day, illiteracy continues as a significant barrier to industrialization in developing countries. However, by the 20th century, developed countries had achieved near universal literacy.

In addition to literacy, citizens of the post-industrial and increasingly participatory democracies of the 20th century required an unprecedented level of mathematical adroitness. Numerical skills once needed only by accountants and a few specialists became necessary as people came to routinely work at complex technological professions and negotiate complicated long-term personal investment and insurance decisions. Moreover, to participate fully in the democratic process, the public became required to comprehend the comparative costs and benefits of important policy alternatives. As with the literacy campaigns of the 19th century, the 20th century saw calls for ending innumeracy as nations with numerate populaces enjoyed a competitive edge in global markets. As a consequence, the basic mathematical competence of the public improved.

The information age is characterized by an unprecedented dependence on the analysis and understanding of complex data at all levels of society. Such data are fundamentally uncertain and all purportedly precise estimates are just that—estimates, which are subject to imprecision, error, and a host of other sources of incertitude and variability. Indubiety is a fundamental contributor to imprudent, inefficient, contentious, or counterfactual decisions in virtually all domains of public and private life in the information age. Thus, just as the industrial age required widespread literacy and post-industrial society required widespread numeracy, so competitive and truly democratic societies in the information age will require dubiety.

If you are interested in our ideas about the need to foster dubiety and how we might combat indubiety, request further information from the RiskComm Study Group at research@riskinstitute.uk.

These issues pervade deep into daily life, well beyond merely professional concerns. With the rise of the internet and hyperconnectivity from social media, paucity of information is no longer the problem. Arguably there is too much information available today. What is in critically short supply, however, is the sophistication to tell what part of this information is trustworthy, what is merely opinion, and what is error or disinformation. What we need to be able to discern is what part of the available information is important. Most people, including most of the professional classes and even many scientists, simply do not have the skills they need to do this.

What is our call to action? We must promulgate a handful of new dubiety tools to a broad swath of the public and foster the sophistication to employ them. Environmental, safety, health, and financial risk analyses are performed by people from all walks of life, including both highly trained experts as well as ordinary citizens. Experts may use complex mathematical models that employ specialized probabilistic techniques to estimate the precise risk (i.e., probability) of possible adverse events. More commonly experts and lay people alike use spreadsheets, rules of thumb, and personal experience to estimate the risks associated with alternative courses of action. What these calculations have in common is that they generally treat information acquired by measurement and experience, and thus the risk calculated using that information, as if it were precisely known. Yet such certainty is seldom, perhaps never, the case.

This ideology of false precision permeates professional practice and results in unacceptable society-wide levels of error and inefficiency that directly and indirectly wastes many billions of dollars every year. Although some small and incremental steps have been taken towards acknowledging this fundamental flaw in analysis practice, a sea change of national proportions on par with the literacy movement of the 19th century and the numeracy movement of the 20th century will be required to effect substantial improvement. In short, decisions between alternatives taken without respect to the true amount and quality of knowledge available are simultaneously under-informed and overconfident, and thus ill-advised. We refer to this overconfidence in the precision of calculations made with imprecise information as indubiety and propose the development of new technologies and the integration of new and existing technologies that will make it possible to generally and comprehensively calculate with and draw conclusions from data of all degrees of quality and quantity. These new technologies could have impacts at the national level, streamlining policy development, reducing the need for and thus cost of litigation, and improving competitiveness in global markets. As with literacy and numeracy, however, the national impact of this proposal will require improved appreciation of and ability to calculate with uncertainty at all levels of society.

Recognition of the cost of such unwarranted overconfidence has spread in recent years, and specialized solutions and workarounds have been developed. For instance, the NIST methods for dealing with measurement error, the numerical probabilistic methods of modern risk analysis, and Bayesian methods generally have made important fundamental but special-purpose contributions. However the methods to calculate with general and gross uncertainty remain underdeveloped because of a handful of significant technical and sociological barriers. Existing methods are of limited general use because they sidestep these barriers either by strictly limiting the kinds of uncertainty to which they are applicable, or by conflating different forms of uncertainty and treating them as if they were the same. Neither of these solutions offers a path, incremental or otherwise, to a generic technology for calculation with generally and grossly uncertain information. Yet generally and grossly uncertain data is far and away the most common, and it is a generic and ubiquitous technology that is needed if broad scale efficiencies are to be realized.

Conclusions based on overconfident analyses result in poor, unclear, contentious, and otherwise inefficient decisions leading to multiple billions of dollars of waste each year. We propose to create new tools that overcome both the technological and sociological barriers and offer analysts and decision makers the methods they need to acknowledge and calculate with all manner of imprecise information at all scales of analysis. We propose to make these tools widely available in the everyday spreadsheet software used for the vast majority of calculations performed, as well as in special-purpose statistical software designed for use by expert analysts. Economic returns from improved decision making efficiency due to this new technology are estimated at $23 billion dollars per year in the environmental risk arena alone. Applications in health, safety, finance, business planning and other sectors would revolutionize the way decisions are made in the face of uncertainty in this country and could account for billions of dollars more in additional productivity each year.

The fundamental qualities of every quantity measured are accuracy, precision, and units of measurement. Interest in analysis of uncertainties has been driven by the need to estimate the accuracy of measurements and computations. The core innovations that are needed are the creation of new web-based technologies for making calculations with quantities that are fundamentally imprecise in nature. This technology extends to both aleatory and epistemic uncertainty, respecting both variability and ignorance by treating them distinctly. The two are dimensions of uncertainty, not unlike real and imaginary parts of a complex number. Both dimensions of uncertainty must be treated together as a two-dimensional object. Fundamental capabilities include descriptive and inferential statistics for data that cannot or should not be summarized with a single precise value, and new graphs and plots for these statistics. Also necessary is the integration of these methods with the arithmetic operations necessary for mathematical modeling performed in common commercially available spreadsheet applications, and universal invisible units propagation, checking, and conversion.

Strategies to combat indubiety and its ill effects on people and society

Develop personal skills
- Know their tricks (Photoshop fakery techniques, lies big and small, but also sowing doubt, Merchants of Doubt)
- Exegesis (follow the money, motivations, conflicts)
- Rhetoric (deploy So what? and Specify)
- Insist on citations, and read them
- Find out sample sizes, wield statistics
- Translate numbers into plain terms like Gigerenzer’s fact boxes
- Develop analogous tools for non-health risks
Deploy resources
- Learn whom to trust
- Snopes, PolitiFact, Hoaxes.org, etc.
- Wikipedia…but you can edit it in 2 seconds
- BBC has been hacked, ‘Bengali’ tweet, crashed by DDOS
- Elon Musk’s “Pravda” is a horrible idea
- Facebook, Google may come with something
- WT.Social (Wikitribune) evidence-based socialized journalism
Collaborate
- Gang up on offenders (e.g., Sleeping Giants on Twitter)
- Hit them in the pocketbook (AdSense)
- Starve them of advertisers; yeah, fake news is mostly about the money
- Push for reform (rather than leaving the community)
- Brexit, #DeleteFacebook won’t solve problems, but democratization, new leadership might
Don’t expect to win right away
- Correcting hoaxes and misconceptions is hard
- 12 years of Wakefield’s autism/vaccine (1998-2010)
- 40 years of Piltdown Man (1912-1953)
- 60 years of Burt’s twin studies (1909-1975)
Reform education to teach the children about hoaxes and uncertainty*
- No one really knows the answer here
- Teach a little history about hoaxes
- Encourage people to be aware of their own ignorance
- Build uncertainty into claims and answers
- Start with the young, interval analysis in middle school
- Teach students how to (politely) question/challenge authority
Restructure social media
- Limit the number of posts to reduce information load
- Curb abuse by bots flooding media with low-quality information to manipulate public discourse (Qui et al. 2017), link
- Reviewing, checking and access to research
  - Snopes, PolitiFact, Museum of Hoaxes, WT.Social, Wikipedia
- Reputation arbiters
  - “B.S. Detector”, a Chrome extension for Facebook link, link
  - Southern Poverty Law Center’s “hate group” designation link
Better practices in engineering and science
- Teach interval arithmetic in middle school, some probability and risk in high school*
- Improve accountability in publication
  - Make motivations plain
  - Universally require conflict-of-interest and incentivization disclosures
  - Publish peer reviews along with the papers
  - Follow Retraction Watch
- Reform communication standards on psychometric principles
  - Never use percent signs
  - Don’t use relative rates or conditional probabilities
  - Use natural frequencies
  - Require sample size information in displays and numeric results
  - Disallow point estimates
  - Treat ensemble specifications as important as measurement units
- Insist on honest uncertainty analysis in all quantitative analyses
- Generalize Gigerenzer fact boxes for common decisions across engineering
- Implement ubiquitous uncertainty projection with Quiet Doubt
  - In all software and devices on the Internet of Things
  - Facilities in our spreadsheets and programming languages to handle uncertainty in calculations
  - Automate uncertainty analysis (it’s too important to leave to humans)
- Quiet Doubt
  - Unobtrusive, like automated spell checking
  - Users don’t have to do anything special
    - Overload functions/operators so users needn’t change cell formulas or scripts
  - Recognize implicit uncertainty in inputs
    - Significant digits, ISO/NIST conventions
    - Hedged expressions like ‘about 23.6’ or ‘almost 160’
  - Adaptive output
    - Only significant digits, interval ranges, or verbose characterisations
  - Smart dependency tracking

*We are interested in developing some course content for middle and high school teachers that could help them to introduce uncertainty into curricula. These teachers often don’t have a lot of flexibility in what they can teach, but such content could be welcomed by teachers of “advanced placement” and “gifted” classes where it could be useful in fostering in students greater sophistication about uncertainty.

Figure redrawn from the famous hoaxed photograph of the Loch Ness monster

Cultural change

Eradicating indubiety would imply substantive changes in many disciplines, including

• Software design: making uncertainty projection and mathematics checking as pervasive and automated as spellchecking is today,
• Spreadsheets: computing natively with uncertain numbers and linguistic hedges (“about 4.8”), and checking units in calculation streams,
• Professional practice: replacing incomplete and misleading conventions such as that of tracking significant digits with radically improved practices,
• Communication: adopting natural frequencies, equivalent binomial counts, and psychometrically appropriate conventions for communicating probabilities,
• Education: introducing notions of uncertainty, and ways to handle it in high school or even middle school, including scenario modeling and interval analysis,
• Medicine: educating doctors and inform patients about the possibility of false results in all medical testing,
• Engineering: distinguishing between aleatory and epistemic uncertainty in computation and characterize predictive capabilities of models both prospectively and via validation,
• Statistics: going beyond just accounting for small sample sizes (which has been the primary focus of the discipline over the entire last century) to address other kinds of uncertainties,
• Computer science: creating programming languages that include uncertain numbers (intervals, probability distributions, p-boxes) as first-class citizens, and
• Mathematics: generalizing probability theory to account for non-Laplacian uncertainty in a theory of imprecise probabilities.

If successful, this effort will initiate a wave of cultural change that revolutionizes the way uncertainty is assessed, calculated with, and used when decisions must be made.

Central role of uncertainty handling skills

As the world fully enters the information age, it has become clear that the numeracy skills of the last age are not sufficient to meet the needs of individual citizens or of society as a whole because they lack a calculus for reckoning uncertainty. Uncertainty arises because all data about the real world are incomplete and imprecise, and our knowledge of how the world works is imperfect at best. While people are aware of this, few methods for drawing conclusions or making calculations that respect the inherent coarseness of our information are used in practice. Instead, calculations are generally made and decisions taken under tacit assumptions that every detail is known with certainty. Missing and censored data, error, natural variability, and sheer ignorance, so pervasive in real world observations, are routinely replaced in calculations with deterministic point estimates, falsely certain information, and the central tendencies of data distributions. Calculations and decisions made in this way may often be good enough for some purpose, but they can sometimes simply be wrong. Decision makers, from top policy officials to private citizens in their daily lives, are overly credulous, and they make statements with little or no expression or even recognition of doubt, regardless of the actual quality of evidence or reliability of inferences. At the same time, people sometimes exhibit an insensitivity to objectively strong evidence, often predicated by the false notion that all facts are negotiable.

This poorly calibrated skepticism is indubiety, which is the combination of credulousness and insensitivity to evidence, exhibited by professionals and the general public alike, is a direct result of having no ready means for accounting for the actual state of ignorance. It has grown into a serious problem in the information age which is characterized by an unprecedented dependence on the analysis and understanding of complex data at all levels of society. Such data are uncertain to some degree and all purportedly precise estimates are in fact subject to imprecision, error, and a host of other sources of incertitude and variability. Indubiety is a fundamental contributor to imprudent, inefficient, contentious, or counterfactual decisions in virtually all domains of public and private life in the information age. Thus, just as the industrial age required widespread literacy and post-industrial society required widespread numeracy, so competitive and truly democratic societies in the information age will require dubiety.

Uncertainty can be categorized into two primary types: variability and incertitude. Variability (also called randomness, aleatory uncertainty, objective uncertainty, or irreducible uncertainty) arises from natural stochasticity, environmental variation across space or through time, genetic heterogeneity among individuals, and other sources. Incertitude (also called epistemic uncertainty, or subjective uncertainty) arises from incomplete knowledge about the world. Sources of incertitude include measurement uncertainty, small sample sizes, detection limits and data censoring, ignorance about the details of the mechanisms and processes involved, and other imperfections in scientific understanding. Although the terminology and mathematical underpinnings are common to both phenomena, variability and uncertainty are fundamentally different. Incertitude can in principle be reduced by focused empirical effort. Variability can often be better characterized by further specific study, but it is not generally reducible. Incertitude and variability also have profoundly different implications for policy, with the former forcing analysts and decision-makers to confront whether and to what extent we should be “better safe than sorry,” and the latter forcing them to confront whether and to what extent “some will be safe while others will be sorry” (Finkel, 2002).

We need to be able to discern when uncertainty can overwhelm us and drain a conclusion of its justification—and just as importantly—when a conclusion is evident despite uncertainty. Professionals but also citizens generally need to be able to make calculations with imprecise quantities. We must also be able to mix together in calculations quantities with very different precisions, some well characterized and some about which we have very little knowledge. We need ways to track the provenance of information, and to infer how reliable conclusions will be that are based on details with different origins or from different sources. Finally, we also need tools that can check the integrity of those calculations so that we can be confident that their results are sensible. This should include straightforward but often neglected checks such as confirming that the calculations even make sense dimensionally and that the units of numbers conform so that we are not trying to add meters and seconds, or raise a number to the "2 inch" power. It is also important to review dangerously overused assumptions about linearity, independence, stationarity, unbiasedness, equiprobability, and uniformity.

One fundamental need is a bridge between interval analysis and traditional statistics. It is not unfair to say that the last hundred years of statistics has focused almost exclusively on accounting for the implications of small sample sizes. With the advent over the last few decades of remote sensing technologies, robotic and microscale laboratory work, and network-distributed observation collection, the limitations of sample size per se have become less and less important. In many situations, “big data” are now being collected. Sample size is no longer the constraint. But having big data does not mean the uncertainty disappears. Other forms of uncertainty such as mensurational imprecision (i.e., the plus-or-minus part of measurements), data censoring, demographic stochasticity, can sometimes be very important, yet they have not received nearly as much attention in statistics. Likewise, distributions, even though they may be based on very large data sets may still not be normal, or independent of other variables. A bridge between these two disciplines would represent a novel and satisfying approach to calculation with the sort of uncertain numbers that are ubiquitous in real world applications. Until the widespread availability of inexpensive personal digital computers that marked the advent of the information age, calculations of this sort simply could not be conveniently made. Techniques dependent on simplifying assumptions required that important uncertainty be ignored, treated qualitatively, or misspecified. However the time is now right to begin calculating with actual instead of ideal quantities, giving full consideration to the uncertain nature of much of what is known.

In the past, most calculations have been performed by hand and were thus limited to closed form solutions or tractable algorithms. To simply achieve any result at all, researchers were forced to make strong simplifying assumptions. For instance, real world measurements might appear to be distributed similarly to a named probability distribution, for instance Gaussian, therefore calculations are made using the closed form solution for that distribution parameterized by the data. Yet “distributed similarly to” and “distributed precisely as” are very different statements of knowledge. In risk analysis in general, probabilities are the answers, and they....

Indubiety may be fed by culturally induced or socially structured ignorance or doubt created by biased focus, historical revisionism, information suppression or strategic delay, misleading advertising, propaganda, distraction or disinformation campaigns or other unknowledge (Schick) or agnotological (sensu Proctor) mechanisms. Other reasons for indubiety include cognitive inertia, and mechanisms to mitigate cognitive dissonance, learned self-defeating behaviours (what Wells colourfully calls ‘stupidity’), and more general agnoiological (Ferrier) causes unrelated to any mischief making or defect owing to fundamental limits on what can be known. Indubiety seems to also be related to what Asimov calls a ‘cult of ignorance’ born from willful anti-science bias or anti-intellectualism more broadly. It is probably not related to the interesting but esoteric phenomena of dilation (Seidenfeld) or negative information (Horodecki et al.) in which gaining some information can actually reduce one’s aggregate knowledge. Examples of such effects are all technical but can be likened to the situation when hiring employees based on resumes and interviews yields worse decisions than hiring based on resumes alone. ...

Additional topics for discussion

Biological purpose of communication
All writing is fiction or advertising
Hoaxes and scams are nothing new
Cottingley Fairies and Sir Arthur Conan Doyle
Yet truth really can be stranger than fiction
Pareidolia
Wishful thinking
Recognizing fakery
Warrants: evidence versus trust and tribe
Exegesis for tracking motives
False flags
Savvy farmers and tinfoil-hat city dwellers
Beautiful websites are considered more trustworthy, especially by the young
Ferson's Quiet Doubt

Europeans ban fake news: Germany has an anti-fake news law, the Network Enforcement Act (NetzDG) which requires internet platforms with over 2 million users to report and scrub fake news, hate speech and other illicit content. French president Macron will seek legislation to allow the media watchdog agency CSA to combat misinformation on social media sites during elections as a threat to democracy.

Disneyland actors must make up a reality: According to former employees, actors at Disneyland are not allowed to answer a guest's question with "I don't know", which the theme park thinks would spoil the magic. They must make up answers even if they are silly for the sake of the illusion.

Assessing the successes and failures of the UK government response to covid:
https://www.kingsfund.org.uk/publications/assessing-englands-response-covid-19-framework
https://www.bmj.com/company/newsroom/uks-response-to-covid-19-too-little-too-late-too-flawed/
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7577497/