This was a Causal Inference Seminar at the Harvard Data Science Initiative on 7 March 2024. It is a live seminar with two excellent discussants and lively audience discussion. However, the slides are harder to read here than in the 2026 version of essentially the same talk. The talk is an empirical example of an observational study structured to be insensitive to unmeasured confounding. Part of the goal is to show that decisions made during study design have a big impact on the degree to which unmeasured confounding can explain the conclusions of an observational study.
Books
The best reference for the talk is my book, Introduction to the Theory of Observational Studies (Springer 2025). The book contains the example and uses it repeatedly to illustrate theoretical points. Also relevant is my earlier book, Design of Observational Studies (Springer 2020). A very short, nontechnical introduction to design sensitivity is in Chapter 10, "Design Sensitivity," is in my book, Observation and Experiment: An Introduction to Causal Inference (Harvard University Press, 2017); however, it is limited in scope. In this web page, see the tab for Books.
Articles Directly Related to the Talk
Rosenbaum Paul R. Bahadur efficiency of observational block designs. Journal of the American Statistical Association. 2024 Jul 2;119(547):1871-81. https://doi.org/10.1080/01621459.2023.2221402
Rosenbaum Paul R. Sensitivity analyses informed by tests for bias in observational studies. Biometrics. 2023 Mar;79(1):475-87. https://doi.org/10.1111/biom.13558
Rosenbaum Paul R. Can we reliably detect biases that matter in observational studies?. Statistical Science. 2023 Aug;38(3):440-57. Available from ProjectEuclid DOI: 10.1214/23-STS882
Rosenbaum Paul R, Rubin Donald B. Propensity scores in the design of observational studies for causal effects. Biometrika. 2023 Mar 1;110(1):1-13. https://doi.org/10.1093/biomet/asac054
Rosenbaum Paul R. A second evidence factor for a second control group. Biometrics. 2023 Dec;79(4):3968-80. https://doi.org/10.1111/biom.13921
More Recent Articles Developments of These Issues
These newer articles below go beyond the talk in two ways. In the first and third article, the outcome is length of life, not HDL cholesterol. In the second and third article, a new conditioning tactic increases design sensitivity.
Rosenbaum Paul R. Does a Daily Glass of Wine Lengthen Life? Insight from a Second Control Group. Chance. 2025 Jan 2;38(1):25-30. https://doi.org/10.1080/09332480.2025.2473291
Rosenbaum Paul R. A conditioning tactic that increases design sensitivity in observational block designs. Journal of the Royal Statistical Society B. 2025 Sep;87(4):1085-99. https://doi.org/10.1093/jrsssb/qkaf007
Rosenbaum PR. A New Construction of Evidence Factors in an Observational Study of Light Daily Alcohol Consumption and Longevity. Journal of the American Statistical Association. 2026, to appear. https://doi.org/10.1080/01621459.2026.2624858
Early Articles Concerning These Issues
I have been thinking about these topics for a while. Bahadur efficiency is simpler in the matched pairs case, discussed in the JASA 2015 article below, than in a block design, as in discussed in both the talk and the relevant article above. Block designs are different, and they offer a nearly free increase in design sensitivity. For this, the talk uses rank tests, but it is true quite generally, and the 2013 and 2014 articles below discuss this issue using Huber's M-statistics, a class of statistics that includes the mean. A reasonable theoretical starting point for understanding design sensitivity is the 2010 article in JASA. That quasi-experimental techniques have formal statistical properties relevant to unmeasured confounding is discussed in the two articles below from the 1980s.
Rosenbaum PR. The role of a second control group in an observational study. Statistical Science. 1987 Aug;2(3):292-306. In ProjectEuclid DOI: 10.1214/ss/1177013232 In JSTOR https://www.jstor.org/stable/2245766
Rosenbaum Paul R. The role of known effects in observational studies. Biometrics. 1989 Jun 1:557-69. https://doi.org/10.2307/2531497 In JSTOR: https://www.jstor.org/stable/2531497
Rosenbaum PR. Design sensitivity and efficiency in observational studies. Journal of the American Statistical Association. 2010 Jun 1;105(490):692-702. https://doi.org/10.1198/jasa.2010.tm09570 In JSTOR: https://www.jstor.org/stable/29747075
Rosenbaum PR. Impact of multiple matched controls on design sensitivity in observational studies. Biometrics. 2013 Mar;69(1):118-27. https://doi.org/10.1111/j.1541-0420.2012.01821.x In JSTOR: https://www.jstor.org/stable/41806073
Rosenbaum PR. Weighted M-statistics with superior design sensitivity in matched observational studies with multiple controls. Journal of the American Statistical Association. 2014 Jul 3;109(507):1145-58. https://doi.org/10.1080/01621459.2013.879261 In JSTOR: https://www.jstor.org/stable/24247442
Rosenbaum PR. Bahadur efficiency of sensitivity analyses in observational studies. Journal of the American Statistical Association. 2015 Jan 2;110(509):205-17. https://doi.org/10.1080/01621459.2014.960968 In JSTOR: https://www.jstor.org/stable/24739298