Abstract:
Economists are often asked deceptively simple questions: if (x) changes, what happens to (y)? The difficulty is that, outside the laboratory, (x) rarely changes in isolation. Prices respond to demand, teachers favor good students, and safety courses are attended by the most timid. Correlation alone therefore seldom reveals what would happen under any kind of policy change. This talk offers an intuition-first introduction to how economists attempt to recover causal effects from messy real-world data. The talk explores why prediction and causation require different kinds of evidence, how economic models can expose the failures of naïve comparisons, and how natural experiments can sometimes reveal the missing counterfactual. Three case studies bring these ideas to life: the consequences of bank failures for firms, the surprising relationship between shingles vaccination and dementia, and the debate over whether remote work or AI is responsible for deteriorating outcomes among early-career workers. No prior economics is required.
Bio:
Yannick Schindler is a Senior Research Economist at the Ellison Institute of Technology in Oxford, where he leads research teams studying how AI and robotics are reshaping the economy, with a particular focus on their impacts on science and innovation. He has previously held research positions at Stanford University, Princeton University, London Business School, and the European Central Bank, and holds a PhD in Economics from the London School of Economics. He also guest lectures at the University of Oxford.
Summary:
Focus: overview of economic modeling
Key question: if x changes, what happens to y?
Data modeling culture: linear regression, logistic regression, cox model
98% of statisticians, 99.99% of economists
Goodness-of-fit test, residual examinations
Algorithmic modeling culture: decision trees, neural networks
Metric: predictive accuracy
Challenges:
Confounding:
If I know x, I can predict y but x may not actually cause x. It may be that some z causes both x and y, which is why x and y correlate
y=f(x) + error (unobserved phenomena that may matter a lot)
Desired object: effect of changing a given x from a to b on the average outcomes of a population
We never observe both x=a and x=b for the same individual/unit, we we must infer
Example: impact of going to war on people’s subsequent health and employment
But… people join the military for reasons that independently affect their subsequent outcomes
The Vietnam war draft randomized people’s participation in war, which allows us to study this phenomenon
Economists leverage such “natural experiments”
Example: measure impact of water quality on health
Natural experiment: random assignments in military to locations with different water quality
Very similar populations of people drinking different water, assigned randomly
Controlled experiments allow researchers to directly manipulate the system, where this is ethical (e.g. hospital trials)
Models
Physical analogy: Equilibrium of a pressure/temperature system
An external driver is affecting the system via a pistons, which is the real causal driver
The data measures pressure, temperature pairs
We can train a model on the data but will miss the causality
Pressure and temperature don’t cause each other
The piston causes both
Economists typically study systems that are assumed to be in equilibrium
Shocks are a great way to study causal drivers
If there’s a correlation that may be non-causal, we want to actively break it via randomization
Example: what’s the causal connection between health of financial institutions and unemployment?
Dataset: 30m loan documents to understand which small businesses use which banks
Trace out the impact of local bank failures on small businesses
Observation: most such failures don’t cause businesses to lose their money, just a switch from one bank to another, so does it really matter?
Challenge: many potential correlations between failing banks and businesses that use them (do the banks mess with the businesses or do the businesses make the bank fail?)
The major driver of a business choosing a given bank is proximity to the nearest branch
Over time, as banks buy/sell branches this changes
These acquisitions could not have been predicted by businesses, so these acquisitions act as an “instrument” (randomization on drivers)
Another: a single entity can own multiple banks
Can move bad debt to just one to let just it fail
To control this, the requirement is now to make banks within the same group be responsible for each other’s debt
This creates randomness in bank failures since banks within the same group an unrelated
Example: natural experiment for establishing causal link between shingles vaccine and dementia
At one point in time the Welsh government made a certain population eligible for the shingles vaccine
Birthdate threshold is based on birthday
Individuals on both sides of the threshold are identical other than their eligibility
So can study shingles->dementia causal link for people around this threshold
Example: causal impact of AI on junior hiring (people with little experience)
Observation: the recent trend of stops started mid 2021, before ChatGPT
What else happened? This is mid-/late-COVID, so work from home was also a major change
Exposure of skills to AI correlates well with skills that can be done from home
So, perhaps if people are working from home, they can’t train new workers, making them less productive…
Approach:
Plot the drop in junior employment for work-from-home exposed jobs, controlled for GenAI exposure (big effect)
Compare to junior employment for GenAI-exposed jobs, controlled for work-from-home exposure (small effect)