Abstract: A brief survey of the literature on dynamic methods for economics ranging from variational inequalities to Markov games.

Abstract: 

Heterogeneous agent models with aggregate shocks are a class of macroeconomic models used to describe the dynamics of distributions of income and wealth induced by a population of intertemporally optimizing agents facing common stochastic variability and feedback through markets. These are frequently solved numerically by methods based on combining linearization with projection approximation of the cross-sectional distribution and individual agent decision rules. In this talk I provide theoretical and computational foundations for such methods. I characterize equilibria of these models as solutions of a system of stochastic integral equations, describe conditions under which they are amenable to local solution by linearization in function space, and provide convergence conditions for projection-based approximation of local solutions based on perturbation inequalities for infinite-dimensional generalized eigenvalue problems. Numerical representations are provided which are amenable to linearization by automatic differentiation and which yield optimal convergence rates in the sense of information-based complexity. The method and principles for model building are illustrated with an application to a version of the incomplete markets model of Huggett [1993] with continuously distributed idiosyncratic and aggregate income risk.

Abstract: 

We analyze the intergenerational consequences of rare disasters in a calibrated overlapping generations model featuring realistic household portfolios and equilibrium asset prices. Households own houses and additionally trade in bonds and equity. In a disaster, young households suffer from reduced labor income and tightened borrowing constraints. Older households lose a large portion of their savings invested in risky assets. The relative winners are households shortly before retirement, who have a comparatively stable labor income, are not borrowing constrained, and are young enough to benefit from large returns of assets purchased during the disaster at depressed prices. In order to solve the model, we advance contemporary deep learning based solution methods along two complementary dimensions. First, we introduce an economics-inspired neural network architecture that, by construction, ensures that market clearing conditions are always satisfied. Second, we illustrate how to solve models with multiple assets by introducing them step-wise into the economy. These two innovations enable us to reduce the number of equilibrium conditions, that are not fulfilled exactly, and to substantially improve the stability of the training algorithm.

Abstract: 

In this paper, we introduce generalized Markov games, a generalization of Markov games in which the action taken by other players not only determines the reward a player receives at any state but also the set of actions available to it, and we establish the existence of Markov perfect generalized Nash equilibria in concave generalized Markov games. While the computation of a Markov perfect generalized Nash equilibrium is in general PPAD-hard, we show that a policy profile which is a stationary point of the exploitability (i.e., the players’ cumulative maximum regret) can be computed in polynomial-time under mild assumptions. Then, we associate a generalized Markov game with the Arrow-Debreu economy with (possibly incomplete) financial asset markets, and show that the set of recursive competitive equilibria of this economy is equal to the set of Markov perfect generalized Nash equilibria, thereby establishing the existence of recursive competitive equilibrium in this dynamic stochastic general equilibrium model. Finally, going beyond our theory, we introduce generative adversarial policy networks (GAPs): generative adversarial networks comprising two neural networks, a generator (resp. discriminator) which seeks a policy profile that minimizes (resp. maximizes) the players’ cumulative regret for unilaterally deviating from the generator’s policy profile to that of the discriminator. We then specialize GAPs to solve for recursive competitive equilibrium, and experiment with them on a test suite of incomplete Arrow-Debreu markets, demonstrating that they can more effectively handle large numbers of buyers and goods and a variety of shock types than existing solution methods.

Abstract: 

Leveraging their intermediary role, online platforms utilize vast amounts of user data to manage operations, from setting fees and matching queries to informing the launch of their own products (e.g., AmazonBasics, Apps by Apple). While learning from such data may enhance efficiency, concerns and lawsuits have arisen regarding potential unfair competition and selfish value capture. To understand platform behavior and seller-platform dynamics, we start with empirical evidence from Keepa and introduce a multi-agent economic model. In this model, sellers rely on the platform to reach more buyers, while the platform may use sales data from third-party sellers to inform its own product entry decisions, thereby competing with sellers. We begin with a simple Bayesian bandit setting and extend to multi-agent learning under simulated market conditions to study the effects of the platform’s matching and entry policies on third-party sellers. If time permits, I will discuss various regulatory policies on platforms and their implications on overall market efficiency and product diversity.

Abstract: 

Choice architecture and recommender systems both address information overload but have developed largely independently of each other and make strong assumptions about decision-makers’ unobserved preferences. In this paper, we introduce cognitive information filters as an algorithmic approach to choice architecture that mitigates information overload in a more principled and effective manner: our method combines machine learning with a cognitive model of choice behavior to solve the economic problem of nudging or persuading decision-makers by tailoring information to their revealed preferences and cognitive constraints. We first develop a rational-inattention model of multi-attribute choice to describe the behavior of a consumer (receiver) facing information costs. We then use reinforcement learning to solve the information design problem of a sender choosing which options and attributes are accessible to the receiver. Observing only the receiver’s choices, the sender learns from repeated interactions which information is most effective in attaining desirable behavioral outcomes. By inferring preferences from boundedly rational behavior, our methodology can optimize for revealed welfare and hence promises to be (1) less paternalistic than traditional nudging and (2) less susceptible to misalignment than recommender systems optimizing for imperfect welfare proxies such as engagement. This has implications beyond economics and marketing, for example for digital platforms and alignment research in artificial intelligence.

Abstract:  

This paper examines the alignment of inductive biases in machine learning  (ML) with structural models of economic dynamics.  Unlike dynamical systems found in physical and life sciences, economics models are often specified by differential equations with a mixture of easy-to-enforce initial conditions and hard-to-enforce infinite horizon boundary conditions (e.g. transversality and no-ponzi-scheme conditions).  Traditional methods for enforcing these constraints are computationally expensive and unstable.  We investigate algorithms where those  infinite horizon constraints are ignored, simply training unregularized kernel machines and neural networks to obey the differential equations.  Despite the inherent underspecification of this approach, our findings reveal that the inductive biases of these ML models innately enforce the infinite-horizon conditions necessary for the well-posedness.  We theoretically demonstrate that  (approximate or  exact) min-norm ML solutions to interpolation problems are sufficient conditions for these infinite-horizon boundary conditions in a wide class of problems.  We then provide empirical evidence that deep learning and ridgeless kernel methods are not only theoretically sound with respect to economic assumptions, but may even dominate classic algorithms in low to medium dimensions.  More importantly, these results give confidence that, despite solving seemingly ill-posed problems, there are reasons to trust the the plethora of black-box ML algorithms used by economists to solve previously intractable,  high-dimensional dynamical systems---paving the way for future work on estimation of inverse problems with embedded optimal control problems.

Abstract: 

We propose and compare new global solution algorithms for continuous time heterogeneous agent economies with aggregate shocks. First, we approximate the state space so that equilibrium in the economy can be characterized by one high, but finite, dimensional partial differential equation. We consider different approximations: discretizing the number of agents, discretizing the state variables, and projecting the distribution onto a set of basis functions. Second, we approximate the value function using neural networks and solve the differential equation using deep learning tools. We refer to the solution as an Economic Model Informed Neural Network (EMINN). The main advantage of this technique is that it allows us to find global solutions to high dimensional, non-linear problems. We demonstrate our algorithm by solving canonical models in the macroeconomics literature.

Abstract: 

From one perspective, deep reinforcement learning is nothing more than a collection of techniques for approximating solutions to Bellman equations. Its promise — the reason many people find it worthwhile to deal with its many practical frustrations — is that one can find good solutions to dynamic programming problems in settings beyond the reach of other techniques, given only black-box access to a simulation of the environment or game in question. Described in this way, it sounds like a tempting tool for economic modeling: one can simulate an economic environment with potentially complicated dynamics, with the RL training algorithm ensuring that all the decision-making economic agents learn to behave rationally in this environment. There is much recent work in this direction. I will present on some of this work and discuss its successes and limitations, with a focus on the nuts and bolts of RL and on the interplay between economic modeling choices and computational techniques.