Guillermo Alvaerz
Sequential Optimal Contracting in Continuous Time
I will present a Principal-Agent problem in continuous time with multiple lump-sum payments (contracts) paid at different deterministic times. We use BSDEs to reduce the Stackelberg game between the Principal and the Agent to a standard stochastic optimal control problem. We apply our result to a benchmark model to investigate how different inputs affect the Principal’s value. This is a joint work with Erhan Bayraktar, Ibrahim Ekren, and Liwei Huang.
Bahman Angoshtari
Predictable Forward Performance Processes for Loss-Averse Agents
Predictable forward performance processes (PFPPs) are stochastic optimal control frameworks for an agent who controls a randomly evolving system but can only prescribe the system dynamics for a short period ahead. This is a common scenario in which a controlling agent frequently re-calibrates her model. In this talk, I discuss a new class of PFPPs for loss-averse agents who update their loss threshold and loss-aversion level throughout their investment horizon. Construction of these loss-averse PFPPs reduces to solving a sequence of inverse optimal control problems, in which the value function is known and a (linear-concave) utility function is to be found. Existence and uniqueness results for the inverse problems are obtained by the analysis of a corresponding Fredholm integral equation. Illustrative numerical examples are provided in the context of conditionally complete Black-Scholes model. Joint work with Shida Duan and Thaleia Zariphopoulou.
Shuoqing Deng
Distributional-Constrained multiple optimal stopping
In this work, we consider the distributional-constrained multiple optimal stopping problem. First, we will prove a duality of the problem; then by considering the appropriate definition of stop-go pair in the enlarged space, we prove the associated monotoncity principle of the problem.
Arash Fahim
Boosting efficiency in computational methods for continuous-time control, PDEs, and finance
In certain problems, in optimal control, reinforcement learning, and partial differential equations, a coarse discretization does not provide a sufficiently accurate answer and a fine one is resource intensive. In this talk, we discuss three approaches which can independently improve the efficiency of existing numerical schemes.. One of the ideas allows the computational resources to be allocated adaptively in multiple steps. In this method, multi-scale policy gradient method, our theoretical study suggests implementation strategies for any arbitrary desired level of efficiency. In another study, we replace Euler-Maruyama discretization with Walk on Sphere, and incorporated deep learning to solve linear PDEs in domains with challenging geometry. Our numerical study shows a reduction of 76.32% in error while using only 8% of samples compared to the conventional walk on sphere. Finally, in the light of recent Schrödingerization of linear equations which allows quantum computers to solve linear PDEs exponentially faster than classical solvers, we developed a method which reduces fully nonlinear PDEs into a maximization problem on a class of linear PDEs. Then, we analytically characterized how to implement a gradient ascend scheme to solve linear equations to approach the fully nonlinear one.
Qi Feng
Constrained Sampling via Langevin dynamics based MCMC
In this talk, I will discuss the constrained sampling problem, where the goal is to sample from a target distribution defined on a constrained domain. I will focus on how boundary conditions and the geometry of the loss function landscape influence the corresponding Langevin dynamics. Both theoretical convergence guarantees and numerical examples will be presented.
Bingyan Han
Goal-based portfolio selection with mental accounting
We present a continuous-time portfolio selection framework that aligns with goal-based investment philosophy and mental accounting behaviors. In this framework, an investor with multiple investment goals constructs separate portfolios, each corresponding to a specific goal, with penalties imposed on fund transfers between these goals, referred to as mental costs. By applying the stochastic Perron's method, we demonstrate that the value function is the unique constrained viscosity solution of a Hamilton-Jacobi-Bellman equation system. Numerical analysis yields several insights: the free boundaries show complicated shapes with bulges and notches; the optimal investment in one portfolio depends on the wealth level of another; investors must diversify both among stocks and across portfolios; and they may postpone reallocating surplus from an important goal to a less important one until the former's deadline approaches.
Xihao He
Comparison for semi-continuous viscosity solutions for second-order PDEs on the Wasserstein space
In this paper, we prove a comparison result for semi-continuous viscosity solutions of a class of second-order PDEs in the Wasserstein space. This allows us to remove the Lipschitz continuity assumption with respect to the Fourier-Wasserstein distance in AriX: 2309.05040 and obtain uniqueness by directly working in the Wasserstein space. In terms of its application, we characterize the value function of a stochastic control problem with partial observation as the unique viscosity solution to its corresponding HJB equation. Additionally, we present an application to a prediction problem under partial monitoring, where we establish an upper bound on the limit of regret using our comparison principle for degenerate dynamics.
Yu-Jui Huang
Partial Information in a Mean-Variance Portfolio Selection Game
We consider finitely many investors who perform mean-variance portfolio selection under relative performance criteria. That is, each investor is concerned about not only her terminal wealth, but how it compares to the average terminal wealth of all investors. At the inter-personal level, each investor selects a trading strategy in response to others' strategies. This selected strategy additionally needs to yield an equilibrium intra-personally, so as to resolve time inconsistency among the investor's current and future selves (triggered by the mean-variance objective). A Nash equilibrium we look for is thus a tuple of trading strategies under which every investor achieves her intra-personal equilibrium simultaneously. We derive such a Nash equilibrium explicitly in the idealized case of full information (i.e., the dynamics of the underlying stock is perfectly known) and semi-explicitly in the realistic case of partial information (i.e., the stock evolution is observed, but the expected return of the stock is not precisely known). The formula under partial information consists of the myopic trading and intertemporal hedging terms, both of which depend on an additional state process that serves to filter the true expected return and whose influence on trading is captured by a degenerate Cauchy problem. Our results identify that relative performance criteria can induce downward self-reinforcement of investors' wealth--if every investor suffers a wealth decline simultaneously, then everyone's wealth tends to decline further. This phenomenon, as numerical examples show, is negligible under full information but pronounced under partial information.
Melih Iseri
The Learning Approach to Games
This work provides a unified framework for exploring games. In existing literature, strategies of players are typically assigned scalar values, and the concept of Nash equilibrium is used to identify compatible strategies. However, this approach lacks the internal structure of a player, thereby failing to accurately model observed behaviors in reality. To address this limitation, we propose to characterize players by their learning algorithms, and as their estimations induce a distribution over strategies, we introduced the notion of equilibrium in terms of characterizing the recurrent behaviors of the learning algorithms. This approach allows for a more nuanced understanding of players, and brings the focus to the challenge of learning that players face. While our explorations in discrete games, mean-field games, and reinforcement learning demonstrate the framework's broad applicability, they also set the stage for future research aimed at specific applications.
Ali Kara
Reinforcement Learning for Stochastic Control under Weak Feller Continuous Models and General Information Structures
I will first present a class of stochastic iterations defined by potentially non-Markovian stochastic processes. The convergence of these iterations is established under an asymptotic ergodicity condition on the underlying processes. Next, I will revisit the Q-learning algorithm, one of the most widely used reinforcement learning algorithms, and show its convergence for control problems where the perceived state variable of the controller does not follow a Markovian structure. I will provide a precise
characterization of the limit of the iterates, along with conditions on the environment and initializations necessary for convergence. In particular, we will see that the limit of the iterations corresponds to the optimal value of an auxiliary Markov decision process, which can be used for the optimality analysis of the learned control policies.
Finally, I will show how these iterates can be used to learn control policies for partially observed Markov decision processes (POMDPs) by employing a sliding window of finite memory observations as the perceived state. The analysis reveals that the near-optimality of the learned policies for POMDPs is closely linked to the stability of the controlled non-linear filter."
Christian Keller
Mean viability and path-dependent 2nd-order HJB equations
I present a new approach of proving uniqueness for viscosity solutions of fully nonlinear path-dependent 2nd-order HJB equations. This approach is purely probabilistic. It uses the concept of mean viability and the closely related notion of quasi-contingent solution. Unlike all existing methods in the literature, my approach does not rely on finite-dimensional results.
Donghan Kim
Roughness of continuous paths via Schauder representation
We present a notion of pathwise p-th variation of a continuous function along a given partition sequence in terms of ℓp-norm of Schauder coefficients. We compare this characterization with the fact that the Hölder exponent of the function is connected to ℓ∞-norm of the Schauder coefficients.
Mike Ludkovski
Groundwater Markets
We introduce the problem of groundwater trading, capturing the emergent groundwater
market setups among stakeholders in a given groundwater basin. The agents optimize
their production, taking into account their available water rights, the requisite water
consumption, and the opportunity to trade water among themselves. The goal is to study the resulting Nash equilibrium of a nonzero-sum noncooperative game with N agents. In the first half of the talk, I will discuss the 1-period setting, providing a full characterization of Nash equilibria. The second half would be devoted to the multi-period game where agents may bank their water rights in order to smooth out the intertemporal shocks. I will conclude with some preliminary numerical algorithms for the resulting problem. (Joint work with I. Cialenco)
Dominykas Norgilas
Injective martingale couplings
The classical Strassen's theorem states that, if X and Y are two random variables in convex order, then there exists a martingale with marginal distributions matching the laws of X and Y. In this talk we show how to build one such martingale that has the following additional feature: given a terminal value of this martingale, we know its whole path.
Jinniao Qiu
Recent Progress on Stochastic HJB Equations
In this talk, we shall present the recent progress in the study of stochastic Hamilton-Jacobi-Bellman (HJB) equations, which arise naturally in the context of non-Markovian control problems with random coefficients. The non-Markovian nature of these problems may also involve path dependence or mean-field interactions, in addition to general randomness in the coefficients. The discussion will cover various aspects, including the well-posedness of such stochastic HJB equations, numerical approximation methods, and their applications.
Ronnie Sircar
Some Continuum Aggregate Games
Some recent game-theoretic models have interaction between players through the aggregate sum of their actions and/or states. This includes two models of cryptocurrency mining, Cournot-type models of oil production and prices, and two related models of renewable capacity expansion. We discuss an approach in which a sum is approximated by the (large) number of players multiplied by the continuum mean, in order that mean field game analysis can be applied. We study convergence of equilibria in very simple (static) settings and give preliminary general results.
Qingshuo Song
Ergodic Non-zero Sum Differential Game with McKean-Vlasov Dynamics
In this talk, we explore a two-player ergodic game problem with mean field terms. To determine the Nash equilibrium, we introduce an auxiliary control problem. A verification theorem establishes the connection between this auxiliary problem and a system of coupled Hamilton-Jacobi-Bellman equations. As an example, we analyze the LQG problem and derive the corresponding algebraic Riccati equations.
Gu Wang
Continuous Policy and Value Iteration for Stochastic Control Problem and Its Convergence
We propose a policy iteration algorithm, in which the approximations of value function and the control are updated simultaneously, for both the entropy regularized relaxed control problem and the classical control problem, with infinite horizon. We show the policy improvement and the convergence to the optimal control. Since both the value function and the control are updated according to differential equations in a continuous manner, we also confirm the convergence rate of the proposed algorithm.
Zhenhua Wang
On Time-Inconsistency in Discrete-Time Mean Field Games
We investigate an infinite-horizon time-inconsistent mean-field game (MFG) in a discrete time setting. We first present a classic equilibrium for the MFG and its associated existence result. This classic equilibrium aligns with the conventional equilibrium concept studied in MFG literature when the context is time-consistent. Then we demonstrate that while this equilibrium produces an approximate optimal strategy when applied to the related N-agent games, it does so solely in a precommitment sense. To address this limitation, we propose a new consistent equilibrium concept in both the MFG and the N-agent game. We show that a consistent equilibrium in the MFG can indeed function as an approximate consistent equilibrium in the N-agent game. Additionally, we analyze the convergence of consistent equilibria for N-agent games toward a consistent MFG equilibrium as N tends to infinity. This talk is based on a joint work with Erhan Bayraktar.
Ruoyu Wu
Some large deviation principles for load-balancing queueing systems
We consider the large-scale load balancing queueing system under the Join-the-Shortest-Queue-d(n) policy. The system consists of one dispatcher and n servers. When a task arrives at the dispatcher, d(n) servers are chosen uniformly at random, and then the task is routed to the one with the shortest queue length. We establish large or moderate deviation principles for the system occupancy process, for different choices of d(n) such as d(n)=d and d(n)=n. Proofs rely on certain variational representations for exponential functionals of Poisson random measures, stochastic controls, and weak convergence arguments. This is based on joint works with Amarjit Budhiraja, Eric Friedlander, and Zhenhua Wang.
Hao Xing
Optimal Contract, Delegated Investment, and Information Acquisition
This paper examines a model of delegated investment within the framework of a noisy rational expectations equilibrium. Portfolio managers can acquire costly signals about asset payoffs but incur portfolio management costs. They receive compensation from delegated investors and make investment decisions on their behalf. The optimal contract includes a benchmark component that mitigates agency frictions arising from portfolio management costs. The precision of private signals chosen by portfolio managers is determined by equilibrium market conditions rather than the specifics of their individual contracts. When portfolio management costs decrease, both the performance and benchmark components of the optimal contract are reduced, portfolio managers acquire less precise private signals, but market price efficiency improves. This is a joint work with Yuyang Zhang.
Song Yao
Zero-sum Games of Optimal Stopping with Expectation Constraints
We study a zero-sum game of optimal stopping in which both players have some expectation constraints. If the payment functions are Lipschitz, one can readily show that the lower/upper value functions of such a game are continuous in current state and budget levels. For measurable payment functions, we show that the zero-sum game of optimal stopping with expectation constraints in an arbitrary probability setting is equivalent to the constrained game in weak formulation and thus the value function is independent of a specific probabilistic setup. Using a martingale-problem formulation and a Polish space of stopping times, we make an equivalent characterization of the probability classes in weak formulation, which implies the semi-analyticity of the lower/upper value functions. Then we adopt a measurable selection argument to establish dynamic programming principles for lower value and upper value in weak formulation respectively, in which the conditional expected costs act as additional states for constraint levels at the intermediate horizon. Moreover, we characterize the lower (resp. upper) value function as a viscosity supersolution (resp. viscosity subsolution) to the related fully non-linear Hamilton-Jacobi-Bellman equations, and study its comparison theorem in order to match the lower and upper value of the zero-sum game of optimal stopping with expectation constraints.
Xiang Yu
Continuous-time q-learning for mean-field control problems with common noise
This paper investigates the continuous-time entropy-regularized reinforcement learning (RL) for mean-field control problems with common noise. We study the continuous-time counterpart of the Q-function in the mean-field model, coined as q-function in the single agent's model. It is shown that the controlled common noise gives rise to a double integral term in the exploratory dynamic programming equation, rendering the policy improvement iteration intricate. The policy improvement at each iteration can be characterized by a first-order condition using the notion of partial linear derivative in policy. To devise some model-free RL algorithms, we introduce the integrated q-function (Iq-function) on distributions of both state and action, and an optimal policy can be identified as a two-layer fixed point to the soft argmax operator of the Iq-function. The martingale characterization of the value function and Iq-function is established by exhausting all test policies. This allows us to propose several algorithms including the Actor-Critic q-learning algorithm, in which the policy is updated in the Actor-step based on the policy improvement rule induced by the partial linear derivative of the Iq-function and the value function and Iq-function are updated simultaneously in the Critic-step based on the martingale orthogonality condition. In two examples, within and beyond LQ-control framework, we implement and compare our algorithms with satisfactory performance.
Xin Zhang
Exciting games and Monge-Ampère equations
In this talk, we consider a competition between d+1 players, and aim to identify the “most exciting game” of this kind. This is translated, mathematically, into a stochastic optimization problem over martingales that live on the d-dimensional subprobability simplex and terminate on the vertices of the simplex, with a cost function related to a scaling limit of Shannon entropies. We uncover a surprising connection between this problem and the seemingly unrelated field of Monge-Ampère equations, and identify the optimal martingale via a detailed analysis of boundary asymptotics of a Monge-Ampère equation.
Antonios Zitridis
Model Calibration via Optimal Transport and Schrödinger Bridges: Recent Advancements
In this talk, we explore model calibration problems, focusing on approaches utilizing optimal transport and Schrödinger bridges. We will review existing results in the literature and present recent advancements in the field. The proofs of the results rely on techniques commonly used in mean field game theory.