Mehrdad Moharrami

Broad overview.

The traditional approach in designing engineering systems is to use physics-driven mathematical structural models with a few experimental observations as tests. In recent times this approach has been upended in many cases by data-driven methods that make limited structural assumptions owing to the availability of vast amounts of observational data. My research is two-fold, overlapping with both perspectives: (1) I aim to improve the reliability and performance of real-world systems using observational data, and (2) I aim to understand the economics of real-world networks and to develop computationally efficient algorithms based on their underlying structure. Specifically, my goal is (1) to devise data-driven algorithms for behavioral adaptation of an agent that is interacting with an environment and learning from it and (2) to predict the results of interactions of individuals with given behaviors in real-world networks modeled by random graphs. Since most real-world systems consist of multiple agents interacting with each other and the environment to learn and improve their algorithms, my long-term research agenda is to marry my two current research areas, (3) designing, optimizing, and analyzing decentralized data-driven algorithms in multi-agent environments.

(1) Developing reliable data-driven algorithms with minimal structural assumptions.

Using the data generated in a controlled environment, how might you design reliable algorithms to be implemented in real-world settings? Often, the environment in which the algorithm is deployed has different statistical properties than the training environment. This might be due to safety concerns or unmodeled changes in the behavior of real-world environments: a drone used for package deliveries, a cloud server with some underlying policy for processing arrivals. Despite these, we need to develop algorithms that are robust against these model uncertainties. A common approach in this setting is to model the environment as a Markov Decision Process (MDP), and then to optimize the system's performance with robustness considerations.

My research here lies in Reinforcement Learning (RL) which has been remarkably successful in a wide variety of applications, including video gaming, robotics, communications networks, pricing, transportation, and product management. RL algorithms are trained on data from an environment that is assumed to correspond to a Markov Decision Process. The trained algorithm is then deployed with the hope that it will perform well in environments similar to the ones it was trained in. My research goals in this area are (2.1) to design RL algorithms that are robust against model uncertainties, (2.2) to propose RL algorithms with finite time guarantees, and (2.3) to develop new techniques to study the finite-time performance of known RL algorithms.

(2) Understanding real-world networks using mathematical modeling.

Given some examples of real-world networks, how might you reason that observed phenomena are ``typical'' to such networks? Often, there is only limited information available about the network. This may arise due to a lack of data or difficulty analyzing a large data set. Despite this, we need to understand and forecast the behavior of a process on a typical example of the network. Running a process over the given network can be impossible or too costly and is prone to error because of a limited number of samples. Based on these concerns, a reasonable and fruitful procedure is to study graphs that are generated randomly based on the structural properties of real-world networks.

My research here uses random graph models to analyze behaviors and algorithms on large networks. Random graph models have many applications in different fields, including, but not limited to, network sciences, social sciences, life sciences, biological sciences, health studies, macroeconomics, etc. They have also been studied in a large body of research over many years, but despite this, many challenges and open questions remain. My research goals in this area are (1.1) to improve our understanding of the economics of real-world networks based on their structural properties, (1.2) to propose new graph models that capture different aspects of real-world networks, and (1.3) to develop new techniques in the study of random graphs.

(3) Decentralized data-driven algorithms in Multi-agent setting.

Most of the data-driven algorithms are deployed in settings that have many agents interacting with each other: robots in a warehouse need to cooperate to make deliveries, autonomous cars need to communicate with each other and the environment to avoid any collisions, and trading strategies should take into account the strategy of other agents in the system, etc. Despite many success stories of multi-agent data-driven algorithms in applications, theoretical guarantees are lacking.

Due to technical difficulties, the theoretical foundation of multi-agent reinforcement learning algorithms is heavily under-explored. Carrying out successful research in this area needs an interdisciplinary effort, combining expertise in different areas of RL, MDPs, game theory, mechanism design, and applied probability. Given my training and expertise in systems, mathematics and economics, I aim to devise analytic techniques applicable to studying multi-agent RL algorithms.