Plenary Speakers

Abstract

Many statistical learning problems involving sparsity and other structural constraints can be formulated as mixed-integer convex optimization problems involving indicator variables and constraints on these indicators. As motivation, the first part of this talk will focus on the problem of learning directed acyclic graphs (DAGs) from continuous observational data, also known as the causal discovery problem. Current state-of-the-art structure learning methods for this problem face significant limitations: (i) they lack optimality guarantees and often yield suboptimal solutions; (ii) they rely on the restrictive assumption of homoscedastic noise. To address these shortcomings, we propose a computationally efficient mixed-integer programming framework. Numerical experiments demonstrate that our method outperforms existing algorithms and is robust to noise heteroscedasticity.

In the second part of the talk, we address the challenge of weak continuous relaxations inherent in natural mixed-integer convex formulations for such structured learning problems, which hinder the performance of branch-and-bound-based solvers. We develop novel methods to strengthen these formulations by deriving a convex hull description of the associated mixed-integer set in an extended space. Notably, this approach reduces the convexification of these problems to the characterization of a polyhedral set in the extended formulation. Our new theoretical framework unifies several previously established results and provides a foundation for applying polyhedral methods to strengthen the formulations of such mixed-integer nonlinear sets.

Abstract

Many data-driven machine learning techniques are nowadays used in scientific discovery applications including symbolic regression. In symbolic regression, the functional form is not fixed a priori and one instead learns a function of bounded expression length that can be composed of an input list of operators and variables.

However, finding formulas that are consistent with known theory is a challenging task. We describe our recent software implementation, named AI Hilbert, that draws on techniques from polynomial optimization and algebraic geometry to obtain a scientific formula when both the desired formula and background theory are expressible via polynomial expressions. We also discuss recent work on improving polynomial optimization techniques that are motivated by discrete optimization methods.

Abstract

Combinatorial optimization augmented machine learning (COAML) is a novel and rapidly growing field that integrates methods from machine learning and operations research to tackle data-driven problems that involve both uncertainty and combinatorics. These problems arise frequently in industrial processes, where firms seek to leverage large and noisy data sets to better optimize their operations. COAML typically involves embedding combinatorial optimization layers into neural networks and training them with decision-aware learning techniques. This talk provides an overview of the field, covering its main applications, algorithms, and theoretical foundations. We also demonstrate the effectiveness of COAML on contextual and dynamic stochastic optimization problems, as evidenced by its winning performance on the 2022 EURO-NeurIPS challenge on dynamic vehicle routing.