Research Interests

Mathematical Optimization (convex, nonconvex, nonlinear programming etc.)
Proximal/Operator Splitting Methods (projective splitting, ADMM, forward-backward splitting, proximal gradient method, Tseng's method, the extragradient method, inertial methods, FISTA, heavy-ball method, first-order methods, the subgradient method.)
Convergence Analyses (convergence and rates of iterative algorithms for optimization, monotone inclusions, variational inequalities, and fixed-point problems. Growth conditions, error bounds, and the KL inequality.)
Parallel Algorithms (asynchronous and distributed implementations.)
Machine Learning and AI (applications of the above methods in these areas and more.)
Formal Verification Methods (Can we formally verify our algorithms using modern tools such as automated theorem provers? Can we write proof-carrying code? Can we train AI to prove new theorems in these theorem provers?)

More Detailed Summary of Research Interests

I am interested in designing and analyzing algorithms that solve optimization problems and games. The sort of optimization problems I study arise mainly in image/signal processing and machine learning/AI. More specifically I am interested in nonsmooth problems, convex and nonconvex problems, min-max problems, and noncooperative games. The sorts of methods I study include proximal/operator splitting, inertial/momentum methods, and accelerated methods. My day-to-day research revolves around convergence analyses for such algorithms. Typically these methods are iterative, meaning they produce a sequence of approximations to the solution of the problem. Mathematically one would like to establish that this sequence converges to a solution, i.e. that its limit is a solution. One would also like to establish how fast this happens, i.e. what is the convergence rates of the algorithm. Also fundamental is the overall computational complexity of an algorithm for finding a solution to within a specified accuracy.

Projective Splitting with Forward Steps

My project at Rutgers was on asynchronous parallel projective splitting methods. This is a new and highly flexible class of operator splitting algorithms with many advantages over classical techniques. Previously these methods had made use of proximal steps w.r.t. the operators in the problem. In this paper I figured out how to use forward steps w.r.t. any operators which are Lipschitz continuous. This leads to new variants of projective splitting with lots of interesting features. In general, the stepsize must be bounded by the inverse of the Lipschitz constant. But it is easy to do a backtracking linesearch technique when this constant is unknown. In fact, in later work we showed that you only need continuity, not Lipschitz continuity (However we do require that the operator has full domain). Furthermore, for affine operators, Prof. Eckstein and I developed an adaptive stepsize which obviates the need to do backtracking.

In another paper I worked out the convergence rates of projective splitting. It established an O(1/k) function-value rate for convex minimization problems. It also established linear convergence for strongly monotone and cocoercive inclusion problems (i.e. strongly convex and sufficiently smooth minimization problems).

Stochastic Projective Splitting

At BNL, I have continued to work on projective splitting. In this paper, I worked out the first stochastic projective splitting method, based on a stochastic version of my previous two-forward-step work. This is the first stochastic splitting method that can solve convex-concave min-max problems involving multiple nonsmooth regularizers and constraints. Previous methods could only handle a single regularizer/constraint. What is a "splitting" method? A method that can handle each individual constraint by its projection and each regularizer by its proximal operator. In applications where such projection and proximal operators are known in closed form, splitting methods can be effective.

Subgradient Methods under Holderian Growth

Towards the end of my PhD, I developed subgradient methods for minimizing functions satisfying Holder growth conditions. These conditions are extremely useful for deriving convergence rates and are present in many interesting optimization problems. One interesting application is robust phase retrieval, which despite being nonconvex, is weakly convex (a.k.a. prox-regular) and satisfies the sharp Holder Growth condition. Therefore the subgradient method I developed can be applied.

Local Convergence Results for Sparse Optimization

My early work in my PhD focused on sparse optimization which is applied mainly in compressed sensing-like problems. I managed to derive some results for an inertial method, proving that the method identifies the correct support set in a finite number of iterations and obtains linear convergence thereafter. While this was already known for non-inertial variants, my contribution extended this to inertial variants.

Formal Verification

I developed the (afaik) first formalization of the basic convergence proofs of gradient descent. These were done in the Lean Theorem Prover in early 2022. See link.

Early Work

Early in my PhD I did some work on statistical hypothesis testing; determining the asymptotic behavior of the error probabilities of certain statistical tests. I also worked out the behavior of the Kullback-Leibler divergence under the central limit theorem; specifically what happens to the divergence of a sum of i.i.d. random variables. See my publications.