Spring 2022
Statistics, Optimization and Machine Learning Seminar
Quick info
We meet Tuesdays 3:30-4:20pm MT in ECCR 257 (Newton Lab). To sign up for the mailing list and receive announcements about talks (you do not need to be enrolled in the seminar), visit our google groups website and add yourself to the group
The seminar is organized by Professors Rafael Frongillo (CS Dept) and Stephen Becker (Applied Math Dept)
Spring 2022 Update: the seminar is happening!
The seminar was canceled Fall 2020 and Spring 2021, but now it is back for Fall 2021 and Spring 2022. We anticipate running in hybrid mode for most talks: there will be an audience in person, but anyone wishing to join remotely can do so. Please join the google groups list, as that is where we will distribute remote access information (we'll use zoom).
The seminar is always open to the public!
You do not have to be enrolled in the class ... but if you are enrolled in the class, for a passing grade, you must:
attend all the talks (missing an occasional talk is permissible if you have a valid reason, so email the instructors), and
give a ~20 min presentation at some point in the semester, either on your own research, or presenting a paper
Support
For about the past 5 years, the seminar has been grateful for various support from the CU engineering college, the CU computer science and CU applied math departments, and from donations from Jake Knigge and CoBank.
List of Talks
April 12, 2022. Students talks
Seung Kim, Title: "Online Stochastic Gradient Methods Under Sub-Weibull Noise and the Polyak-Łojasiewicz Condition"
Nitin Kumar, presenting the paper "Are Convolutional Neural Networks or Transformers more like human vision? Shikhar Tuli, Ishita D. et. al."[1]
Everly Tseng, Title: VizWiz-FewShot: Locating Objects in Images Taken by People with Visual Impairments
March 29, 2022. Yi-Kai Liu (NIST Gaithersburg)
Title: Compressed Sensing Measurement of Long-Range Correlated Noise
Remote only
March 15, 2022. April Tran (CU Applied Maths)
Title: “Data-driven discovery of partial differential equations.”, Rudy, Samuel H. et al. [1]
March 8, 2022. Nicholas Dwork (Clinical Informatics, University of Colorado Anschutz)
Title: Scanning a Placenta and Fetus Quickly with MRI
March 1, 2022. Zachariah Malik
Title: Generalizing FISIK to accurately simulate VEGFR2
Feb 22, 2022. Jacob Spainhour
Title: Optimizing Ultrasound Encoding
Feb 15, 2022. Liam Madden
Title: Approximate quantum compiling
Feb 8, 2022. Maziar Raissi (CU Boulder, Applied Math dept)
Feb 1, 2022. Steven Flammia (AWS Center for Quantum Computing / Caltech)
Title: Averaged circuit eigenvalue sampling [1]
Remote only
Jan 25, 2022. Alex Nowak-Vila, Research Scientist at Owkin [Recording]
Jan 18, 2022. Lecture 1 on youtube by Artur Ekert on Intro to Quantum Information Science
Jan 11, 2022. No talk, just organizational meeting for students who are enrolled
Abstracts
April 12: Seung Kim
Title: "Online Stochastic Gradient Methods Under Sub-Weibull Noise and the Polyak-Łojasiewicz Condition"
Abstract: This paper focuses on the online gradient and proximal-gradient methods with stochastic gradient errors. In particular, we examine the performance of the online gradient descent method when the cost satisfies the Polyak-Łojasiewicz (PL) inequality. We provide bounds in expectation and in high probability (that hold iteration-wise), with the latter derived by leveraging a sub-Weibull model for the errors affecting the gradient. The convergence results show that the instantaneous regret converges linearly up to an error that depends on the variability of the problem and the statistics of the sub-Weibull gradient error. Similar convergence results are then provided for the online proximal-gradient method, under the assumption that the composite cost satisfies the proximal-PL condition. In the case of static costs, we provide new bounds for the regret incurred by these methods when the gradient errors are modeled as sub-Weibull random variables. Illustrative simulations are provided to corroborate the technical findings.
April 12: Nitin Kumar
Presenting the paper "Are Convolutional Neural Networks or Transformers more like human vision? Shikhar Tuli, Ishita D. et. al."[1]
April 12: Everly Tseng
Title: VizWiz-FewShot: Locating Objects in Images Taken by People with Visual Impairments
Abstract: In this paper that we recently submitted, we introduce a few-shot localization dataset originating from photographers who authentically were trying to learn about the visual content in the images they took. It includes over 8,000 segmentations of 100 categories in over 4,000 images that were taken by people with visual impairments. Compared to existing few-shot object detection and instance segmentation datasets, our dataset is the first to locate holes in objects (e.g., found in 12.4% of our segmentations), it shows objects that occupy a much larger range of sizes relative to the images, and text is over five times more common in our objects (e.g., found in 24.7% of our segmentations). Analysis of two modern few-shot localization algorithms demonstrates that they generalize poorly to our new dataset. The algorithms commonly struggle to locate objects with holes, very small and very large objects, and objects lacking text. To encourage a larger community to work on these unsolved challenges, we publicly share our annotated few-shot dataset.
March 29: Yi-Kai Liu (NIST Gaithersburg)
Title: Compressed Sensing Measurement of Long-Range Correlated Noise
Abstract: Long-range correlated errors can severely impact the performance of NISQ (noisy intermediate-scale quantum) devices, and fault-tolerant quantum computation. Characterizing these errors is important for improving the performance of these devices, via calibration and error correction, and to ensure correct interpretation of the results. We propose a compressed sensing method for detecting two-qubit correlated dephasing errors, assuming only that the correlations are sparse (i.e., at most s pairs of qubits have correlated errors, where s << n(n-1)/2, and n is the total number of qubits). In particular, our method can detect long-range correlations between any two qubits in the system (i.e., the correlations are not restricted to be geometrically local). Our method is highly scalable: it requires as few as m = O(s log n) measurement settings, and efficient classical post-processing based on convex optimization. In addition, when m = O(s log^4(n)), our method is highly robust to noise, and has sample complexity O(max(n,s)^2 log^4(n)), which can be compared to conventional methods that have sample complexity O(n^3). Thus, our method is advantageous when the correlations are sufficiently sparse, that is, when s < O(n^(3/2) / log^2(n)). Our method also performs well in numerical simulations on small system sizes, and has some resistance to state-preparation-and-measurement (SPAM) errors. The key ingredient in our method is a new type of compressed sensing measurement, which works by preparing entangled Greenberger-Horne-Zeilinger states (GHZ states) on random subsets of qubits, and measuring their decay rates with high precision. (Joint work with Alireza Seif and Mohammad Hafezi.)
March 15, 2022, April Tran (CU Applied Maths)
Title: “Data-driven discovery of partial differential equations.”, Rudy, Samuel H. et al. [1]
Abstract: "We propose a sparse regression method capable of discovering the governing partial differential equation(s) of a given system by time-series measurements in the spatial domain. The regression framework relies on sparsity-promoting techniques to select the nonlinear and partial derivative terms of the governing equations that most accurately represent the data, bypassing a combinatorially large search through all possible candidate models. The method balances model complexity and regression accuracy by selecting a parsimonious model via Pareto analysis. Time series measurements can be made in an Eulerian framework where the sensors are fixed spatially, or in a Lagrangian framework where the sensors move with the dynamics. The method is computationally efficient, robust, and is demonstrated to work on a variety of canonical problems spanning a number of scientific domains. Moreover, the method is capable of disambiguating between potentially non-unique dynamical terms by using multiple time series taken with different initial data. Thus for a traveling wave, the method can distinguish between a linear wave equation or the Korteweg-deVries equation, for instance. The method provides a promising new technique for discovering governing equations and physical laws in parametrized spatio-temporal systems where first-principles derivations are intractable. "
March 8, 2022, Nicholas Dwork, Assistant professor at University of Colorado Anschutz
Title: Scanning a Placenta and Fetus Quickly with MRI
Abstract: MRI is an inherently slow imaging modality; thus, collecting enough data to satisfy the Nyquist-Shannon sampling theorem is time-consuming, creating blurring and artifacts when imaging a moving fetus. Existing technologies, such as parallel imaging and compressed sensing, make scanning faster and help overcome these challenges. We will review recent advancements in these technologies that have improved the quality of a given amount of data. These advancements present opportunities to reduce the reconstruction time further with a proper choice of algorithm to solve the formulated optimization problems. Moreover, incorporating additional known physics into the optimization problem shrinks the feasible set and further reduces the amount of data needed for an accurate reconstruction. After reviewing this material, we will discuss possible future directions of this work.
March 1, 2022. Zachariah Malik
Title: Generalizing FISIK to accurately simulate VEGFR2
An understanding of receptor interaction kinetics is crucial to elucidate several complex biological processes. Often, these are obtained through live-cell single-molecule imaging. While this method does provide tremendous data on receptor interaction kinetics, it fails to provide a complete picture. The inherent diffraction limit in a microscope restricts it to a resolution of approximately 500nm and 200nm along the axial and lateral directions, respectively. Furthermore, imaged molecules need to be tracked over a period of time. Hence, only a small fraction of molecules can be labeled at a time. Live-cell single-molecule imaging requires sparse labeling, in which only a small subset of interaction events can be captured. For example, if 10% of all molecules are labeled, then only about 1% of merge events would be fully captured4. This presents a problem if one wants to accurately determine the association rates of a set of molecules. Sparse labeling negatively affects a whole host of interaction parameters, such as oligomeric state, dissociation rates, etc. Thus, it is necessary to develop alternative methods to capture interaction kinetics. Mathematical modeling is one such alternative method. Our group has already developed the Framework for the Inference of In-Situ Interaction Kinetics (FISIK) with this express purpose4. This work builds upon FISIK with the eventual goal of accurately simulating vascular endothelial growth factor receptor-2 (VEGFR-2). First, several software updates are implemented to generalize the existing model. Then simulations are compared with experimental data as we adjust input parameters.
Feb 22, 2022, Jacob Spainhour, PhD candidate in Applied Math at CU Boulder
Title: Optimizing Ultrasound Encoding
Abstract: In this presentation, I will introduce the model of transmit encoding for ultrasound imaging, and describe ways in which machine learning can be used in a robust imaging framework that produces images with higher resolution and greater contrast. In the transmit encoding model, the divergent sound waves produced by an ultrasound device are timed and weighted in a way that allows for reconstruction of individual element responses, which are in turn refocused into a high-resolution image at all focal depths, independent of the original beam geometry. However, different choices for this beam geometry can produce images of dramatically different quality. To account for this, we propose a machine learning approach that searches the space of possible encodings for one that performs optimally on a set of simulated ultrasound data using PyTorch. Practical difficulties of this approach include expensive loss functions, sensitivity to initial conditions, and generalization to multiple classes of image targets. I will discuss these issues alongside strategies we have developed to address them.
Feb 15, 2022, Liam Madden
Title: Approximate quantum compiling
Abstract: I will introduce approximate quantum compiling by building up to a definition of quantum logic gates as unitary matrices. Reversible logic gates can be seen as permutation matrices of size 2^n by 2^n for n bits and this naturally extends to quantum logic gates as unitary matrices of size 2^n by 2^n for n qubits. CNOT and CCNOT are important gates in both reversible and quantum computing. By explaining the algebra of circuit diagrams and introducing quantum rotation gates, we have everything we need to construct quantum logic gates.
Feb 8, 2022. Maziar Raissi (CU Boulder, Applied Math dept)
Title: Data-Efficient Deep Learning using Physics-Informed Neural Networks
Abstract: A grand challenge with great opportunities is to develop a coherent framework that enables blending conservation laws, physical principles, and/or phenomenological behaviors expressed by differential equations with the vast data sets available in many fields of engineering, science, and technology. At the intersection of probabilistic machine learning, deep learning, and scientific computations, this work is pursuing the overall vision to establish promising new directions for harnessing the long-standing developments of classical methods in applied mathematics and mathematical physics to design learning machines with the ability to operate in complex domains without requiring large quantities of data. To materialize this vision, this work is exploring two complementary directions: (1) designing data-efficient learning machines capable of leveraging the underlying laws of physics, expressed by time-dependent and non-linear differential equations, to extract patterns from high-dimensional data generated from experiments, and (2) designing novel numerical algorithms that can seamlessly blend equations and noisy multi-fidelity data, infer latent quantities of interest (e.g., the solution to a differential equation), and naturally quantify uncertainty in computations.
Feb 1, 2022. Steven Flammia (AWS Center for Quantum Computing / Caltech)
Title: Averaged circuit eigenvalue sampling [1]
Abstract: We introduce ACES, a method for scalable noise metrology of quantum circuits that stands for Averaged Circuit Eigenvalue Sampling. It simultaneously estimates the individual error rates of all the gates in collections of quantum circuits, and can even account for space and time correlations between these gates. ACES strictly generalizes randomized benchmarking (RB), interleaved RB, simultaneous RB, and several other related techniques. However, ACES provides much more information and provably works under strictly weaker assumptions than these techniques. Finally, ACES is extremely scalable: we demonstrate with numerical simulations that it simultaneously and precisely estimates all the Pauli error rates on every gate and measurement in a 100 qubit quantum device using fewer than 20 relatively shallow Clifford circuits and an experimentally feasible number of samples. By learning the detailed gate errors for large quantum devices, ACES opens new possibilities for error mitigation, bespoke quantum error-correcting codes and decoders, customized compilers, and more.
Jan 25, 2022. Alex Nowak-Vila (Research Scientist at Owkin)
Title: Consistency of Max-Margin Methods for Structured Prediction [1, 2]
Abstract: The foundational concept of Max-Margin in machine learning is ill-posed for output spaces with more than two labels such as in structured prediction. In the first part of the talk, we show that the Max-Margin loss can only be consistent with the classification task under highly restrictive assumptions on the discrete loss measuring the error between outputs and we generalize partial consistency results existing for the classification error. In the second part, we design two Max-Margin loss extensions: the Restricted-Max-Margin loss for which we prove consistency under mild assumptions, and the Max-Min-Margin loss for which consistency always holds. Finally, we provide an efficient algorithm for the latter working in structured prediction settings based on projections to the marginal polytope and provide generalization guarantees with respect to the original task of interest.
Biography: Alex Nowak is a Research Scientist at Owkin, working at the intersection of machine learning and healthcare. He holds a PhD in Machine Learning advised by Francis Bach and Alessandro Rudi from INRIA and Ecole Normale Supérieure at the SIERRA project team in Paris, France.
Jan 18, 2022. Lecture 1 on youtube by Artur Ekert on Intro to Quantum Information Science
Jan 11, 2022. No talk, just organizational meeting for students who are enrolled