Machine Learning Seminar Series

Engineered proteins have emerged as novel diagnostics, therapeutics, and catalysts. Often, poor protein developability — quantified by expression, solubility, and stability — hinders commercialization. The ability to predict protein developability from amino acid sequence would reduce the experimental burden when selecting candidates. Recent advances in screening technologies enabled a high-throughput (HT) developability dataset for 105 of 1020 possible variants of protein scaffold Gp2. In this work, we evaluate the ability of neural networks to learn a developability representation from the HT dataset and transfer the knowledge to predict recombinant expression beyond the observed sequences. Our model predicts expression levels 42% closer to the experimental variance compared to a non-embedded control. We then seek to exploit this knowledge to design new sequences with high developability. To overcome the intractability of a brute force search of 1020 possible Gp2 variants, we descend through the protein fitness landscape by Nested Sampling, a Monte Carlo scheme for Bayesian parameter estimation and model selection, which is particularly suited for the analysis of multimodal distributions. In addition to identifying high-developability libraries, we obtain unprecedented insight into the structure of the protein fitness landscape through a “topographical” analysis and statistical mechanical interpretation of the results.

Inverse optimization refers to the inference of unknown optimization models given decision data that are assumed to be optimal or near-optimal solutions to the unknown optimization problem. In this work, we consider data-driven inverse linear optimization, where the underlying decision-making process can be modeled as a linear program whose cost vector is unknown. We propose a two-phase algorithm to determine the best estimate of the cost vector given noisy observations. Moreover, we propose an efficient decomposition algorithm to solve large instances of the problem. The algorithm extends naturally to an online learning environment where it can be used to provide quick updates of the cost estimate as new data becomes available over time. For the online setting, we further develop an effective adaptive sampling strategy that guides the selection of the next samples. The efficacy of the proposed methods is demonstrated in computational experiments involving two applications, customer preference learning and cost estimation for production planning. The results show significant reductions in computation and sampling efforts.

Center for Excellence in Sensing Technologies and Analytics (CESTA) at the University of Minnesota brings together researchers from diverse domain fields that use sensing in their work. While a number of CESTA members work on development of novel sensing technologies, there is also an increasing demand for using modern data analytic techniques to process and analyze diverse and large volumes of data produced by sensor arrays. I will survey the machine/deep learning needs of CESTA researches across multiple domain fields, highlighting the research projects that could benefit from machine/deep learning expertise.

Large, condensed phase, and extended systems impose a challenge for theoretical studies due to the compromise between accuracy and computational cost in their calculations. Machine learning methods are an approach to solve this trade-off by leveraging large data sets to train on highly accurate calculations using small molecules and then apply them to larger systems. In this study, we are developing a method to train a neural network potential with high-level wavefunction theory on targeted systems of interest that are able to describe bond breaking. We combine density functional theory calculations and higher level ab initio wavefunction calculations, such as CASPT2, to train our neural network potentials. We first train our neural network at the DFT level of theory and using an adaptive active learning training scheme, we retrain the neural network potential to a CASPT2 level of accuracy. We demonstrate the process as well as report current progress and performance of this neural network potential for molecular dynamic simulations.

In the recent literature on estimating heterogeneous treatment effects, each proposed method makes its own set of restrictive assumptions about the intervention's effects and which subpopulations to explicitly estimate. Moreover, the majority of the literature provides no mechanism to identify which subpopulations are the most affected--beyond manual inspection--and provides little guarantee on the correctness of the identified subpopulations. Therefore, we propose Treatment Effect Subset Scan (TESS), a new method for discovering which subpopulation in a randomized experiment is most significantly affected by a treatment. We frame this challenge as a pattern detection problem where we efficiently maximize a nonparametric scan statistic over subpopulations. Furthermore, we identify the subpopulation which experiences the largest distributional change as a result of the intervention, while making minimal assumptions about the intervention's effects or the underlying data generating process. In addition to the algorithm, we demonstrate that the asymptotic Type I and II error can be controlled, and provide sufficient conditions for detection consistency--i.e., exact identification of the affected subpopulation. Finally, we validate the efficacy of the method by discovering heterogeneous treatment effects in simulations and in real-world data from a well-known program evaluation study.

This work is motivated by the possibility of a small number of automated vehicles (AVs) or partially automated vehicles that may soon be present on our roadways, and the impacts they will have on traffic flow. This automation may take the form of fully autonomous vehicles without human intervention (SAE Level 5) or, as is already the case in many modern vehicles, may take the form of driver assist features such as adaptive cruise control (ACC) or other SAE Level 1 features. Regardless of the extent of automation, the introduction of such vehicles has the potential to substantially alter emergent properties of the flow while also providing new opportunities for traffic state estimation. In this talk, I present some recent experimental and modeling work conducted to understand how AVs may be able to influence traffic flow.

Classification methods that leverage the strengths of data from multiple sources (multi-view data) simultaneously have enormous potential to yield more powerful findings than two step methods: association followed by classification. We propose two methods, sparse integrative discriminant analysis (SIDA) and SIDA with incorporation of network information (SIDANet), for joint association and classification studies. The methods consider the overall association between multi-view data, and the separation within each view in choosing discriminant vectors that are associated and optimally separate subjects into different classes. SIDANet is among the first methods to incorporate prior structural information in joint association and classification studies. It uses the normalized Laplacian of a graph to smooth coefficients of predictor variables, thus encouraging selection of predictors that are connected. We demonstrate the effectiveness of our methods on a set of synthetic and real datasets. Our findings underscore the benefit of joint association and classification methods if the goal is to correlate multi-view data and to perform classification.

Graph-based learning is a field within machine learning that uses similarities between datapoints to create efficient representations of high-dimensional data for tasks like semi-supervised classification, clustering and dimension reduction. For semi-supervised learning, the widely used Laplacian regularizer has recently been shown to be ill-posed at very low label rates, leading to poor classification similar to random guessing. In this talk, we will give a careful analysis of the ill-posedness of Laplacian regularization via random walks on graphs, and this will lead to a new algorithm for semi-supervised learning that we call Poisson learning. Poisson learning replaces the assignment of label values at training points with the placement of sources and sinks, and solves the resulting Poisson equation on the graph. The outcomes are provably more stable and informative than those of Laplacian learning. Poisson learning is efficient and simple to implement, and we will present numerical results on MNIST, FashionMNIST and Cifar-10 showing that the method is superior to other recent approaches to semi-supervised learning at low label rates.

This talk is joint work with Brendan Cook (UMN), Dejan Slepcev (CMU) and Matthew Thorpe (University Manchester).

Due to accessible big data collections from consumers, products, and stores, advanced sales forecasting capabilities have drawn great attention from many companies especially in the retail business because of its importance in decision making. Improvement of the forecasting accuracy, even by a small percentage, may have a substantial impact on companies' production and financial planning, marketing strategies, inventory controls, supply chain management, and eventually stock prices. Specifically, our research goal is to forecast the sales of each product in each store in the near future. Motivated by tensor factorization methodologies for personalized context-aware recommender systems, we propose \iv{a novel approach} called the Advanced Temporal Latent-factor Approach to Sales forecasting (ATLAS), which achieves accurate and individualized prediction for sales by building a single tensor-factorization model across multiple stores and products. Our contribution is a combination of: tensor framework (to leverage information across stores and products), a new regularization function (to incorporate demand dynamics), and extrapolation of tensor into future time periods using state-of-the-art statistical (seasonal auto-regressive integrated moving-average models) and machine-learning (recurrent neural networks) models. The advantages of ATLAS are demonstrated on \iv{eight datasets} collected by the Information Resource, Inc., where a total of 165 million weekly sales transactions from more than 1,500 grocery stores over 15,560 products are analyzed.

Medical image analysis for oncology applications is plagued by problems of small labeled datasets and the difficulty in analyzing images that often have very low soft-tissue contrast to differentiate foreground and background structures. Deep learning techniques such as deep domain adaptation developed in computer vision, have limited success when applied to medical images. This is due to the wide variations in the images even in the same modalities and even larger variations of the different tissues appearing on multiple imaging modalities. In this talk, I will present some solutions developed by our group for solving these two problems. In brief, I will present an approach that uses joint distribution matching to learn from entirely different imaging modalities to segment on a target modality without any expert labels. Next, I will show how cross-domain adaptation can be combined with distillation learning to train models to segment centrally located lung tumors on CT and more challenging cone-beam CT images. I will also show how these methods are being translated from lab into clinical practice for image-guided radiation treatments in our hospital.