Session Chair: Nikos Sidiropoulos, University of Virginia
08:30–09:20
Evrim Acar, Simula
Abstract: Datasets from diverse sources are often collected to gain insights about complex systems such as the human brain and human metabolome, or to detect early risk factors for various diseases. For instance, joint analysis of omics data (e.g., metabolomics, microbiome, genomics) holds the promise to improve our understanding of the human metabolism and facilitate precision health. Such multimodal datasets are often noisy, incomplete, and multiway (i.e., with more than two axes of variation such as subjects, metabolites, and time). Some datasets change in time (e.g., longitudinal measurements) while some are static.
How can we jointly analyze such heterogeneous data in the form of matrices/higher-order tensors and reveal interpretable patterns? How can we effectively incorporate prior information, e.g., the fundamental principles encapsulated in mechanistic models, in real data analysis to reveal insights from such complex data sets?
Tensor factorizations have been successfully used to reveal the underlying patterns in higher-order tensors, and extended to joint analysis of multimodal data through coupled matrix/tensor factorizations (CMTF). Coupled matrix/tensor factorizations provide an explainable framework revealing interpretable patterns. Recent developments in coupled factorizations have enabled incorporation of constraints, different loss functions to account for different data types, and different coupling relations to handle datasets with different resolutions or partial couplings. In this talk, we discuss CMTF models and algorithms for multimodal data mining as well as using CMTF models to guide real data analysis with mechanistic models. Through various applications, we discuss the advantages and limitations of available methods.
Coffee Break 09:30–09:50
09:50–10:40
Eric F. Lock, University of Minnesota
Abstract: First we propose an empirical variational Bayesian approach to factorization of linked matrices, EV-BIDIFAC, that has several advantages over existing techniques. It has the flexibility to accommodate shared signal over any number of row or column sets (i.e., bidimensional integration), an intuitive model-based objective function that yields appropriate shrinkage for the inferred signals, and a relatively efficient estimation algorithm with no tuning parameters. It also accommodates imputation of missing data, including "blockwise" imputation in which an entire row or column is missing. A general result establishes conditions for the uniqueness of the underlying decomposition for a broad family of methods that includes the proposed approach. Extensive simulations show that the method performs very well under different scenarios with respect to recovering underlying low-rank signal, accurately decomposing shared and specific signals, and accurately imputing missing data. Second, we propose a Multiple Linked Tensors Factorization (MULTIFAC) method extending the CANDECOMP/PARAFAC (CP) decomposition to simultaneously factorize multiple linked multi-way arrays (i.e., tensors). We first introduce a version of the CP factorization with L2 penalties on the latent factors, leading to rank sparsity. When extended to multiple linked tensors, the method automatically reveals latent components that are shared across data sources or individual to each data source. We also extend the decomposition algorithm to its expectation–maximization (EM) version to handle incomplete data with imputation. Extensive simulation studies are conducted to demonstrate MULTIFAC’s ability to (i) approximate underlying signal, (ii) identify shared and unshared structures, and (iii) impute missing data. Both EV-BIDIFAC and MULTIFAC are illustrated with examples in biomolecular data integration.
10:50–11:40
David Hong, University of Delaware
Abstract: Tensor decompositions generalize matrix decompositions to multiway data and form a foundational family of workhorse techniques for unsupervised learning and pattern discovery. Conventionally, tensor decompositions seek low-rank approximations that best fit data in the least-squares sense, i.e., with respect to the least squares (or Euclidean) loss. Generalized tensor decompositions generalize one step further by allowing arbitrary user-defined loss functions to be used instead. Allowing these general losses enables users to choose notions of fit that best suit the data, e.g., likelihood-based losses coming from statistical models or robust losses to handle potential outliers. This talk will review generalized tensor decompositions and discuss ongoing work on further generalizing these decompositions to incorporate constraints (such as symmetry) and to make connections with coupled matrix and tensor factorizations.
Lunch 12:00–13:00
Session Chair: Lieven de Lathauwer, KU Leuven
13:15–14:05
Jean Kossaifi, NVIDIA
Abstract: Tensor methods have a long and rich history across diverse scientific fields and provide a rigorous framework for manipulating multi-dimensional arrays. Yet, despite enabling significant parameter savings, computational efficiency, and enhanced performance in a variety of applications, their adoption in deep learning remains limited. However, with latest developments in deep learning model architectures, as well as the emergence of Neural Operators for scientific innovation (“AI4Science”), presenting new opportunities for wider adoption of tensor based methods. This presentation will delve into the current landscape of tensor methods in deep learning. I will examine the barriers to their broader adoption and discuss how these challenges can be addressed to be able to leverage the potential of tensor methods in modern machine learning.
14:15–15:05
Jeremy Cohen, CNRS Creatis
Abstract: Nonnegative Matrix factorization (NMF) is a method of choice to extract interpretable patterns from matrix data. This method is also called Multivariate Curve Resolution in chemometrics. However it is well known that NMF itself, while "more interpretable" than other low-rank matrix approximations such as PCA, often suffers from a lack of uniqueness without further assumptions. Model-based approaches rely on well chosen regularizations such as minimum volume or total-variation regularizations to enhance the interpretability of the output of NMF. Recently, data-driven approaches have emerged that leverage both advances in deep learning methods and the existence of (potentially scarce) training data in source separation applications. In this presentation, we explore two different such data-driven strategies. The first one adapts the widely used Multiplicative Updates (MU) algorithms for computing NMF to introduce trainable parameters. We discuss the effectiveness of such a trained/unrolled MU algorithm on the task of spectral unmixing in various scenarios. The second strategy makes use of pretrained deep denoisers to serve as proximal (projection) operators in a classical linear inverse problem where ultimately we aim at performing both image reconstruction and source separation simultaneously.
Coffee Break 15:15–15:35
15:35–16:25
Kim Batselier, Delft University of Technology
Abstract: The ability to express a learning task in terms of a primal and a dual optimization problem lies at the core of a plethora of machine learning methods. For example, Support Vector Machine, Ridge Regression, Lasso Regression, Principal Component Analysis (PCA), and more recently the Singular Value Decomposition (SVD) have all been defined either in terms of primal weights or in terms of dual Lagrange multipliers. The primal formulation is computationally advantageous in the case of large sample size while the dual is preferred for high-dimensional data. Crucially, said learning problems can be made nonlinear through the introduction of a feature map in the primal problem, which corresponds to applying the kernel trick in the dual. In this talk I will introduce a primal-dual formulation of the Multilinear Singular Value Decomposition (MLSVD), which recovers as special cases both PCA and the SVD. Besides enabling computational gains through the derived primal formulation, we propose a nonlinear extension of the MLSVD using feature maps, which results in a dual problem where a kernel tensor arises.
16:35–17:25
Kazu Ghalamkari, Technical University of Denmark
Abstract: We can extract features or patterns from tensor-formatted data by approximating a given tensor with a low-rank tensor, represented as a linear combination of a small number of bases components. However, traditional low-rank approximation methods have well-known challenges, such as initial value dependency, ill-posedness, and the non-trivial rank tuning. To address these issues, this talk introduces an alternative approach, called many-body approximation, which reduces higher-order interactions among tensor modes. Specifically, we describe the interactions among tensor modes using an energy function, and approximate the tensor by restricting the model to dominant low-order interactions. Our information geometrical analysis guarantees that our approach always finds the globally optimal solution that minimizes the KL divergence from the given tensor, regardless of initial values. The proposed method also facilitates intuitive model design by representing interactions among modes using a factor graph. This talk further shows how to convert the factor graph representation into a traditional tensor network, and illustrates the mathematical relationship between conventional low-rank approximations and the proposed approach. We demonstrate the effectiveness of the proposed method through numerical experiments on representation learning and missing value imputation.
Dinner 18:30 - 20:30