Session Chair: Jeremy Cohen, CNRS Creatis
08:30–09:20
Martin Haardt, Ilmenau University of Technology
Abstract: Obtaining a reliable estimate of the joint probability mass function (PMF) of a set of random variables from observed data is a significant objective in statistical signal processing and machine learning. Modelling the joint PMF as a tensor that admits a low-rank canonical polyadic decomposition has enabled the development of efficient PMF estimation algorithms. However, these algorithms require the rank of the tensor to be specified beforehand. In real-world applications, the true rank is unknown. Therefore, an appropriate rank is usually selected from a candidate set either by observing validation errors or by computing various likelihood-based information criteria, a procedure which is computationally expensive for large data sets.
In this talk, we present a new Bayesian framework for estimating the joint PMF and automatically inferring its rank from the observed data. We specify a Bayesian PMF estimation model and employ appropriate prior distributions for the model parameters, allowing for tuning-free rank inference via a single training run. Then, we derive a deterministic solution based on variational inference (VB-PMF) to approximate the posterior distributions of various model parameters. We validate the automatic rank detection performance of VB-PMF via synthetic-data experiments. Moreover, we explore the applicability of VB-PMF to real-data problems such as recommender systems and flow cytometry data analysis. Specifically, we apply VB-PMF to movie recommendation data for the prediction of missing ratings and to the quantification of pollen concentrations from flow cytometry measurements of various pollen species. For the latter application example, we exploit VB-PMF to first estimate the joint PMF of the pure pollen substances and then apply this knowledge along with the estimated PMF of the mixture to quantify the concentration of each pollen species in the mixture. The results illustrate the advantages of our proposed approach in terms of estimation accuracy, automatic rank detection, and computational efficiency.
This is joint work with Joseph K. Chege, Alla Manina, and Arie Yeredor.
Coffee Break 09:30–09:50
09:50–10:40
Charlotte Vermeylen, KU Leuven
Abstract: A novel optimization framework is proposed for solving the low-rank tensor approximation problem using the canonical polyadic decomposition (CPD). This can be a difficult optimization problem for certain tensors, e.g., due to degeneracy, i.e., a tensor that can be approximated arbitrarily closely by an ill-conditioned tensor of lower rank. This is one of the phenomena that are encountered in regions of slow convergence, informally known as swamps. Numerical experiments with state-of-the-art optimization algorithms indicate that in a swamp often only a few rank-1 terms are modified while others stagnate. Often, the non-stagnant terms are problematic and form an ill-conditioned decomposition. To address this, we propose to temporarily freeze the stagnant terms. The lower number of terms has several benefits: it simplifies the problem by reducing the number of variables and reduces the cost per iteration significantly. Furthermore, in many cases the residual tensor can be compressed, and an algebraic (re)initialization can be carried out, even if this was not possible for the original tensor. A refinement step can further improve the accuracy if desired. We provide theoretical insights of why terms can stagnate. More specifically, we prove that terms that are close to the solution are not modified anymore in further optimization steps under certain assumptions. Extensive numerical experiments show that our framework greatly facilitates escaping from swamps. The resulting algorithm outperforms current state-of-the-art approaches on difficult-to-decompose tensors, both in accuracy and computation time, and has similar performance on easier problems.
10.50–11:40
Tomass Andersons, University of Rostock
Abstract: Nonnegative tensors often have a unique nonnegative factorization due to the inherent trilinearity property of the CANDECOMP/PARAFAC decomposition. A general uniqueness condition has been given by Kruskal [1].
This is in a sharp contrast to nonnegative matrix factorization problems, which often suffer from the ambiguity of the solutions. Thus, there has been a strong development of methods for analyzing the set of solutions. In the field of chemometrics, this goes back to the seminal works of Lawton and Sylvestre [2] and Borgen and Kowalski [3]. More recently, it has been developed into a comprehensive mathematical theory [4, 5, 6].
The question arises: What happens if the uniqueness of the tensor factorization is not satisfied For example, this can be caused by a rank deficiency [7]. There has been relatively little research in this direction [8]. We propose a systematic analysis of the ambiguity of nonnegative tensor factorizations by adapting techniques from the corresponding matrix factorization problems.
References
[1] J.B. Kruskal, Three-way arrays: rank and uniqueness of trilinear decompositions, with applications to arithmetic complexity and statistics, Linear Algebra Appl. 18 (1977) 95–138.
[2] W.H. Lawton, E.A. Sylvestre, Self modelling curve resolution, Technometrics 13 (1971) 617–633.
[3] O.S. Borgen, B.R. Kowalski, An extension of the multivariate component-resolution method to three components, Anal. Chim. Acta 174 (1985) 1–26.
[4] R. Rajkó, K. István, Analytical solution for determining feasible regions of self-modeling curve resolution (SMCR) method based on computational geometry, J. Chemom. 19 (2005) 448–463.
[5] K. Neymeyr, M. Sawall, On the set of solutions of the nonnegative matrix factorization problem, SIAM J. Matrix Anal. Appl. 39 (2018) 1049–1069.
[6] N. Gillis, Nonnegative Matrix Factorization, SIAM Philadelphia, PA (2020)
[7] M. Sawall, K. Neymeyr, On the area of feasible solutions for rank-deficient problems: I. Introduction of a generalized concept, J. Chemom. 35 (2020) e3316.
[8] N. Omidikia, H. Abdollahi, M. Kompany-Zareh, R. Rajkó, Trilinear self-modeling curve resolution using Borgen-Rajkó plot, J. Chemom. 34 (2020) e3161.
Lunch 12:00–13:00
13.15–14:05
Nico Vervliet, KU Leuven
Abstract: Tensor decompositions are powerful tools for analyzing (sets of) multiway data. Numerous algorithms have been developed to compute these decompositions, and their application has led to effective solution strategies. Tensorlab is a MATLAB toolbox designed for tensor computations and complex optimization, enabling users to leverage these algorithms and insights in a user-friendly manner. It includes high-level algorithms for canonical polyadic decomposition (CPD), Tucker, and block term decomposition, which automatically perform steps such as compression, initialization, and refinement for dense, sparse, incomplete, or structured tensors. Specialized algorithms, such as those using randomization or updating techniques, are also available. The structured data fusion framework allows easy incorporation of prior knowledge and data coupling through a custom domain-specific language.
Tensorlab+ serves as a repository focused on reproducing published results, including data and graphs. It provides well-documented, user-friendly, and rigorously tested MATLAB implementations of algorithms from algorithmic papers. Additional demos and tutorials illustrate their usage and properties. Tensorlab+ is more frequently updated with state-of-the-art results based on recent publications.
We also introduce pyTensorlab, the Python version of Tensorlab, which offers tools and algorithms for real and complex tensor decompositions, including CPD, Tucker, and tensor train decompositions. PyTensorlab emphasizes ease of use, with tensor classes that behave like NumPy arrays, high-level routines, adaptable low-level methods, extensive documentation, and typing.
In this talk, I will provide an overview of the capabilities of these software projects through various examples, including a new result on the decomposition into a sum of multilinear rank (Mr, Nr, .) terms. I will explain the rationale behind the different versions and shed light on their development