research
exploring geometric and topological structure in data - manifolds, dimension reduction
networks, graphs and relational machine learning - spectral embedding
simulation-based inference - Monte Carlo methods, particle filters
uncertainty dynamics - uncovering latent structure in dynamic and streaming data
selected recent projects
Hierarchical clustering with dot products recovers hidden tree structure
NeurIPS 2023 Spotlight.
In this paper we offer a new perspective on the well established agglomerative clustering algorithm, focusing on recovery of hierarchical structure. We recommend a simple variant of the standard algorithm, in which clusters are merged by maximum average dot product and not, for example, by minimum distance or within-cluster variance. We demonstrate that the tree output by this algorithm provides a bona fide estimate of generative hierarchical structure in data, under a generic probabilistic graphical model. The key technical innovations are to understand how hierarchical information in this model translates into tree geometry which can be recovered from data, and to characterise the benefits of simultaneously growing sample size and data dimension. We demonstrate superior tree recovery performance with real data over existing approaches such as UPGMA, Ward's method, and HDBSCAN.
See the arXiv posting, plus code on GitHub.
Statistical Exporation of the Manifold Hypothesis
The Manifold Hypothesis is a widely accepted tenet of Machine Learning which asserts that nominally high-dimensional data are in fact concentrated near a low-dimensional manifold, embedded in high-dimensional space. This phenomenon is observed empirically in many real world situations, has led to development of a wide range of statistical methods in the last few decades, and has been suggested as a key factor in the success of modern AI technologies. We show that rich and sometimes intricate manifold structure in data can emerge from a generic and remarkably simple statistical model -- the Latent Metric Model -- via elementary concepts such as latent variables, correlation and stationarity. This establishes a general statistical explanation for why the Manifold Hypothesis seems to hold in so many situations. Informed by the Latent Metric Model we derive procedures to discover and interpret the geometry of high-dimensional data, and explore hypotheses about the data generating mechanism. These procedures operate under minimal assumptions and make use of well known, scaleable graph-analytic algorithms.
See the arXiv posting, an earlier NeurIPS paper, plus code on GitHub.
Fast and consistent inference in compartmental models of epidemics using Poisson Approximate Likelihoods
To appear in JRSSB.
Addressing the challenge of scaling-up epidemiological inference to complex and heterogeneous models, we introduce Poisson Approximate Likelihood (PAL) methods. In contrast to the popular ODE approach to compartmental modelling, in which a large population limit is used to motivate a deterministic model, PALs are derived from approximate filtering equations for finite-population, stochastic compartmental models, and the large population limit drives consistency of maximum PAL estimators. Our theoretical results appear to be the first likelihood-based parameter estimation consistency results which apply to a broad class of partially observed stochastic compartmental models and address the large population limit. PALs are simple to implement, involving only elementary arithmetic operations and no tuning parameters, and fast to evaluate, requiring no simulation from the model and having computational cost independent of population size. Through examples we demonstrate how PALs can be used to: fit an age-structured model of influenza, taking advantage of automatic differentiation in Stan; compare over-dispersion mechanisms in a model of rotavirus by embedding PALs within sequential Monte Carlo; and evaluate the role of unit-specific parameters in a meta-population model of measles.
See the arXiv posting, plus code on GitHub.
Implications of sparsity and high triangle density for graph representation learning
AISTATS 2023 selected for oral presentation
Recent work has shown that sparse graphs containing many triangles cannot be reproduced using a finite-dimensional representation of the nodes, in which link probabilities are inner products. Here, we show that such graphs can be reproduced using an infinite-dimensional inner product model, where the node representations lie on a low-dimensional manifold. Recovering a global representation of the manifold is impossible in a sparse regime. However, we can zoom in on local neighbourhoods, where a lower-dimensional representation is possible. As our constructions allow the points to be uniformly distributed on the manifold, we find evidence against the common perception that triangles imply community structure.
See the AISTATS 2023 website.
Exploiting locality in high-dimensional factorial hidden Markov models
Using the Dobrushin Comparison theorem to rigorously justify approximate inference methods in factorial hidden Markov models, which allow them to be scaled up to high-dimensional problems. The approximations are accurate and inexpensive to compute when the model has natural spatial or network structure. The algorithms are demonstrated by analyzing Oyster Card "tap" data to predict passenger flow on the London Underground.
See the arXiv posting, the paper is published in the Journal of Machine Learning Research, and check out code on GitHub.