Research

Abstract: 

We present our position on the elusive quest for a general-purpose framework for specifying and studying deep learning architectures. Our opinion is that the key attempts made so far lack a coherent bridge between specifying constraints which models must satisfy and specifying their implementations. Focusing on building a such a bridge, we propose to apply category theory -- precisely, the universal algebra of monads valued in a 2-category of parametric maps -- as a single theory elegantly subsuming both of these flavours of neural network design. To defend our position, we show how this theory recovers constraints induced by geometric deep learning, as well as implementations of many architectures drawn from the diverse landscape of neural networks, such as RNNs. We also illustrate how the theory naturally encodes many standard constructs in computer science and automata theory.

Most recent  draft 24 February 2024

Abstract: 

This paper is the first in a series of two papers, Z-Categories I and Z-Categories II, which develop the notion of Z-category, the natural bi-infinite analog to strict ω-categories, and show that the (∞,1)-category of spectra sits inside the (∞,1)-category of homotopy coherent pointed Z-categories as the pointed groupoids. In this work we provide a 2-categorical treatment of the combinatorial spectra of [Kan] and argue that this description is a simplicial avatar of the abiding notion of homotopy coherent Z-category. We then develop the theory of limits in the 2-category of categories with arities of [BergerMelliesWeber] to provide a cellular category which is to Z-categories as △ is to 1-categories or Θ-n is to n-categories. In an appendix we provide a generalization of the spectrification functors of 20th century stable homotopy theory in the language of category weighted limits.

Most recent draft 2 Jun 2022

Abstract: 

This paper develops some combinatorics of the lax Gray cylinder on the cells of Θ  understood as a full subcategory of the category of strict ω -categories. More, we construct a span relating the Cartesian cylinder, the Gray cylinder, and the shift functor.

Most recent draft 27 Jun 2022

Abstract:

This paper is the second in a series of two papers,  Z-Categories I and  Z-Categories II, which develop the notion of  Z-category, the natural bi-infinite analog to strict ω -categories, and show that the (∞ ,1)-category of spectra sits inside the (∞ ,1)-category of homotopy coherent pointed  Z-categories as the locally finite groupoids. In particular we develop the combinatorics and formal homotopy theory necessary to take that argument to its conclusion, by developing a cellular description of  Z-categories and then showing that the model category which presents homotopy coherent pointed   Z-groupouds in that cellular language is Quillen equivalent to the usual model categories of spectra.

We develop a type type for symmetric monoidal bicategories generalizing Shulman's practical type theory for symmetric monoidal categories.

We develop variants of Shulman's practical type theory for symmetric monoidal categories for symmetric monoidal categories. As a result the "abstract index notation" suggested by Fritz for all Markov categories is formalized as syntactic sugar for MPTT.

See my talk at the TOPOS Institute

Abstract: With Zipf's law being originally and most famously observed for word frequency, it is surprisingly limited in its applicability to human language, holding over no more than three to four orders of magnitude before hitting a clear break in scaling. Here, building on the simple observation that phrases of one or more words comprise the most coherent units of meaning in language, we show empirically that Zipf's law for phrases extends over as many as nine orders of rank magnitude. In doing so, we develop a principled and scalable statistical mechanical method of random text partitioning, which opens up a rich frontier of rigorous text analysis via a rank ordering of mixed length phrases. 

Warning : This contains a non-trivial error in the development of what I call the Eckmann-Hilton degeneracy. Also, the local finiteness hypothesis is unnecessary up to homotopy. These mistakes are fixed in the pre-prints.

Warning : This is almost certainly not the document I would write now, but I stand by many of the ideas. Also, I think the figures are pretty great! In a sense they are far better than those I make now having been forced/convinced to use TikZ instead of things like Processing.