2021-04-28 APR

Journal Club 9:30-10:30

Luke Melas-Kyriazi. The Mathematical Foundations of Manifold Learning (Oct 2020) - Jeya

Manifold learning is a popular and quickly-growing subfield of machine learning based on the assumption that one's observed data lie on a low-dimensional manifold embedded in a higher-dimensional space. This thesis presents a mathematical perspective on manifold learning, delving into the intersection of kernel learning, spectral graph theory, and differential geometry. Emphasis is placed on the remarkable interplay between graphs and manifolds, which forms the foundation for the widely-used technique of manifold regularization. This work is written to be accessible to a broad mathematical audience, including machine learning researchers and practitioners interested in understanding the theorems underlying popular manifold learning algorithms and dimensionality reduction techniques.


Worksheet: https://colab.research.google.com/drive/1lrkoC1VzomMslUkje5OQAIQvAgvhMpBt?usp=sharing


Useful Links

  1. TF Playground: https://playground.tensorflow.org/

  2. ConvNetJS: https://cs.stanford.edu/people/karpathy/convnetjs/

  3. Overview: Goodfellow, I., Bengio, Y. and Courville, A., 2016. Representation learning. Deep learning, pp.517-548. (https://www.deeplearningbook.org/contents/representation.html)

  4. Christopher Olah's blog: http://colah.github.io/posts/2014-03-NN-Manifolds-Topology/

  5. Empirically testing the manifold hypothesis (from data): Fefferman, C., Mitter, S. and Narayanan, H., 2016. Testing the manifold hypothesis. Journal of the American Mathematical Society, 29(4), pp.983-1049. (https://arxiv.org/abs/1310.0425)

  6. Using RBF kernels in neural networks: Rahimi, A. and Recht, B., 2007, December. Random Features for Large-Scale Kernel Machines. In NIPS (Vol. 3, No. 4, p. 5). (https://people.eecs.berkeley.edu/~brecht/papers/07.rah.rec.nips.pdf)

Implementation: https://www.tensorflow.org/api_docs/python/tf/keras/layers/experimental/RandomFourierFeatures


___ Audience Notes ___

How does TF help us with manifolds

Hackathon 10:30 - 11:30

Custom Elements

Update from Lee

RStudio - the other Web browser

Daniel, Wendy, Lorena, anyone...

Quest

finish discussion by Kundan

...

...