Lecture notes (not revised!) are weekly uploaded in the Teams channel - please register here if you want to join.
Main references
On geometric regularity priors
Michael M. Bronstein, Joan Bruna, Taco Cohen, Petar Veličković. Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges. https://geometricdeeplearning.com/
On wavelet scattering networks
Joan Bruna and Stéphane Mallat. Invariant scattering convolution networks. IEEE Transactions on pattern analysis and machine intelligence (PAMI), 35(8) (2013), 1872-1886.
Stéphane Mallat. Group invariant scattering. Comm. Pure Appl. Math. 65 (2012), no. 10, 1331-1398.
Collection of papers from Mallat's group on theory and applications of scattering networks: https://www.di.ens.fr/data/publications/
On generalized scattering networks
Philipp Grohs, Thomas Wiatowski and Helmut Bölcskei. Deep convolutional neural networks on cartoon functions. In: 2016 IEEE International Symposium on Information Theory (ISIT), 2016, 1163–1167.
Thomas Wiatowski and Helmut Bölcskei. A mathematical theory of deep convolutional neural networks for feature extraction. IEEE Trans. Inform. Theory 64 (2018), no. 3, 1845-1866.
Thomas Wiatowski and Helmut Bölcskei. Deep convolutional neural networks based on semi-discrete frames. In: 2015 IEEE International Symposium on Information Theory (ISIT), Hong Kong, China, 2015, 1212-1216.
Stability of convolutional neural networks
Radu Balan, Maneesh Singh and Dongmian Zou. Lipschitz properties for deep convolutional networks. Contemporary Mathematics 706 (2018), 129-151.
Alberto Bietti and Julien Mairal. Group invariance, stability to deformations, and complexity of deep convolutional representations. Journal of Machine Learning Research (JMLR) 20(25) (2019):1-49.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In: 2017 IEEE Symposium on Security and Privacy (SP), San Jose, CA, USA, 2017, 39–57.
Michael Koller, Johannes Großmann, Ullrich Monich and Holger Boche. Deformation stability of deep convolutional neural networks on Sobolev spaces. In: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, AB, 2018, 6872–6876.
Fabio Nicola and S. Ivan Trapasso. On the stability of deep convolutional neural networks under irregular or random deformations. arXiv:2104.11977.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow and Rob Fergus. Intriguing properties of neural networks. In: 2nd International Conference on Learning Representations, ICLR 2014.
Dongmian Zou, Radu Balan and Maneesh Singh. On Lipschitz bounds of general convolutional neural networks. IEEE Trans. on Info. Theory 66(3) (2020), 1738–1759.
General mathematical references
Ingrid Daubechies. Ten Lectures on Wavelets. Society for industrial and applied mathematics, 1999.
Karlheinz Gröchenig. Foundations of Time-Frequency Analysis. Applied and Numerical Harmonic Analysis. Birkhäuser Boston, Inc., Boston, MA, 2001.
Christopher Heil. An introduction to weighted Wiener amalgams. In: Wavelets and their applications. Ed. by S. Thangavelu, M. Krishna, R. Radha. Allied Publishers, New Dehli, 2003, 183–216.
Stéphane Mallat. A Wavelet Tour of Signal Processing. The Sparse Way. Third edition. With contributions from Gabriel Peyré. Elsevier/Academic Press, Amsterdam, 2009.
List of possible topics for the final exam (non-exhaustive, weekly updates)
No free lunch theorem
PAC learnability of infinite classes (VC dimension, Rademacher complexity, etc.)
Perceptron algorithm and SVM
Expressivity of neural networks (universal approximation vs regularity)
Geometry of balls, spheres and curves in high dimensions
Training and optimization of neural networks (SGD, backpropagation, etc.)
Barron's theorem
The role of depth (deep vs shallow NN expressivity)
Adversarial perturbations in NN and false structures
Mathematics of kernel methods
Neurogeometry and visual perception
Proofs of the main scattering properties from [Mallat CPAM 2012]
Integral scattering transform
From scattering transforms to scattering networks: design, fast scattering transform, algorithms
Energy propagation in scattering networks
Scattering inversion, wavelet phase retrieval
Scattering representations of stochastic processes, with applications to texture discrimination
Multifractal scattering
Invariant scattering over general groups of symmetries, with applications
Separable vs. joint scattering (roto-translations, time-frequency)
Non-Euclidean scattering (graphs, manifolds)
Generative scattering, microcanonical models
Generative networks as inverse problems with scattering transforms
Scattering models for financial time series
Nonlinear activation functions and phase harmonic correlations
Sparse scattering and homotopy dictionary learning