Statistical models and learning machines which have hidden variables or hierarchical structures have singularities in their parameter spaces. To study such models and machines, first, we go to algebraic geometry which seems to be far far away from statistical learning theory. 

Second, we take an extremely difficult journey back from algebraic geometry to statistical learning theory. At that time, a new field called singular learning theory will be developed. The path passes through resolution theorem, zeta functions, Schwartz distributions, state density functions, partition functions, empirical process theory, and finally reaches free energy and generalization loss. 

Remark : You might think that algebraic geometry has no relation to statistics. However, if you walk the road back from algebraic geometry to statistical learning theory, you will find the completely new, beautiful, and useful world by your own experience. 

Why algebraic geometry ?

In deep learning,  a  statistical sub model corresponds to an algebraic variety in the parameter space. Statistical properties of such models can be captured by two birational invariants. This is the reason why algebraic geometry is necessary to understand deep learning process.

Note: There are many singular learning machines, for example, neural networks, normal mixtures, Boltzmann machines, reduced rank regressions, latent Dirichlet allocations, and so on. 

S. Watanabe. Almost all learning machines are singular, IEEE Symposium on Foundations of Computational Intelligence, 2007. DOI: 10.1109/FOCI.2007.371500 

In deep learning, most of eigenvalues of Fisher information matrix are zero. Hence the classical regular theory does not hold. The maximum likelihood estimator often diverges and the posterior distribution is far from any normal distribution. 

Basic Answer by Algebraic Geometry

The problem in machine learning theory caused by singularities is solved by the basic theorem in algebraic geometry. For the statement and concrete example, please see Hironaka resolution theorem. An arbitrary singularity can be made normal crossing in each local coordinate on an appropriate manifold by using a birational transform. This theorem is the most basic and important one in algebraic geometry proved in 1964, whose proof was given by Professor Heisuke Hironaka in RIMS Kyoto University. 

Birational Invariants

The learning process is captured by two birational invariants, real log canonical threshold and singular fluctuation, both of which can be explicitly defined by using the resolution theorem. You will be able to understand the fact that these two mathematical concepts play the central role in machine learning theory.