Neural Network

Here let us study a supervised learning using a neural network. In a supervised learning, an conditional probability of an output y for a given input x is estimated. 

bayes041.mp4

Let an input x is in a 2-dimendional Euclidean space, and y in {0,1}. An unknown probability distribution q(y|x) is estimated by a neural network. In the movie, a network which is in a posterior distribution is displayed. 


bayes042.mp4

An estimated posterior predictive distribution is  displayed for each sample. The generalization error is far smaller than that by estimated using AIC. In a neural network, the essential dimension of a learning machine is far smaller than the dimension of the parameter. This is one of the main reason why deep learning has a better predictive performance. 

A simple model selection procedure is introduced. 5 candidate models are compared according to the small generalization loss.

Generalization loss can be estimated by WAIC and LOOCV. Note that these are smaller than estimated by AIC.