(this site is under construction)
Deep Machine Learning is a new and old paradim of machine learning. On the one hand old, because the basic motivation are neural networks/multi-layer perceptron structures (besides others). On the other hand new, as learning deep networks with lots of hidden layers was not possible before or did not provide any benefit. One challenge was the "vanishing gradient" problem in backpropagation. I refer to great work from Hochreiter & Schmidhuber in 1991 [1] or 2001 [2].
The initial starting point of the deep learning wave was a contribution from Hinton and Salakhutdinov in the well known Science maganzine in 2006 [3]. They demonstrated efficient learning of deep nets by means of a stacked encoder pre-training strategy. Instead of learning the complete network, they divided the network in small encoder stages, stacked together the single stages and finally applied a fine tuning stage with the complete net.
There are tons of links and tutorials in the web to get in touch with deep learning. I highly recommend the following for newbie: Before you try to go deeper into deep learning, please, get an overview about other existing machine learning techniques. Great sources are for example massively open online courses (MOOC) provided by Coursera, Udacity, edX or Stanford Online Courses. An outstanding class on deep Convolutional Neural Networks (CNN) is provided by Standford, here.
A few links to tutorials or other link lists:
How to start with a deep network?
For sure, it totally depends on the problem you want to solve. The next steps are usefull for deep CNN, which are well established in image and video processing.
A personal view on Deep Machine Learning:
Literature:
[1] S. Hochreiter (1991): Untersuchungen zu dynamischen Neuronalen Netzen. Diploma Thesis, University of Munich. (supervised by Schmidhuber) LINK
[2] S. Hochreiter, Y: Bengio, P. Frasconi, J. Schmidhuber (2001): Gradient flow in recurrent nets: the difficulty of learning long-term dependencies. A Field Guide to Dynamical Recurrent Neural Networks, IEEE Press. LINK
[3] G. Hinton, R.R. Salakhutdinov (2006): Reducing the dimensionaly of data with neural networks. Science, 313, p. 504-507 LINK