Deep Learning

Geoffrey Hinton- God father of the Deep Learning

Introduction:

A new big trend in machine learning.
It refers to Artificial Neural Networks with many layers.
It can be used when the target function is very complex and the datasets are large.
An algorithm is deep if the input is passed through several non-linearities before being output.
Most of the modern learning algorithms (including decision trees, SVM and naive bayes) are "shallow".
Artificial Neural Networks are inspired from Biological Networks as follows

Biological Network - Artificial Neural Network(ANN)

Neuron - Node or PE

Dendrites - input

Axon - output

Synapse - weight

Artificial Neural Network (ANN):

It is a mathematical function that maps a given input to a desired output.
Neural Networks consist of the following components
- An input layer, x
- An arbitrary amount of hidden layers
- An output layer, ŷ
- A set of weights and biases between each layer, W and b
- A choice of activation function for each hidden layer, σ.
Note that the input layer is typically excluded when counting the number of layers in a Neural Network
The right values for the weights and biases determines the strength of the predictions
The process of fine-tuning the weights and biases from the input data is known as training the Neural Network.
Each iteration of the training process consists of the following steps:
- Calculating the predicted output ŷ, known as feedforward
- Updating the weights and biases, known as backpropagation
If we have the derivative of the loss function, we can simply update the weights and biases by increasing/reducing with it. This is known as gradient descent.
Our goal in training is to find the best set of weights and biases that minimizes the loss function

What we have achieved do far with Deep Learning?

Deep Learning Resources:

Learning:

Learning is an incremental change in weight between connected neurons in a neural network through time.
Supervised Learning
If the output variable/feature is discrete then the problem is called as classification problem

Ex. Predict tomorrow, it rains or not

Predict sanctioning a loan to a customer is high-risk or low-risk.

If the output variable/feature is continuous then the problem is called as regression problem

Ex. What is the temperature tomorrow?

Predict price of a car based on its features

Predict price of a house based on its attributes

Ex. Convolution Neural Networks (CNN), Recurrent Neural Networks (RNN)

Ex. Self Organizing Maps (SOM), Boltzmann Machines (BM), Auto Encoders (AE)

Reinforcement Learning
Agent will do some action in the Environment based on Reward/Penalty he aquired.

Activation Function:

1. Linear Activation Functions

2. Non-linear Activation Functions

Sigmoid or Logistic
Tanh or hyperbolic tangent
ReLU (Rectified Linear Unit)
- The ReLU is the most used activation function in the world right now.
- Since, it is used in almost all the convolutional neural networks or deep learning.
- f(z) is zero when z is less than zero and f(z) is equal to z when z is above or equal to zero.
- f(z)=max(0,z)
Leaky ReLU

Typical process for Deep Learning Project:

Deep Learning Libraries

Google Sites

Report abuse