History

The algorithms are trying to mimic brain.

It is very popular in 80s and early 90s. However, popular just diminished in late 90s.

It recently become popular becasue it 's the basement of the artificial intellengence.

Idea behind Neural Network

What Neural Network can do?

For supervise tasks, if the output is numeric, it can do predictions; if the output is categorical variables, it can do classifications.

For unsupervise learning, if the output node equals to input node, the hidden node information can used to present information at input node. So, it can do the feature extraction as PCA.

Data Preparation

Before the neural network process, it need standaradize the data. Because the neural network are data depend algorithms. The scale of variables can impact the performance of the result.

So, before the Neural Network process, scale and standard process need to be apply to the data.

Questions about structures

Terminology

Loss function:

To evaluate how model performs, like gini and impurity in decision tree.

The nerual network algorithms is about to minimize Loss

Learning rate:

An important parameter in neural network

range from 0 to 1

if learning rate is too big, you may miss the lowest point. if learning rate is too small, it takes time to find the point.

Activation function:

get point from input to output

usually considered as blackbox procedure

common activation functions:

Tanh, Relu, Sigmoid(classification) , Softmax( multi class classfication)

Forward Algorithms and backward propagation

Pros & Cons

Pros:

• Good predictive performance

• Have high tolerance to noisy data

• Ability to capture highly complicated relationships between the predictors and an outcome variable

Cons

• Blackbox reputation

• Extrapolation is a serious danger

• Does not have a built-in variable selection mechanism

• Relies heavily on having sufficient data for training purposes

• Computation time

I am still thinking put the example to above to help understanding.

Since it's kind of hard to understand what happens in Neural Network Procedures

But anyways, hope you quite enjoy this part

Happy neural network!