Homework 1:
Please reproduce the gradient explosion and vanishing phenomenon in the training of deep neural network. You are encouraged to look up the internet or discuss with other students in the class for potential solutions. Submission of a SINGLE .pdf file describing:
1) Algorithms
2) Value of the gradients during each iteration until vanishing/explosion, preferred to plot as figures
3) Attach code.
Submission by groups encouraged (there is no limit on the size of the group). Producing of gradient vanishing and explosion each carries 40% of grade. Clarity and presentation of results carry 20% of the grade. Due by the time announced in class.