Artificial Neural Networks

January 27 / January 29 (Week 2)

The Hundred-Page Machine Learning Book:

• Preface

• Chapter 1 - Introduction

• Chapter 2 - Notation

Deep Learning with Python:

• Chapter 1 - What is deep learning?

• Chapter 4, Section 4.1 - Four branches of machine learning

February 3 / February 5 (Week 3)

Neural Networks and Deep Learning:

• What this book is about

• On the exercises and problems

• Chapter 1 - Using neural nets to recognize handwritten digits

  • Perceptrons

  • Sigmoid neurons

Wikipedia - Perceptron

February 10 / February 12 (Week 4)

Neural Networks and Deep Learning:

• Chapter 1 - Using neural nets to recognize handwritten digits

  • The architecture of neural networks

  • A simple network to classify handwritten digits

  • Learning with gradient descent

The Hundred-Page Machine Learning Book:

• Chapter 2 - Notation

February 17 / February 19 (Week 5)

The Hundred-Page Machine Learning Book:

• Chapter 4 - Anatomy of a Learning Algorithm

Neural Networks and Deep Learning:

• Chapter 1 - Using neural nets to recognize handwritten digits

  • Learning with gradient descent

  • Implementing our network to classify digits

  • Toward deep learning

A Whirlwind Tour of Python:

• Chapter 16 - A Preview of Data Science Tools

Deep Learning with Python:

• Chapter 2 - Before we begin: the mathematical building blocks of neural networks

February 24 / February 26 (Week 6)

Project 1, due March 11

Neural Networks and Deep Learning:

• Chapter 2 - How the backpropagation algorithm works

  • Warm up: a fast matrix-based approach to computing the output from a neural network

  • The two assumptions we need about the cost function

3blue1brown - Neural networks

March 2 / March 4 (Week 7)

Extra Credit added to Project 1.

Calculus on Computational Graphs: Backpropagation

Neural Networks and Deep Learning:

• Chapter 2- How the backpropagation algorithm works

  • The Hadamard product

  • The four fundamental equations behind backpropagation

  • Proof of the four fundamental equations

  • The backpropagation algorithm

  • The code for backpropagation

  • In what sense is backpropagation a fast algorithm?

  • Backpropagation: the big picture

March 9 / March 11 (Week 8)

Neural Networks and Deep Learning:

• Chapter 3 - Improving the way neural networks learn

  • The cross-entropy cost function

  • Overfitting and regularization

  • Handwriting recognition revisited: the code

  • How to choose a neural network's hyper-parameters?

Deep Learning:

• Section 3.13 - Information Theory

• (Reference) Chapter 7 - Regularization for Deep Learning

Project 1 due March 11

March 16 / March 18 (Week 9)

Project 2, due April 10

Neural Networks and Deep Learning:

• Chapter 3 - Improving the way neural networks learn

  • Weight initialization

  • Other techniques

• Chapter 4 - A visual proof that neural nets can compute any function

Deep Learning:

• Section 6.4.1 -  Universal Approximation Properties and Depth

• (Reference) Chapter 8 - Optimization for Training Deep Models

March 23 / March 25 (Week 10)

Midterm Exam

March 30 / April 1 (Week 11)

Spring Break

April 6 / April 8 (Week 12)

Neural Networks and Deep Learning:

• Chapter 5 - Why are neural networks hard to train?

An Intuitive Explanation of Why Batch Normalization Really Works

2018 Turing Award

The data that transformed AI research—and possibly the world

Project 2 due April 10

April 13 / April 15 (Week 13)

Presentations, Weeks 15 and 16

Project 3, due May 13

Neural Networks and Deep Learning:

• Chapter 6 - Deep learning

  • Introducing convolutional networks

  • Convolutional neural networks in practice

  • The code for our convolutional networks

Deep Learning:

• Chapter 9 - Convolutional Networks

  • Section 9.1 - The Convolution Operation

  • Section 9.2 - Motivation

  • Section 9.3 - Pooling

  • Section 9.5 - Variants of the Basic Convolution Function

  • Section 9.7 - Data Types

  • Section 9.10 - The Neuroscientific Basis for Convolutional Networks

  • Section 9.11 - Convolutional Networks and the History of Deep Learning

April 20 / April 22 (Week 14)

Neural Networks and Deep Learning:

• Chapter 6 - Deep learning

  • Recent progress in image recognition

  • Other approaches to deep neural nets

  • On the future of neural networks

Deep Learning with Python:

• Chapter 3 - Getting started with neural networks

• Chapter 5 - Deep learning for computer vision

April 27 / April 29 (Week 15), May 4 / May 6 (Week 16)

Presentations

May 11 (Finals Week)

Final Exam - 5:00 - 6:50 pm 

Project 3 due