Outline

We are covering chapters 5 to 10 of "Deep Learning" book by Ian Goodfellow, Yoshua Bengio, and Aaron Courville

Final project requirements:

1. Choose one area of application of machine learning. You can use chapter 12 of the “Deep Learning” book to learn about some of the applications

2. Then you can find a code and a data set you want to work on from the web.

3. Check with the instructors to make sure your choice is not too easy or hard

4. Explore different concepts (such as over-fitting, under-fitting, hyperparameters), different neural networks, different number of hidden layers, and optimization methods in your code.

5. Here is the heart of the project: by modifying your example, explore different concepts that you learned in this course, such as over-fitting, under-fitting, the (non)trainability of hyperparameters, different neural network architectures such as different numbers of hidden layers, and the relative performance of various optimization methods and various types of neural networks such as Feedforward, RNN, CNN, etc.

6. At the end, you need to be able to explain what your code is doing.

7. You need to be able to elaborate on the changes you have made and explain how and why they change the performance of your code for better or worse.

8. You can also elaborate on why the architecture or type of network that you have chosen is proper for the type of application you are exploring. For example, architectures and networks that are performing well in image recognition may not do well for translation tasks.

You will present your code and findings at the end of the term during the exam period.

Further details on the project and report formats:

  1. Your 20 min presentation should include
    • an introduction/brief background on the topic that you are exploring. For example, you can explain why that question is important, why it is difficult, what is the stage of the latest progress, etc.
    • your code, description of how it works and explanation of the modifications you have made and how and why the results and performance of the code have changed.

2. Your one-page report should include a brief introduction to the problem you are exploring and an explanation about your code including the modifications that you have studied and how/why they have changed the results.

3. You are submitting a one-page report, your code (annotated to show where you have made changes), and your presentation slides on the day or before December 6th.