I spent a lot of time putting together my own course of study for deep learning, machine learning, natural language processing, and practical Python coding. I benefitted from recommendations from many friends & colleagues. Below are my pooled & vetted recommendations by topic. All of the material is free online.
1: Deep Learning in Practice: FastAI’s “Practical Deep Learning for Coders” course with Jeremy Howard
What: Fantastic high-level exposure to many cutting-edge tools in deep learning, with a focus on implementable Python code (using the FastAI library, built on top of Facebook’s PyTorch library).
Why: Learn a little about a lot (breadth, not depth), providing a terrific knowledge base to build upon
Topics covered: Neural networks, image classification, gradient descent, collaborative filtering & latent factors, decision trees (bagging, boosting, random forest), natural language processing with recurrent neural networks
2020 course link: https://course20.fast.ai/
2022 course link: https://course.fast.ai/
2A: Mechanics of Deep Learning: MIT 6.036 Intro to Machine Learning with Leslie Kaelbling
What: The course is available through MIT OpenCourseWare. The course PDFs read like a textbook, giving a terrific detailed overview of many machine learning methods, building everything up from the basics. The course assumes knowledge of only basic statistics, probability, and econometrics.
Why: Explains the mechanics (ie: details) of many popular deep learning methods. For example, weeks 6 & 7 go through the nitty-gritty math of optimizing a neural net, giving me a much better understanding of what happens under-the-hood of neural net packages!
Topics covered: Classification (logistic regression), gradient descent (batch, mini-batch, stochastic), regression (OLS & ridge to illustrate gradient descent), neutral networks (feed-forward single & multi-layer nets, convolutional neural nets, recurrent neural nets), reinforcement learning, recommender systems (content-based and collaborative filtering), and several non-parametric methods (decision trees & nearest neighbors)
Course link: https://ocw.mit.edu/courses/6-036-introduction-to-machine-learning-fall-2020/
2B: Mechanics of Deep Learning: Stanford CS 229 Machine Learning with Andrew Ng
What: Course textbook provides a complementary detailed overview of many machine learning methods, with a greater level of detail & complexity than MIT 6.036.
Why: Adds greater detail to the mechanics of popular deep learning methods. Chapter 7 (Deep Learning) provides a lot of incremental value for understanding neural nets – the rest of the material is somewhat redundant after the MIT 6.036 course however.
Topics covered: Classification, kernel methods, neural networks, bias-variance tradeoff & double descent, clustering
Course link: https://cs229.stanford.edu/
Textbook: https://cs229.stanford.edu/lectures-spring2022/main_notes.pdf
YouTube playlist: https://www.youtube.com/playlist?list=PLoROMvodv4rMiGQp3WXShtMGgzqpfVfbU
3: Natural Language Processing in Practice: FastAI’s Code-First Intro to Natural Language Processing with Rachel Thomas
What: Great high-level overview of current technology in the rapidly-advancing natural language processing field. This is a good starting point to get a sense of WHICH tools people actually use today for NLP — for example, parse trees seem quite outdated but are popular in textbooks from the 2000’s. The course is taught with integrated Jupyter notebooks and is VERY applied, although a variety of technical methods are only loosely explained. I found it very helpful to look up the suggested references (eg: Jay Alammar’s blog posts on transformers & attention —very excellent!) and pause to search for supplementary materials when I was struggling to understand what was going on.
Why: NLP courses quickly become outdated. This course covers cutting-edge methods, provides implementable code, and gives you a good sense of what else is relevant. Excellent starting point!
Topics covered: Topic modeling (with singular value decomposition and non-negative matrix factorization), term-frequency inverse-document-frequency representations, sentiment classification (with naive Bayes and n-grams), language modeling, transfer learning, ULMFit (Jeremy Howard has an excellent 2018 paper on it), RNNs, sequence-to-sequence (seq2seq) translation (LSTM, GRU, teacher forcing, attention), and transformers.
Course Link: https://www.fast.ai/2019/07/08/fastai-nlp/
YouTube playlist: https://www.youtube.com/playlist?list=PLtmWHNX-gukKocXQOkQjuVxglSDYWsSh9
GitHub Repo: https://github.com/fastai/course-nlp
4: Applied Python: Quant Econ with Thomas Sargent, John Stachurski, and many others
What: QuantEcon – developed by a large team of quantitative economists and funded by the Alfred P. Sloan Foundation – is a text course providing a terrific overview for how to program in Python for Econ & Finance applications. It’s useful for both academic & industry careers.
Why: I never had proper training in Python or programming. This taught me what tools are out there to make data manipulation & analysis easier. For example, if you don’t know what a pivot table is, this course will knock your socks off! I became a much better Python coder in just a week.
Topics covered: There are a variety of sub-courses.
Python Programming for Economics and Finance: Terrific intro to Python, object oriented programming, and the main scientific libraries (NumPy, Matplotlib, SciPy, Numba, & Pandas)
Introduction to Economic Modeling and Data Science: The Pandas module is detail-oriented and taught me a lot about: pandas functionality (aggregation, transforms, hierarchical indexing), data reshaping (eg: stacking/unstacking, pivot tables, GroupBy), merging (with concat, merge, join), datetime objects (parsing, rolling windows, resampling/changing frequency). Additional modules discuss plotting (types of plots & common customizations), regression (basic concepts & Python code for linear/multivariate regression, lasso & k-fold cross validation, regression trees & random forests, neural nets), classification (logistic regression, neural nets), text data (Naive Bayes Classifier, Latent Semantic Analysis, Latent Dirichlet Analysis), and creative applications of machine learning in economics (partially linear regression). Much of the pandas functionality was new to me, and the additional modules provided a great high-level overview of many methods & tools.