1. Adversarial Robustness

2. Alignment and Safety;  AI, Philosophy & Ethics

3. Compression and Fast Inference

4. Computational Psychology & Foundation Models

5. Continual  Learning  at Scale

6. Emergence, Phase Transitions and Stat Physics of ML

7. Foundation Models

8. Generalization (in-  and  out-of-distribution)

9. HPC and Distributed Model Training

10. Knowledge Fusion Across  Foundation Models 

11. Neural Scaling Laws

12. Out-of-Distribution Generalization

13. Scaling  Laws  in  Nature

14. State Space Models

15. Time Series Foundation Models

Other Relevant Courses on Scaling and Alignment Topics

AGI Safety Fundamentals    (2022 AI Safety Fundamentals course at Cambridge)

Jacob Hilton's Deep Learning Curriculum

Scaling bibliography by Gwern

Other recent courses:

U Waterloo course

Blogs

Scaling: Motivation

• The Bitter Lesson 

• Scaling Hypothesis

• The scaling hypothesis: a plan for building AGI

Large-Scale Pretrained Models

• ChatGPT: Optimizing Language Models for Dialogue - OpenAI 

• OpenAI's GPT-3 Language Model: A Technical Overview

• Conversations with GPT-3

• Do large language models understand us? 

• The FLOPs Calculus of Language Model Training

• New Scaling Laws for Large Language Models - LessWrong 

• All Machine Learning Algorithms You Should Know for 2023

• GPT 3 Demo and Explanation - An AI revolution from OpenAI

• GPT-3 Language Models are Few-Shot Learners (Paper Explained) - by Yannic Kilcher

• OpenAI GPT-3 Now Open to Public [FREE]

• Neural Scaling Laws and GPT-3

• GPT-3 Demo: New AI Algorithm Changes How We Interact With Technology

• 14 Cool Apps Built on OpenAI's GPT-3 API

• AI Chat Goes Horribly Wrong (GPT-3 interview)

• AI Robin Williams 2021 (GPT-3 / GPT 3 / GPT-J-6B) artificial intelligence

On alignment

• Alignment Research Center (lead by Paul Christiano) : blog

• AI Alignment Forum

• LessWrong

• Concepts Portal: Alignment Terminology

• Why AI alignment could be hard with modern deep learning  (Ajeya Cotra)

• Forecasting Transformative AI from Biological Anchors (Ajeya Cotra)

• Draft report on AI timelines (Ajeya Cotra)

• The Billion Dollar AI Problem That Just Keeps Scaling  (Ajeya Cotra)

• How does bee learning compare with machine learning?

• What failure looks like (Paul Christiano)

• Another (outer) alignment failure story (Paul Christiano)

• Christiano, Cotra, and Yudkowsky on AI progress

• Transformative AI and Compute

• Welcome to LessWrong 

• AGI Ruin: A List of Lethalities

• Simulators

• The Best Textbooks on Every Subject

Plans for AI Future: Different Perspectives

Toward Next-Generation Artificial Intelligence: Catalyzing the NeuroAI Revolution

The Alberta Plan for AI Research 

Toward Next-Generation Artificial Intelligence 

General Intelligence Requires Rethinking Exploration

Formal Aspects of Language Modeling