1. Adversarial Robustness

2. Alignment and Safety;  AI, Philosophy & Ethics

3. Compression and Fast Inference

4. Computational Psychology & Foundation Models

5. Continual  Learning  at Scale

6. Emergence, Phase Transitions and Stat Physics of ML

7. Foundation Models

8. Generalization (in-  and  out-of-distribution)

9. HPC and Distributed Model Training

10. Knowledge Fusion Across  Foundation Models 

11. Neural Scaling Laws

12. Out-of-Distribution Generalization

13. Scaling  Laws  in  Nature

14. State Space Models

15. Time Series Foundation Models

Other Relevant Courses on Scaling and Alignment Topics

AGI Safety Fundamentals    (2022 AI Safety Fundamentals course at Cambridge)

Jacob Hilton's Deep Learning Curriculum

Scaling bibliography by Gwern

Other recent courses:

U Waterloo course

Blogs

Scaling: Motivation

• The Bitter Lesson 

• Scaling Hypothesis

• The scaling hypothesis: a plan for building AGI

Large-Scale Pretrained Models

• ChatGPT: Optimizing Language Models for Dialogue - OpenAI 

• OpenAI's GPT-3 Language Model: A Technical Overview

• Conversations with GPT-3

• Do large language models understand us? 

• The FLOPs Calculus of Language Model Training

• New Scaling Laws for Large Language Models - LessWrong 

• All Machine Learning Algorithms You Should Know for 2023

• GPT 3 Demo and Explanation - An AI revolution from OpenAI

• GPT-3 Language Models are Few-Shot Learners (Paper Explained) - by Yannic Kilcher

• OpenAI GPT-3 Now Open to Public [FREE]

• Neural Scaling Laws and GPT-3

• GPT-3 Demo: New AI Algorithm Changes How We Interact With Technology

• 14 Cool Apps Built on OpenAI's GPT-3 API

• AI Chat Goes Horribly Wrong (GPT-3 interview)

• AI Robin Williams 2021 (GPT-3 / GPT 3 / GPT-J-6B) artificial intelligence

On alignment

• Alignment Research Center (lead by Paul Christiano) : blog

• AI Alignment Forum

• LessWrong

• Concepts Portal: Alignment Terminology

• Why AI alignment could be hard with modern deep learning  (Ajeya Cotra)

• Forecasting Transformative AI from Biological Anchors (Ajeya Cotra)

• Draft report on AI timelines (Ajeya Cotra)

• The Billion Dollar AI Problem That Just Keeps Scaling  (Ajeya Cotra)

• How does bee learning compare with machine learning?

• What failure looks like (Paul Christiano)

• Another (outer) alignment failure story (Paul Christiano)

• Christiano, Cotra, and Yudkowsky on AI progress

• Transformative AI and Compute

• Welcome to LessWrong 

• AGI Ruin: A List of Lethalities

• Simulators

• The Best Textbooks on Every Subject

Plans for AI Future: Different Perspectives

Toward Next-Generation Artificial Intelligence: Catalyzing the NeuroAI Revolution

The Alberta Plan for AI Research 

Toward Next-Generation Artificial Intelligence 

General Intelligence Requires Rethinking Exploration

Neural Scaling Laws and Foundation Models:

• GPT-3 paper: Language Models are Few-Shot Learners

• Scaling  Laws for Neural Language Models 

• Scaling Laws for Transfer 

• Training compute-optimal large language  models (summary/blog: New Scaling Laws for LLMs)

•  Broken Neural Scaling Laws  

•  Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets   

•  Are emergent abilities of LLMs a Mirage?Continual Learning:

• Effect of scale on catastrophic forgetting in neural networks  

• Fine-tuned Language Models are Continual Learners

• Simple and Scalable Strategies to Continually Pre-train Large Language Models 

• Investigating Continual Pretraining in Large Language Models  

Foundation Models & Alignment:

• Alignment faking in large language models 

• AI Deception: A Survey of Examples, Risks, and Potential Solutions

• RLHF: How to Learn from Human Feedback with Reinforcement Learning    

• Large Language Models can Strategically Deceive their Users when Put Under Pressure   

• Unsolved Problems in ML Safety

• AI alignment: A comprehensive survey

•  Who is ChatGPT? Benchmarking LLMs' Psychological Portrayal Using PsychoBench 

• Investigating the Applicability of Self-Assessment Tests for Personality Measurement of Large Language Models

Formal Aspects of Language Modeling