Intro to Stat Mechanics (Stanford)       

Phase Transitions In Machine Learning (book)       download pdf

DeepAI.org on phase transitions

Exact Phase Transitions in Deep Learning

Grokking

1. Benign Overfitting and Grokking in ReLU Networks for XOR Cluster Data

2. Droplets of Good Representations: Grokking as a First Order Phase Transition in Two Layer Networks

3. Grokking as Compression: A Nonlinear Complexity Perspective

4. Grokking as the Transition from Lazy to Rich Training Dynamics

5. GROKKING TICKETS: LOTTERY TICKETS ACCELERATE GROKKING 

6. The semantic landscape paradigm for neural networks 

7. Predicting Grokking Long Before it Happens: A look into the loss landscape of models which grok 

8. Explaining grokking through circuit efficiency 

9. Progress measures for grokking via mechanistic interpretability

10. Grokking phase transitions in learning local rules with gradient descent 

11. Omnigrok: Grokking Beyond Algorithmic Data 

12. Towards Understanding Grokking: An Effective Theory of Representation Learning

13. Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets

Exact Phase Transitions in Deep Learning

Emergent Abilities of Large Language Models

Predictability and Surprise in Large Generative Models 

In-context Learning and Induction Heads 

Beyond the Imitation Game

Renormalization group theory, scaling laws and deep learning (master thesis by Parviz Haggi)

Using Renormalisation theory to understand how deep neural networks generalises

Renormalization group flow as optimal transport

Is Deep Learning a Renormalization Group Flow? 

Why does deep and cheap learning work so well?

Gwern's list

More examples: Jacob Steinhardt's blog: Future ML Systems Will Be Qualitatively Different - LessWrong 

 Phase Transitions in AI (Irina Rish)  at Neural Scaling workshop

Tutorial on Phase Transitions by Guillaume Dumas    video  (chapter 2)  slides (same workshop)

The statistical mechanics of learning a rule

Computation at the edge of chaos: Phase transitions and emergent computation 

Statistical Mechanics of Deep Learning 

Extended critical regimes of deep neural networks 

A PHASE TRANSITION FOR REPEATED AVERAGES 

A Mathematical Framework for Transformer Circuits

Statistical Mechanics of Learning : Generalization 

Learnability for the Information Bottleneck

Information and phase transitions in socio-economic systems

THE PHASE TRANSITION IN HUMAN COGNITION 

How critical is brain criticality?

Unveiling phase transitions with machine learning

Machine learning dynamical phase transitions in complex networks 

Phase transitions and critical behavior in human bimanual coordination

A universal scaling law between gray matter and white matter of cerebral cortex

Long-Range Temporal Correlations and Scaling Behavior in Human Brain Oscillations 

Power-Law Scaling in the Brain Surface Electric Potential 

 Synergetics: An Introduction Nonequilibrium Phase Transitions and Self-Organization in Physics, Chemistry and Biology

Phase transitions in the assembly of multivalent signaling proteins 

A theoretical model of phase transitions in human hand movements 

SOC

Neurobiologically Realistic Determinants of Self-Organized Criticality in Networks of Spiking Neurons 

Self-organized criticality as a fundamental property of neural systems 

Does the $1/f$ Frequency Scaling of Brain Signals Reflect Self-Organized Critical States? 

Talks & Tutorials on Phase Transitions from the 1st Neural Scaling Laws Workshop: Maximally Beneficial AGI

Phase Transitions in AI (Irina Rish) 

Critical views about critical phenomena:  from complex systems to machine learning (Guillaume Dumas)   video  (chapter 2)  slides

Brief Overview: Physics meets ML (Guillaume Dumas, Maximilian Puelma Touzel, Irina Rish) video

Future ML Systems WIll be Qualitatively Different

 1st Neural Scaling Laws Workshop: Maximally Beneficial AGI

5th Neural Scaling Laws workshop on Emergence and Phase Transitions   (Neural Scaling Laws Workshop series)