Phase Transitions & ML

Intro to Stat Mechanics (Stanford)       

Phase Transitions In Machine Learning (book)       download pdf

DeepAI.org on phase transitions

Exact Phase Transitions in Deep Learning

• Benign Overfitting and Grokking in ReLU Networks for XOR Cluster Data

• Droplets of Good Representations: Grokking as a First Order Phase Transition in Two Layer Networks

• Grokking as Compression: A Nonlinear Complexity Perspective

• Grokking as the Transition from Lazy to Rich Training Dynamics

• GROKKING TICKETS: LOTTERY TICKETS ACCELERATE GROKKING 

• The semantic landscape paradigm for neural networks 

• Predicting Grokking Long Before it Happens: A look into the loss landscape of models which grok 

• Explaining grokking through circuit efficiency 

• Progress measures for grokking via mechanistic interpretability

• Grokking phase transitions in learning local rules with gradient descent 

• Omnigrok: Grokking Beyond Algorithmic Data 

• Towards Understanding Grokking: An Effective Theory of Representation Learning

• Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets

Gwern's list

More examples: Jacob Steinhardt's blog: Future ML Systems Will Be Qualitatively Different - LessWrong 

 Phase Transitions in AI (Irina Rish)  at Neural Scaling workshop

Tutorial on Phase Transitions by Guillaume Dumas    video  (chapter 2)  slides (same workshop)

The statistical mechanics of learning a rule

Computation at the edge of chaos: Phase transitions and emergent computation 

Statistical Mechanics of Deep Learning 

Extended critical regimes of deep neural networks 

A PHASE TRANSITION FOR REPEATED AVERAGES 

A Mathematical Framework for Transformer Circuits

Statistical Mechanics of Learning : Generalization 

Learnability for the Information Bottleneck

Emergent Abilities of Large Language Models

Predictability and Surprise in Large Generative Models 

In-context Learning and Induction Heads 

Beyond the Imitation Game

Information and phase transitions in socio-economic systems

THE PHASE TRANSITION IN HUMAN COGNITION 

How critical is brain criticality?

Unveiling phase transitions with machine learning

Machine learning dynamical phase transitions in complex networks 

Phase transitions and critical behavior in human bimanual coordination

A universal scaling law between gray matter and white matter of cerebral cortex

Long-Range Temporal Correlations and Scaling Behavior in Human Brain Oscillations 

Power-Law Scaling in the Brain Surface Electric Potential 

 Synergetics: An Introduction Nonequilibrium Phase Transitions and Self-Organization in Physics, Chemistry and Biology

Phase transitions in the assembly of multivalent signaling proteins 

A theoretical model of phase transitions in human hand movements 

SOC

Neurobiologically Realistic Determinants of Self-Organized Criticality in Networks of Spiking Neurons 

Self-organized criticality as a fundamental property of neural systems 

Does the $1/f$ Frequency Scaling of Brain Signals Reflect Self-Organized Critical States?