IMPORTANT: Students intending to take the exam are kindly requested to send the teachers an email with the chosen exam paper as soon as possible, and in any case, before the exam booking deadline. Once we have received the complete list, we will write to those interested to fix the schedule.
Winter exam session: .27/1/2025 - 3/2/2025 - 19/2/2025 - 24/2/2025 - 25/2/2025
Summer examination session:
1st: 5-6/6/2025
2nd: 10/7/2025 h 17:00 (NEW TIME) office of Prof. Loreto, "Fermi" building, 2nd floor, office 204
Please (compulsory)
1) send an email to sistemicomplessi.sapienza@gmail.com to express the intention to take the exam and the title of the essay to be presented
2) enroll in INFOSTUD! (It will be available on June 30th)
Autumn exam session: To be announced later. Very likely from 8th to 19th of September.
Exam modalities:
The course is divided into two parts: the first, which constitutes approximately two-thirds of the course, deals in detail with fundamental topics; the second takes the form of seminars that explore specific research topics related to the study of complex systems. The examination consists of two stages, reflecting the course's two phases.
1- Students will have to be able to discuss a short paper on one of the topics addressed during the proposed lectures and seminars. In particular, they will have to refer to one of the papers indicated below and be able to illustrate its content and answer questions on it, possibly reproducing part of the results independently.
2- Students will have to answer questions designed to assess their understanding of the fundamental topics covered in the course (the list will be updated during the course):
-Information theory: Derivation of the Shannon entropy for a discrete probability distribution and related properties. Block entropy, differential entropy and Shannon entropy for a stochastic process. Asymptotic equipartition property (AEP) (Shannon-McMillan-Breiman theorem). Typical sequences. Cross entropy and relative entropy (Kullback-Leibler divergence). Positivity of the Kullback-Leibler divergence: Jensen inequality. Mutual Information. Discussion on the entropy of a language (e.g., English language): block entropy on n-grams and its empirical estimate, Shannon game and proportional gambling. Discussion on symbols codes: non-singular codes, uniquely decodable codes, prefix-free codes, and complete codes. Kraft’s equality and inequality. Bounds on the average length of optimal symbols codes. Huffman code. Algorithmic complexity (Kolmogorov complexity) and its relation with the Shannon entropy. Universal codes and Lempel-Ziv code (LZ77). Demonstration of its optimality. Kac’s lemma on the average recurrence time.
-Inference: Maximum entropy principle. General formulation and examples: probability distributions of microcanonical and canonical ensemble through maximum entropy principle. Exponential, Gaussian and power law distributions through the maximum entropy principle. Inferring the parameters by imposing the constraints. Extracting random variables from a probability distribution. A special case of the Gaussian distribution: the Box-Muller algorithm. Numerical methods: acceptance-rejection method, Markov Chain Monte Carlo Method. Master equation and detailed balance. Young’s argument for convergence to the equilibrium distribution in the case of detailed balance. Metropolis algorithm and Gibbs sampling (heat bath). Maximum likelihood principle. Equivalence of the maximum likelihood principle with the minimization of the Kullback-Leibler divergence between the experimental and the theoretical probability distribution. Bayesian inference. Inference of the success probability of a Bernoulli process.
-Power law: scale invariance, probability distribution and moments computation. Log-log scale for power laws' representation. Logarithmic binning. Representation of a power law distribution through the cumulative function. Frequency-rank distribution and Zipf’s law. Multiplicative processes and the log-normal distribution. Power law distribution through a combination of exponentials. Yule-Simon model. Solution through continuous time approximation. Heaps’ law and its relation with Zipf’s law.
-Graph theory: Graphs representations. Diameter and clustering coefficient of a graph. Random graphs. Diameter in random graphs. Neighbour's degree distribution in random graphs. Clustering coefficient in a random graph. The small world property and the Watts and Strogatz model. Scale-free graphs. The Barabasi-Albert model. Configuration model. Clustering coefficient and diameter in scale-free uncorrelated networks.
Materials for the little essay
Social and Opinion dynamics
“Statistical Physics of Social Dynamics"
C. Castellano, S. Fortunato and V. Loreto,
Rev. Mod. Phys. 81, 591 (2009)
https://journals.aps.org/rmp/abstract/10.1103/RevModPhys.81.591
“Non-equilibrium phase transition in a model of social influence”
C. Castellano, M. Marsili and A. Vespignani,
Phys. Rev. Lett. 85, 3536 (2000)
https://journals.aps.org/prl/abstract/10.1103/PhysRevLett.85.3536
“The dissemination of culture. A model with local convergence and global polarization”
R. Axelrod,
J. Conflict Resolut. 41, 203 (1997).
https://journals.sagepub.com/doi/10.1177/0022002797041002001
"Opinion Dynamics with Disagreement and Modulated Information"
Alina Sîrbu, Vittorio Loreto, Vito D. P. Servedio & Francesca Tria
Journal of Statistical Physics volume 151, pages 218–237 (2013)
https://link.springer.com/article/10.1007/s10955-013-0724-x
"Sharp transition towards shared vocabularies in multi-agent systems"
Andrea Baronchelli, Maddalena Felici, Vittorio Loreto, Emanuele Caglioti and Luc Steels
Journal of Statistical Mechanics: Theory and Experiment, P06014 (2006)
https://iopscience.iop.org/article/10.1088/1742-5468/2006/06/P06014
Innovation dynamics
The dynamics of correlated novelties
F. Tria, V. Loreto, V. D. P. Servedio & S. H. Strogatz
Scientific Reports volume 4, Article number: 5890 (2014)
https://www.nature.com/articles/srep05890
"Dynamics on Expanding Spaces: Modelling the Emergence of Novelties"
Vittorio Loreto, Vito D. P. Servedio, Steven H. Strogatz & Francesca Tria
Chapter on Lecture Notes in Morphogenesis book series
https://link.springer.com/chapter/10.1007/978-3-319-24403-7_5
https://arxiv.org/pdf/1701.00994.pdf
Infosphere
”Quantifying ideological polarization on a network using generalized Euclidean distance.”
Marilena Hohmann, et al.
Science Advances 9(9), (2023)
https://www.science.org/doi/10.1126/sciadv.abq2044
"The COVID-19 social media infodemic"
Matteo Cinelli, Walter Quattrociocchi, Alessandro Galeazzi, Carlo Michele Valensise, Emanuele Brugnoli, Ana Lucia Schmidt, Paola Zola, Fabiana Zollo & Antonio Scala
Scientific Reports volume 10, Article number: 16598 (2020)
https://www.nature.com/articles/s41598-020-73510-5
"The supply and demand of news during COVID-19 and assessment of questionable sources production"
Pietro Gravino, Giulio Prevedello, Martina Galletti & Vittorio Loreto,
Nature Human Behaviour volume 6, pages1069–1078 (2022)
https://www.nature.com/articles/s41562-022-01353-3
"Towards novelty-driven recommender systems"
Pietro Gravino, Bernardo Monechi & Vittorio Loreto
Comptes Rendus Physique, Volume 20, Issue 4, May–June 2019, Pages 371-379
https://www.sciencedirect.com/science/article/pii/S163107051930043X
"Self-induced consensus of Reddit users to characterise the GameStop short squeeze"
Anna Mancini, Antonio Desiderio, Riccardo Di Clemente & Giulio Cimini
Scientific Reports volume 12, Article number: 13780 (2022)
https://www.nature.com/articles/s41598-022-17925-2
"Fast unfolding of communities in large networks"
Vincent D Blondel, Jean-Loup Guillaume, Renaud Lambiotte and Etienne Lefebvre
Journal of Statistical Mechanics: Theory and Experiment, Volume 2008, October 2008
J. Stat. Mech. (2008) P10008
https://iopscience.iop.org/article/10.1088/1742-5468/2008/10/P10008
Social influence and unfollowing accelerate the emergence of echo chambers [PDF]
Sasahara, K., Chen, W., Peng, H. et al.
J Comput Soc Sc 4, 381–402 (2021).
https://doi.org/10.1007/s42001-020-00084-7
Opinion Dynamics with Disagreement and Modulated Information [PDF]
Sîrbu, A., Loreto, V., Servedio, V.D.P. et al.
J Stat Phys 151, 218–237 (2013).
https://doi.org/10.1007/s10955-013-0724-x
Sustainable Cities
"Constructing cities, deconstructing scaling laws" [PDF]
Elsa Arcaute, Erez Hatna, Peter Ferguson, Hyejin Youn, Anders Johansson and Michael Batty
J. R. Soc. Interface 12: 20140745 (2015)
https://royalsocietypublishing.org/doi/10.1098/rsif.2014.0745
A critique on the universal scaling law hypothesis for urban indicators. The authors demonstrate that the definition of a city's boundaries is important for understanding how indicators scale with the city size.
"Optimal design of spatial distribution networks" by Michael Gastner and Mark Newman [PDF]
Phys. Rev. E, 74, 016117 (2006)
https://journals.aps.org/pre/abstract/10.1103/PhysRevE.74.016117
A classical paper that considers the problem of the optimal distribution of facilities given a non-uniform population distribution, such that the average distance from a person’s home to the nearest facility is minimized.
"A universal model for mobility and migration patterns" [PDF]
Filippo Simini, Marta C. González, Amos Maritan & Albert-László Barabási
Nature volume 484, pages96–100 (2012)
https://www.nature.com/articles/nature10856
A classical paper that introduces the radiation model, one of the most used mobility models.
"A universal framework for inclusive 15-minute cities."
Bruno, Matteo, et al.
Nature Cities 1-9, (2024)
https://www.nature.com/articles/s44284-024-00119-4
"Energy laws in human travel behaviour" [PDF]
Robert Kölbl and Dirk Helbing
New Journal of Physics, Volume 5, (2003)
https://iopscience.iop.org/article/10.1088/1367-2630/5/1/348
"Energy laws in human travel behaviour" by R. Kolbl and D. Helbing. Here they extend the idea of the Marchetti constant (a constant travel-time budget) to propose a constant travel-energy budget and from physical arguments derive a universal distribution of travel times.
"Urban economic fitness and complexity from patent data" [PDF]
Straccamore M. et al.
Scientific Reports volume 13, Article number: 3655 (2023)
https://www.nature.com/articles/s41598-023-30649-1
Here, the authors study how patents' technological contents characterise metropolitan areas' development and how innovation is related to GDP per capita.
Suggested reading
"The statistical physics of cities" [PDF]
Marc Barthelemy
Nature Reviews Physics volume 1, pages406–415 (2019)
https://www.nature.com/articles/s42254-019-0054-2
A perspective article with a very short but general review of urban systems from the point of view of statistical physics.
Economic Fitness and Complexity
"A New Metrics for Countries' Fitness and Products' Complexity"
Andrea Tacchella, Matthieu Cristelli, Guido Caldarelli, Andrea Gabrielli & Luciano Pietronero
Scientific Reports volume 2, Article number: 723 (2012)
https://www.nature.com/articles/srep00723 [PDF]
"The different structure of economic ecosystems at the scales of companies and countries"
Dario Laudati, Manuel S. Mariani, Luciano Pietronero & Andrea Zaccaria
https://arxiv.org/abs/2202.01804 [PDF]
"Unfolding the innovation system for the development of countries: coevolution of Science, Technology and Production"
Emanuele Pugliese, Giulio Cimini, Aurelio Patelli, Andrea Zaccaria, Luciano Pietronero & Andrea Gabrielli
Scientific Reports volume 9, Article number: 16440 (2019)
https://www.nature.com/articles/s41598-019-52767-5 [PDF]
"Relatedness in the era of machine learning"
Andrea Tacchella, Andrea Zaccaria, Marco Miccheli, & Luciano Pietronero. Chaos, Solitons & Fractals, 176, 114071 (2023).
www.sciencedirect.com/science/article/pii/S0960077923009724 [PDF]
Artificial Intelligence and Innovation
"Spin-glass models of neural networks"
Daniel J. Amit, Hanoch Gutfreund, H. Sompolinsky
Physical review A, 32, 1007 (1985)
https://www.researchgate.net/publication/283617465_Spin-glass_models_of_neural_networks
“Neural network and physical systems with emergent collective computational abilities”
John J. Hopfield
PNAS, 79 (8) 2554-2558 (1982)
https://www.pnas.org/doi/epdf/10.1073/pnas.79.8.2554
"Autoencoders."
Bank, Dor, Noam Koenigstein, and Raja Giryes.
Machine learning for data science handbook: data mining and knowledge discovery handbook (2023): 353-374.
https://arxiv.org/pdf/2003.05991
"Tutorial on variational autoencoders."
Doersch, Carl.
arXiv preprint arXiv:1606.05908 (2016).
https://arxiv.org/pdf/1606.05908
"Auto-Encoding Variational Bayes"
D. Kingma, and M. Welling.
2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings , (2014)
https://arxiv.org/pdf/1312.6114.pdf
"Generative Adversarial Nets"
Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio
Part of Advances in Neural Information Processing Systems 27 (NIPS 2014)
https://arxiv.org/pdf/1406.2661.pdf
“Nips 2016 tutorial: Generative adversarial networks”
Ian J. Goodfellow
https://arxiv.org/pdf/1701.00160
"Denoising diffusion probabilistic models."
Ho, Jonathan, Ajay Jain, and Pieter Abbeel.
Advances in neural information processing systems 33 (2020): 6840-6851.
"Diffusion Models: Tutorial and Survey."
Ghojogh, Benyamin, and Ali Ghodsi.
(2024).
https://hal.science/hal-04642649/file/Diffusion_tutorial.pdf
AI models collapse when trained on recursively generated data
Shumailov, I., Shumaylov, Z., Zhao, Y. et al.
Nature 631, 755–759 (2024).
https://doi.org/10.1038/s41586-024-07566-y
Dreaming Learning
Alessandro Londei, Matteo benati, Denise Lanzieri, Vittorio Loreto
NeurIPS 2024 workshop on Intrinsically Motivated Open-ended Learning
https://arxiv.org/abs/2410.18156
Dynamics on expanding spaces: modeling the emergence of novelties
Vittorio Loreto, Vito D. P. Servedio, Steven H. Strogatz, Francesca Tria
"Creativity and Universality in Language", 59-83 (2016); Springer International Publishing; isbn:978-3-319-24403-7
https://arxiv.org/abs/1701.00994