This project proposal explores the interplay of fluctuations and stochastic dynamics with singularity and geometry. This foundational scheme is addressed in terms of three principal workpackages: (A) fluctuations in interacting particle systems with degenerate parabolicity (B) stochastic gradient methods in machine learning in interaction with geometric structures, and (C) stochastic dynamics in singular stochastic partial differential equations. The workpackages are closely interconnected through shared mathematical principles and challenges such as large deviations, central limit theorems and modified equations. The project aims to analyse central limit theorems, large deviation principles, and stochastic flows on curved geometries. A key methodological focus is on mathematical modeling across different scaling limits. The project aims to achieve fundamental theoretical insights as well as algorithmic advancements in stochastic methods.