Geometric Deep Learning and Beyond

“Today a reader, tomorrow a leader.”

― Margaret Fuller

What is Geometric Deep Learning?

Geometric Deep Learning (GDL) is a paradigm in machine learning that extends traditional deep learning techniques to handle data with complex geometric structures, such as graphs, meshes, and manifolds. While conventional deep learning models, like convolutional neural networks (CNNs), operate primarily on Euclidean data (e.g., images or sequences), GDL is designed to work with data that lies on non-Euclidean domains.

The core idea of GDL is to generalize neural network architectures to exploit the symmetries, structure, and properties inherent in the data. In many real-world problems, data is naturally represented as graphs (such as social networks, molecules, or transportation systems) or as manifolds (like 3D shapes or surfaces). Geometric Deep Learning incorporates principles from group theory, differential geometry, and graph theory to build models that can efficiently learn from these non-Euclidean data types.

Applications of GDL include molecular modeling, where graph-based representations of molecules allow for better prediction of chemical properties, social network analysis, and computer vision tasks like 3D object recognition. Geometric Deep Learning has led to significant advances in fields where the underlying data structure is complex and cannot be fully captured by traditional grid-based models.

"5Gs" in Geometric Deep Learning

Grids: Refers to data with regular, grid-like structures, such as images and time-series. Traditional deep learning models like CNNs operate effectively on grid data by taking advantage of translational symmetry.
Groups: Groups in this context represent mathematical symmetries that can be found in data. Many modern machine learning approaches incorporate group symmetries to build models that are invariant to transformations (like rotations, translations, or scaling). Convolutional networks are often interpreted as exploiting the group structure of translation invariance in grid data.
Graphs: This category focuses on learning from data represented as graphs, where nodes represent entities and edges represent relationships or interactions. Graph Neural Networks (GNNs) are the primary model in this area, and they have been applied to a variety of domains, including social network analysis, protein folding, and molecular chemistry.
Geodesics: Geodesics are the shortest paths between two points on curved surfaces (manifolds). This aspect of GDL deals with learning from data that lies on manifolds, such as 3D objects or other curved geometrical structures. Techniques that process such data, like mesh convolutional networks, allow for understanding data with curvature or other non-Euclidean properties.
Gauges: Gauges refer to a broader class of geometric data that includes gauge fields and connections, often found in physics. In this context, gauge equivariant neural networks are being developed to deal with problems where local geometric symmetries (like those found in physics) must be preserved by the model.

Best reading

Michael M. Bronstein, Joan Bruna, Taco Cohen, Petar Veličković, Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges

Intuitive Tutorials

Starter: A Gentle Introduction to Graph Neural Networks

Tutorial: NetworkX

Review: Graph neural networks: A review of methods and applications

Stanford CS224W: Machine Learning with Graphs: A full Stanford course covering GNNs and geometric learning. Link to Course Materials

Essential Papers

Foundational Papers:

Bronstein et al., 2017, "Geometric Deep Learning: Going beyond Euclidean data"
This is the seminal work that introduced the term "Geometric Deep Learning" and set the foundation for its development.
Link to Paper
Kipf & Welling, 2017, "Semi-Supervised Classification with Graph Convolutional Networks"
The introduction of Graph Convolutional Networks (GCNs), an essential GDL approach.
Link to Paper
Scarselli et al., 2009, "The Graph Neural Network Model"
A precursor to modern GNNs, this paper formalized learning from graph structures.
Link to Paper

Advanced and Recent Developments:

Velickovic et al., 2018, "Graph Attention Networks"
A paper that introduced GATs, leveraging attention mechanisms in graph learning.
Link to Paper
Defferrard et al., 2016, "Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering"
Introduction of spectral methods for GDL.
Link to Paper
Bouritsas et al., 2022, "Improving Graph Neural Networks with Learnable Structural and Positional Representations"
Link to Paper

Code and Frameworks

Key Frameworks:

PyTorch Geometric (PyG)
A fast, easy-to-use framework for building GNNs and geometric models.
GitHub Repository
DGL (Deep Graph Library)
Another powerful library for scalable GNN models in multiple frameworks.
GitHub Repository

Page updated

Report abuse