Efficient Distributed and Federated Learning

Efficient distributed learning is crucial for handling the growing complexity and scale of modern datasets, especially those represented as graphs. Graph Neural Networks (GNNs) have shown remarkable success in fields like chip design, recommender systems, and drug discovery. However, training GNNs on large-scale graph datasets, such as social networks with billions of nodes and edges, presents significant challenges due to memory and bandwidth limitations.

Our approach for distributed GNN training includes techniques such as graph coarsening, allocation optimization, and refining. Some of our results in this line of work were introduced as AdaGL framework. AdaGL encapsulates a novel coarsening strategy that transforms the original graph into a concise representation. Our coarsening algorithm iteratively identifies a set of core nodes, then leverages tree-based search to aggregate information from node neighborhoods within core nodes. Notably, the proposed coarsening method scales linearly with the number of graph nodes, thereby enabling us to optimize allocation for billion-scale graphs.