We have a strong track record in the area of graph neural networks, graph learning, and scalable graph algorithms. Graphs or networks are fundamental data structures that represent relationships between objects, known as vertices or nodes, connected by edges or links. These graphs can represent a wide range of relationships in the digital, physical, and biological worlds, including online social networks, user behavior on e-commerce sites, supply chain and financial transaction networks, road systems, neural networks in the brain, protein interactions, chemical structures, and knowledge graphs, among others. Due to the increasing availability of digital data, these graphs can be enormous, ranging from 1 million to 1 billion edges, each with geospatial and temporal semantics. Recent research has proposed various methods for automatically learning these graph structures using neural networks, an area known as Graph Neural Networks (GNNs). For instance, GNNs can be used for financial fraud detection by learning the characteristics of fraudulent activities, as well as for recommender systems on online shopping sites and social networks.  One of our lab's research directions centers on GNNs and extends to various other AI research areas and practical applications. 

EvolveGCN: Evolving Graph Convolutional Networks for Dynamic Graphs, AAAI 2020 (Thirty-Second AAAI Conference on Artificial Intelligence)  
(Paper : https://arxiv.org/abs/1902.10191)
Graph representation learning resurges as a trending research subject owing to the widespread use of deep learning for Euclidean data, which inspire various creative designs of neural networks in the non-Euclidean domain, particularly graphs. With the success of these graph neural networks (GNN) in the static setting, we approach further practical scenarios where the graph dynamically evolves. Existing approaches typically resort to node embeddings and use a recurrent neural network (RNN, broadly speaking) to regulate the embeddings and learn the temporal dynamics. These methods require the knowledge of a node in the full time span (including both training and testing) and are less applicable to the frequent change of the node set. In some extreme scenarios, the node sets at different time steps may completely differ. To resolve this challenge, we propose EvolveGCN, which adapts the graph convolutional network (GCN) model along the temporal dimension without resorting to node embeddings. The proposed approach captures the dynamism of the graph sequence through using an RNN to evolve the GCN parameters. Two architectures are considered for the parameter evolution. We evaluate the proposed approach on tasks including link prediction, edge classification, and node classification. The experimental results indicate a generally higher performance of EvolveGCN compared with related approaches.

Efficient Scaling of Dynamic Graph Neural Networks
Supercomputing 2021

Paper : https://arxiv.org/abs/2109.07893
This work is about distributed algorithms for training dynamic Graph Neural Networks (GNN) on large scale graphs spanning multi-node, multi-GPU systems. To the best of our knowledge, this is the first scaling study on dynamic GNN. We devise mechanisms for reducing the GPU memory usage and identify two execution time bottlenecks: CPU-GPU data transfer; and communication volume. Exploiting properties of dynamic graphs, we design a graph difference-based strategy to significantly reduce the transfer time. We develop a simple, but effective data distribution technique under which the communication volume remains fixed and linear in the input size, for any number of GPUs. Our experiments using billion-size graphs on a system of 128 GPUs shows that: (i) the distribution scheme achieves up to 30x speedup on 128 GPUs; (ii) the graph-difference technique reduces the transfer time by a factor of up to 4.1x and the overall execution time by up to 40%