A decade survey of transfer learning (2010–2020) - IEEE Transactions on Artificial Intelligence 2021

Shuteng Niu, Yongxin Liu, Jian Wang, Houbing Song

Abstract: Transfer learning (TL) has been successfully applied to many real-world problems that traditional machine learning (ML) cannot handle, such as image processing, speech recognition, and natural language processing (NLP). Commonly, TL tends to address three main problems of traditional machine learning: (1) insufficient labeled data, (2) incompatible computation power, and (3) distribution mismatch. In general, TL can be organized into four categories: transductive learning, inductive learning, unsupervised learning, and negative learning. Furthermore, each category can be organized into four learning types: learning on instances, learning on features, learning on parameters, and learning on relations. This article presents a comprehensive survey on TL. In addition, this article presents the state of the art, current trends, applications, and open challenges.

ETT-CKGE: Efficient Task-driven Tokens for Continual Knowledge Graph Embedding - 2024 ECML-PKDD

Lijing Zhu, Wenbo Sun, Li Yang, Yixin Xie, Tiehang Duan, Cui Tao, ..., Shuteng Niu 

Abstract: Continual Knowledge Graph Embedding (CKGE) aims to incorporate new knowledge while retaining prior information, but existing methods face challenges in efficiency and scalability. These stem from suboptimal knowledge preservation using manually designed importance scores and costly graph traversal procedures. We propose ETT-CKGE (Efficient, Task-driven Tokens for Continual Knowledge Graph Embedding), a task-guided method that uses learnable tokens to capture task-relevant signals without explicit node scoring or traversal. These reusable tokens enable efficient, token-masked embedding alignment via simple matrix operations. Experiments on six benchmarks show that ETT-CKGE delivers competitive performance with significantly improved training efficiency and scalability.

Flexible Memory Rotation (FMR): Rotated Representation with Dynamic Regularization to Overcome Catastrophic Forgetting in Continual Knowledge Graph Learning - 2024 IEEE International Conference on Big Data 2024

Lijing Zhu, Dong Hyun Jeon, Wenbo Sun, Li Yang, Yixin Xie, Shuteng Niu 

Abstract: Continual Knowledge Graph Learning (CKGL) is crucial in dynamic applications like recommendation systems and personalized services but faces the challenge of catastrophic forgetting. To address this, we propose Flexible Memory Rotation (FMR), a dual-level regularization technique inspired by human learning, which adapts constraints based on the quality of learned knowledge using the Fisher Information Matrix (FIM). FMR overcomes the limitations of traditional FIM assumptions by rotating the parameter space, leading to significant improvements over state-of-the-art CKGL models, as demonstrated on four benchmark datasets. Comprehensive experiments and ablation studies validate the effectiveness of FMR's components and its overall design.

Spatial-Temporal Graph Data Mining for IoT-enabled Air Mobility Prediction - IEEE Internet of Things Journal 2021

Yushan Jiang, Shuteng Niu, Kai Zhang, Bowen Chen, Chengtao Xu, et al.

Abstract: We propose the first air traffic prediction paradigm based on spatial-temporal graph modeling. We provide a detailed study based on Airline On-Time Performance Data of the year 2016 (provided by Bureau of Transportation Statistics), and build an air transportation network (via different inductive biases) for 285 U.S. domestic airports with temporal features. Our framework based on spatial-temporal graph neural networks provides accurate joint predictions of the number, average delay, and average taxiing time of departure and arrival flights at each airport.