Tutorial Date/Time/Location

• Data & Time: December 3rd (Sunday), 2:30-5:30

• Location: Room 6, Shanghai World Trade Mall Co., Ltd. Shang, China

Tutorial Description

Time series data is ubiquitous in various real-world applications, such as Artificial Intelligence for IT Operations (AIOps) in cloud computing, AI-powered Business Intelligence (BI) in E-commerce, Artificial Intelligence of Things (AIoT), etc. In real-world scenarios, time series often exhibit complex patterns with trend, seasonality, outlier, and noise. In addition, as more time series data are collected, handling the massive amount of data efficiently is crucial in many applications. These significant challenges exist in various tasks, such as forecasting, anomaly detection, and fault cause localization. Therefore, designing effective, efficient, and explainable models for different tasks, which are robust enough to address the aforementioned challenging patterns and noise in real-world applications, is of great theoretical and practical interest. 

This tutorial summarizes state-of-the-art algorithms of robust time series analysis through an interdisciplinary approach ranging from robust statistics, signal processing, optimization, and the most recent deep learning-based methods. We will not only introduce the principle of time series algorithms but also provide insights into applying various techniques from multiple disciplines effectively in practical, real-world industrial applications.

Specifically, we organize the tutorial in a bottom-up framework. Firstly, We present preliminaries from different disciplines, including robust statistics, signal processing, optimization, deep learning, and explainable AI (XAI). Then, we identify and discuss those most-frequently processing blocks in robust time series analysis, including periodicity detection, trend filtering, seasonal-trend decomposition, and time series similarity. These blocks can be integrated into different time series tasks as general built-in blocks. Lastly, we discuss recent advances in multiple time series tasks, including forecasting, autoscaling, anomaly detection, and fault cause localization, as well as practical lessons learned from large-scale time series applications in industrial scenarios.

Tutorial Outline and Materials

Tutorial slides [slides]

1. Preliminaries

   • Real-world Challenges and Motivations for Robustness

   • Robust Statistics: Robust Regression, M-estimators

   • Signal Processing: Fourier Transform, Wavelet Transform

   • Optimization Algorithms: Alternating Direction Method of Multipliers (ADMM), Majorize-Minimization (MM) Algorithm

   • Deep Learning: RNN, CNN, GNN, SSL, Transformer, Data Augmentation, and LLMs

   • Explainable Artificial Intelligence (XAI): GAM, LIME, SHAP, Integrated Gradient, PINN

2. Robust Time Series Processing Blocks

   • Time Series Periodicity Detection

   • Time Series Trend Filtering

   • Time Series Seasonal-Trend Decomposition

   • Time Series Similarity

3. Robust Time Series Applications and Practices

   • Forecasting: Statistical Models, Tree Models, Deep Ensemble, Transformer, XAI and LLMs for Forecasting

   • Autoscaling (from Forecasting to Decision-Making): Query Modeling, Scaling Decision

   • Anomaly Detection: Decomposition Model, Deep State Space Model, Graph Model, Transformer

   • Fault Cause Localization (from Anomaly Detection to Localization): Rule Set Learning, Root Cause Analysis

Further Reading: AI for Time Series (AI4TS) Papers, Tutorials, and Surveys [GitHub link]

Key References Published by Lecturers

1. [NeurIPS'23] Yifan Zhang, Qingsong Wen, Xue Wang, Weiqi Chen, Liang Sun, Zhang Zhang, Liang Wang, Rong Jin, Tieniu Tan, "OneNet: Enhancing Time Series Forecasting Models under Concept Drift by Online Ensembling," in Proc. 37th Annual Conference on Neural Information Processing Systems (NeurIPS 2023), New Orleans, LA, Dec. 2023. 

2. [NeurIPS'23] Tian Zhou, PeiSong Niu, Xue Wang, Liang Sun, Rong Jin, "One Fits All: Power General Time Series Analysis by Pretrained LM," in Proc. 37th Annual Conference on Neural Information Processing Systems (NeurIPS 2023), New Orleans, LA, Dec. 2023. 

3. [NeurIPS'23] Minqi Jiang, Chaochuan Hou, Ao Zheng, Songqiao Han, Hailiang Huang, Qingsong Wen, Xiyang Hu, Yue Zhao, "ADGym: Design Choices for Deep Anomaly Detection," in Proc. 37th Annual Conference on Neural Information Processing Systems (NeurIPS 2023), New Orleans, LA, Dec. 2023. 

4. [NeurIPS'22] Chenxiao Yang, Qitian Wu, Qingsong Wen, Zhiqiang Zhou, Liang Sun, Junchi Yan, "Towards Out-of-Distribution Sequential Event Prediction: A Causal Treatment," in Proc. 36th Annual Conference on Neural Information Processing Systems (NeurIPS 2022), New Orleans, LA, Dec. 2022. [arXiv]

5. [NeurIPS'22] Tian Zhou, Ziqing Ma, Xue Wang, Qingsong Wen, Liang Sun, Tao Yao, Wotao Yin, Rong Jin, "FiLM: Frequency improved Legendre Memory Model for Long-term Time Series Forecasting", in Proc. 36th Conference on Neural Information Processing Systems (NeurIPS 2022), New Orleans, LA, Dec. 2022. [arXiv] [code] (NeurIPS Oral)

6. [NeurIPS'21] Fan Yang, Kai He, Linxiao Yang, Hongxia Du, Jingbang Yang, Bo Yang, Liang Sun, "Learning interpretable decision rule sets: a submodular optimization approach," in Proc. 35th Annual Conference on Neural Information Processing Systems (NeurIPS 2021), Virtual Conference, Dec. 2022. [paper]

7. [ICML'22] Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, Rong Jin, "FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting," in Proc. 39th International Conference on Machine Learning (ICML 2022), Baltimore, Maryland, July 17-23, 2022. (Acceptance Rate=1117/5630=19.8%) [arXiv] [Oral] [code] 

8. [KDD'23] Yiyuan Yang, Chaoli Zhang, Tian Zhou, Qingsong Wen*, Liang Sun, "DCdetector: Dual Attention Contrastive Representation Learning for Time Series Anomaly Detection", in Proc. 29th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD 2023), Long Beach, CA, Aug. 2023. (Research Track, Acceptance Rate 313/1416=22.10%) [arXiv] [slides] [code]

9. [KDD'22] Weiqi Chen, Wenwei Wang, Bingqing Peng, Qingsong Wen, Tian Zhou, Liang Sun, "Learning to Rotate: Quaternion Transformer for Complicated Periodical Time Series Forecasting", in Proc. 28th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD 2022), Washington DC, Aug. 2022. (Research Track, Acceptance Rate 254/1695=14.99%) [paper] [Oral] [code]

10. [KDD'22] Qingsong Wen, Linxiao Yang, Tian Zhou, Liang Sun, "Robust Time Series Analysis and Applications: An Industrial Perspective", in Proc. 28th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD 2022), Washington DC, Aug. 2022. [paper] [Website] 

11. [KDD'20] Qingsong Wen, Zhe Zhang, Yan Li, Liang Sun, "Fast RobustSTL: Efficient and Robust Seasonal-Trend Decomposition for Time Series with Complex Patterns," in Proc. 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD 2020), San Diego, CA, Aug. 2020. (Research Track, Acceptance Rate 216/1279=16.9%) [paper] [Oral] 

12. [SIGMOD'21] Qingsong Wen, Kai He, Liang Sun, Yingying Zhang, Min Ke, Huan Xu, "RobustPeriod: Time-Frequency Mining for Robust Multiple Periodicity Detection," in Proc. ACM SIGMOD International Conference on Management of Data (SIGMOD 2021), Xi'an, China, Jun. 2021. (Research Track) [paper] [arXiv] [Oral] 

13. [VLDB'23] Zhicheng Pan, Yihang Wang, Yingying Zhang, Sean Bin Yang, Peng Chen, Yunyao Cheng, Chenjuan Guo, Qingsong Wen, Xiduo Tian, Yunliang Dou, Zhiqiang Zhou, Chengcheng Yang, Aoying Zhou, Bin Yang, "MagicScaler: Uncertainty-aware, Predictive Autoscaling", in the 49th International Conference on Very Large Data Bases (VLDB 2023), Vancouver, Canada, Aug. 2023.

14. [ICDE'22] Huajie Qian, Qingsong Wen, Liang Sun, Jing Gu, Qiulin Niu, Zhimin Tang, "RobustScaler: QoS-Aware Autoscaling for Complex Workloads," in Proc. IEEE 38th International Conference on Data Engineering (ICDE 2022), Kuala Lumpur, Malaysia, May 2022. [arXiv] [Oral], Media Coverage: [Mo4Tech] [Alicloudnative] [Zhihu] [1024sou]

15. [AAAI'23] Yuxuan Liang, Yutong Xia, Songyu Ke, Yiwei Wang, Qingsong Wen, Junbo Zhang, Yu Zheng, Roger Zimmermann, "AirFormer: Predicting Nationwide Air Quality in China with Transformers", in Proc. of The 37th AAAI Conference on Artificial Intelligence (AAAI 2023), Washington DC, Feb. 2023. [arXiv] [code] 

16. [AAAI'23] Zhiqiang Zhou, Chaoli Zhang, Lingna Ma, Jing Gu, Huajie Qian, Qingsong Wen, Liang Sun, Peng Li, Zhimin Tang, "AHPA: Adaptive Horizontal Pod Autoscaling Systems on Alibaba Cloud Container Service for Kubernetes", in Proc. AAAI Conference on Artificial Intelligence and 35th Annual Conference on Innovative Applications of Artificial Intelligence (AAAI/IAAI 2023), Washington DC, Feb. 2023. [arXiv] (AAAI/IAAI 2023 Deployed Application Award) 

17. [AAAI'23] Zhaoyang Zhu, Weiqi Chen, Rui Xia, Tian Zhou, Peisong Niu, Bingqing Peng, Wenwei Wang, Hengbo Liu, Ziqing Ma, Qingsong Wen, Liang Sun, "eForecaster: Unifying Electricity Forecasting with Robust, Flexible, and Explainable Machine Learning Algorithms", in Proc. AAAI Conference on Artificial Intelligence and 35th Annual Conference on Innovative Applications of Artificial Intelligence (AAAI/IAAI 2023), Washington DC, Feb. 2023.  [paper] (AAAI/IAAI 2023 Deployed Application Award)

18. [AAAI'19] Qingsong Wen, Jingkun Gao, Xiaomin Song, Liang Sun, Huan Xu, Shenghuo Zhu, "RobustSTL: A Robust Seasonal-Trend Decomposition Algorithm for Long Time Series," in Proc. 33th AAAI Conference on Artificial Intelligence (AAAI 2019), Honolulu, Hawaii, Jan. 2019. (Acceptance Rate 1150/7095=16.2%) [arXiv] [paper] [Oral] [3rd-party Code], Media Coverage: [Alibaba Tech]

19. [[IJCAI'23] Qingsong Wen, Tian Zhou, Chaoli Zhang, Weiqi Chen, Ziqing Ma, Junchi Yan, Liang Sun, "Transformers in Time Series: A Survey", in the 32nd International Joint Conference on Artificial Intelligence (IJCAI 2023), Macao, S.A.R, Aug. 2023. [arXiv] [Website]

20. [IJCAI'21]  Qingsong Wen, Liang Sun, Fan Yang, Xiaomin Song, Jingkun Gao, Xue Wang, Huan Xu, "Time Series Data Augmentation for Deep Learning: A Survey," in the 30th International Joint Conference on Artificial Intelligence (IJCAI 2021), Montreal, Canada, Aug. 2021. [paper] [arXiv] [Oral].  Selected by Paper Digest into Most Influential IJCAI Papers (Version: 2022-02), Rank 1st (1/600+ IJCAI’21 papers).

21. [IJCAI'19] Qingsong Wen, Jingkun Gao, Xiaomin Song, Liang Sun, Jian Tan, "RobustTrend: A Huber Loss with a Combined First and Second Order Difference Regularization for Time Series Trend Filtering," in Proc. 28th International Joint Conference on Artificial Intelligence (IJCAI 2019), Macao, China, Aug. 2019. (Acceptance Rate 850/4752=17.9%) [arXiv] [paper] [Oral] 

22. [CIKM'22] Chaoli Zhang, Tian Zhou, Qingsong Wen*, Liang Sun, "TFAD: A Decomposition Time Series Anomaly Detection Architecture with Time-Frequency Analysis,” in Proc. 31st ACM International Conference on Information and Knowledge Management (CIKM 2022), Atlanta, GA, Oct. 2022. [arXiv] [paper] [Oral] [code]

23. [CIKM'22] Xiaomin Song, Qingsong Wen, and Liang Sun, "Robust Time Series Dissimilarity Measure for Outlier Detection and Periodicity Detection,” in Proc. 31st ACM International Conference on Information and Knowledge Management (CIKM 2022), Atlanta, GA, Oct. 2022. [arXiv]

24. [CIKM'21] Yingying Zhang, Zhengxiong Guan, Huajie Qian, Leili Xu, Hengbo Liu, Qingsong Wen, Liang Sun, Junwei Jiang, Lunting Fan, Min Ke, "CloudRCA: A Root Cause Analysis Framework for Cloud Computing Platforms," in Proc. 30th ACM International Conference on Information and Knowledge Management (CIKM 2021), Queensland, Australia, Nov. 2021. (Applied Research Track,  Oral, Top 20/290=6.9%) [paper] [arXiv] [Oral] 

25. [ICASSP'23] Qingsong Wen, Linxiao Yang, Liang Sun, "Robust Dominant Periodicity Detection for Time Series with Missing Data", in Proc. IEEE 48th International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2023), Rhodes Island, Greece, June 2023. [arXiv]

26. [ICASSP'23] Hengbo Liu, Ziqing Ma, Linxiao Yang, Tian Zhou, Rui Xia, Yi Wang, Qingsong Wen, Liang Sun, "SaDI: A Self-Adaptive Decomposed Interpretable Framework for Electricity Load Forecasting under Extreme Events", in Proc. IEEE 48th International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2023), Rhodes Island, Greece, June 2023. [arXiv]

27. [ICASSP'22] Chaoli Zhang*, Zhiqiang Zhou*, Yingying Zhang*, Linxiao Yang*, Kai He*, Qingsong Wen*, Liang Sun* (*Equally Contributed), "NetRCA: An Effective Network Fault Cause Localization Algorithm," in Proc. IEEE 47th International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2022), Singapore, May 2022. [arXiv] [Oral] [link] (ICASSP‘22 AIOps Challenge, First Place (1/382))

28. [ICASSP'21] Qingyang Xu, Qingsong Wen, Liang Sun, "Two-Stage Framework for Seasonal Time Series Forecasting," in Proc. IEEE 46th International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2021), Toronto, Canada, Jun. 2021. [arXiv] [Oral] 

29. [ICASSP'21] Linxiao Yang, Qingsong Wen, Bo Yang, Liang Sun, "A Robust and Efficient Multi-Scale Seasonal-Trend Decomposition," in Proc. IEEE 46th International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2021), Toronto, Canada, Jun. 2021. [paper] [Oral] 

30. [ICASSP'20] Qingsong Wen, Zhengzhi Ma, Liang Sun, "On Robust Variance Filtering and Change of Variance Detection," in Proc. IEEE 45th International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2020), Barcelona, Spain, May 2020. [paper] [Oral]

Short Bio of Lecturers

• Qingsong Wen is currently a Staff Engineer / Team Leader with Alibaba DAMO Academy-Decision Intelligence Lab at Bellevue, WA, USA, working in the areas of intelligent time series analysis, data-driven intelligence decisions, machine learning, and signal processing. Before that, he worked at Futurewei, Qualcomm, and Marvell in the areas of big data and signal processing, and received his M.S. and Ph.D. degrees in Electrical and Computer Engineering from Georgia Institute of Technology, Atlanta, USA. He has published over 60 top-ranked journal and conference papers, received AAAI/IAAI 2023 Deployed Application Award, and won the First Place in 2022 ICASSP Grand Challenge Competition (AIOps in Networks). He is an Associate Editor for Neurocomputing, Guest Editor for IEEE Internet of Things Journal, and Guest Editor for Applied Energy. He is an IEEE senior member, and a member of the IEEE Machine Learning for Signal Processing Technical Committee. He also serves as Organizer/Co-Chair of the Workshop on AI for Time Series at AAAI'24, KDD'23, IJCAI'23, and ICDM'23. 

• Linxiao Yang is currently a Senior Engineer with Alibaba DAMO Academy, working in the fields of time series analysis, pattern mining, and interpretable machine learning. He received the B.S. degree in electrical engineering from Southwest Jiaotong University, Chengdu, China, and the Ph.D. degree in Communication and Information Engineering from the University of Electronic of Science and Technology of China, Chengdu, China. He won the First Place in 2022 ICASSP Grand Challenge (AIOps in Networks) Competition. His research interests include data-driven decision making, efficient optimization methods, interpretable machine learning, and signal processing.

• Tian Zhou is currently a Senior Algorithm Engineer in Alibaba DAMO Lab, working mainly on time series forecasting and sequence modeling. He has worked on all aspects ranging from practical model designing to theoretical foundations. Prior to joining Alibaba, Tian obtained a BS in chemistry from Tsinghua University, an MS in Statistics from Rutgers-New Brunswick, an MS in Machine Learning from Rutgers-New Brunswick, and a PhD from Rutgers-New Brunswick, focusing on computational chemistry for organometalic catalysts.

• Weiqi Chen is currently an engineer at DAMO Academy Decision Intelligence Lab, Alibaba Group. He received B.S. from Xi'an Jiaotong University and M.S. from Zhejiang University in computer Science an Technology. His research interest is on time series forecasting and anomaly detection with related applications, e.g., electricity forecasting and weather prediction.

• Bingqing Peng is currently a senior engineer at DAMO Academy Decision Intelligence Lab, Alibaba Group. She received B.S. from Nanjing University with major in Mathematics and M.S in Statistics from University of Washington Seattle. Her research is focus on time series analysis and machine learning, with industrial applications in renewable energy forecasting and weather forecasting.

• Liang Sun is currently a Senior Staff Engineer / Engineering Director at DAMO Academy, Alibaba Group, Bellevue, USA. He received B.S.(2003) from Nanjing University, and Ph.D.(2011) from Arizona State University, both in computer science. Dr. Sun has over 60 publications including 2 books in the fields of machine learning and data mining. His work on dimensionality reduction won the KDD 2010 Best Research Paper Award Honorable Mention, and won the Second Place in KDD Cup 2012 Track 2 Competition. He also won the First Place in 2022 ICASSP Grand Challenge (AIOps in Networks) Competition. At Alibaba Group, he is working on temporal data mining, including time series anomaly detection, forecasting, and their applications.

Related Tutorial:

KDD 2022 Tutorial: Robust Time Series Analysis and Applications: An Industrial Perspective