Preprints & Technical reports
“Convergent Privacy Loss of Noisy-SGD without Convexity and Smoothness,” Eli Chien, Pan Li. [arXiv]
“Privately Learning from Graphs with Applications in Fine-tuning Large Language Models,” Haoteng Yin, Rongzhe Wei, Eli Chien, Pan Li [NeurIPS 2024 SFLLM Workshop, arXiv]
“Linear Classifiers in Mixed Constant Curvature Spaces,” Puoya Tabaghi, Eli Chien, Chao Pan, Olgica Milenkovic. [arXiv]
“Multifaceted roles of cohesin in regulating transcriptional loops,” Minji Kim, Ping Wang, Patricia Clow, I (Eli) Chien, Xiaotao Wang, Jianhao Peng, Haoxi Chai, Xiyuan Liu, Byoungkoo Lee, Chew Yee Ngan, Feng Yue, Olgica Milenkovic, Jeffrey H Chuang, Chia-Lin Wei, Rafael Casellas, Albert Cheng, Yijun Ruan. [bioRxiv]
Highlighted Publications
Machine unlearning theory of Noisy(S)GD
[NeurIPS 2024 (spotlight)] “Langevin Unlearning: A New Perspective of Noisy Gradient Descent for Machine Unlearning”
Eli Chien, Haoyu Wang, Ziang Chen, Pan Li.
[arXiv, ICLR 2024 PrivML workshop (spotlight), TPDP 2024]
[NeurIPS 2024] “Certified Machine Unlearning via Noisy Stochastic Gradient Descent”
Eli Chien, Haoyu Wang, Ziang Chen, Pan Li
[arXiv]
Unlearning via Noisy(S)GD: We propose a new perspective for the machine unlearning problem that unifies the DP learning process and the privacy-certified unlearning process with many algorithmic benefits. Each dataset (D) corresponds to a unique stationary model weight distribution (\nu_D). Learning with Noisy-(S)GD provides an initial privacy loss/DP guarantee (\epsilon_0). Unlearning with the same Noisy-(S)GD will reduce it monotonically, where we may stop until the privacy loss is no larger than \epsilon. First, this argument does not rely on strong convexity, where we leverage Langevin dynamic analysis to prove the desired privacy bound. Second, our approach provides a unique privacy-utility-efficiency trade-off: smaller noise gives larger \epsilon_0, which can later be reduced to \epsilon by unlearning iterations (at the cost of unlearning efficiency). Our second work allows a mini-batch setting and provides the state-of-the-art privacy-utility-efficiency tradeoff for unlearning under the strongly convex assumption. The analysis relies on the shifted divergence analysis instead of Langevin dynamic analysis.
[NeurIPS 2023] “Differentially Private Decoupled Graph Convolutions for Multigranular Topology Protection”
Eli Chien*, Wei-Ning Chen*, Chao Pan*, Pan Li, Ayfer Özgür, Olgica Milenkovic.
[NeurIPS 2023, arXiv, code]
Graph Differential Privacy and Decoupled Graph Convolution: We propose a new DP framework tailored for graph learning that can truly protect user data privacy. Most of the prior works merely focus on ensuring GNN weight is DP. However, the prediction of a node may directly leverage neighbor information at inference. Hence DP GNN weight is insufficient for user privacy protection. Furthermore, prior works focus on the indistinguishability of one edge or node, which can be either too loose or too restrictive as a privacy notion in practice. In reality, the privacy importance of node attributes and graph topology may be different, hence a smooth "interpolation" between edge and node level indistinguishability is desirable. Our unified k-neighbor-level notion of graph data adjacency provides a solution for this need. Surprisingly, our analysis shows that standard graph convolution design has several fundamental weaknesses, including failure to leverage the trade-off between utility and graph topology privacy defined by k. We propose decoupled graph convolution to alleviate all fundamental drawbacks with superior utility in practice.
Machine unlearning on graphs
[ICLR 2023] “Efficient Model Updates for Approximate Unlearning of Graph-Structured Data” (a.k.a Certified Graph Unlearning)
Eli Chien*, Chao Pan*, Olgica Milenkovic.
[ICLR 2023, NeurIPS 2022 GLFrontiers Workshop, arXiv (workshop version), code]
[TheWebConf 2023] “Unlearning graph classifiers with limited data resources”
Chao Pan*, Eli Chien*, Olgica Milenkovic.
[TheWebConf 2023, arXiv, code]
Certified Graph Unlearning: We propose a series of works for graph unlearning with differential privacy types of guarantees. That is, an adversary cannot distinguish model parameters between training on a dataset before and after unlearning request, as their model parameter distributions are approximately the same. Our work is the first to tackle the approximate graph unlearning problem in various settings. We study three different scenarios including node feature, edge and node unlearning, which means one or a few of them are removed according to the unlearning request. Our studied downstream tasks include node (ICLR'23) and graph (TheWebConf'23) classification problems. We show that extending existing machine unlearning to graph is non-trivial in theory and essential in practice. Our methods demonstrate superior privacy-accuracy-complexity tradeoffs compared to retraining from scratch and prior unstructured graph unlearning approaches.
Full Publications (*equal contribution)
My name with ^ indicates the students (in bold font) are mainly advised by me when doing the corresponding project.
Reviews
“Exploring the Opportunities and Challenges of Graph Neural Networks in Electrical Engineering”
Eli Chien, Mufei Li, Anthony Aportela, Kerr Ding, Shuyi Jia, Supriyo Maji, Zhongyuan Zhao, Javier Duarte, Victor Fung, Callie Hao, Yunan Luo, Olgica Milenkovic, David Pan, Santiago Segarra, Pan Li
[Nature Reviews Electrical Engineering 2024]
ML Privacy: Machine Unlearning and Differential Privacy
“Langevin Unlearning: A New Perspective of Noisy Gradient Descent for Machine Unlearning”
Eli Chien, Haoyu Wang, Ziang Chen, Pan Li.
[arXiv, NeurIPS 2024 (spotlight), ICLR 2024 PrivML workshop (spotlight), TPDP 2024]
“Certified Machine Unlearning via Noisy Stochastic Gradient Descent”
Eli Chien, Haoyu Wang, Ziang Chen, Pan Li
[arXiv, NeurIPS 2024]
“Differentially Private Graph Diffusion with Applications in Personalized PageRanks”
Rongzhe Wei, Eli Chien, Pan Li
[arXiv, NeurIPS 2024]
“On the Inherent Privacy Properties of Discrete Denoising Diffusion Models”
Rongzhe Wei, Eleonora Kreacic, Haoyu Peter Wang, Haoteng Yin, Eli Chien, Vamsi K. Potluru, Pan Li
[TMLR]
“Machine Unlearning of Pre-trained Large Language Models”
Jin Yao, Eli Chien, Minxin Du, Xinyao Niu, Tianhao Wang, Zezhou Cheng, Xiang Yue
“Breaking the Trilemma of Privacy, Utility, Efficiency via Controllable Machine Unlearning”
Zheyuan Liu*, Guangyao Dou*, Yijun Tian, Chunhui Zhang, Eli Chien^, Ziwei Zhu^
[TheWebConf 2024, arXiv]
“Differentially Private Decoupled Graph Convolutions for Multigranular Topology Protection”
Eli Chien*, Wei-Ning Chen*, Chao Pan*, Pan Li, Ayfer Özgür, Olgica Milenkovic.
“Efficient Model Updates for Approximate Unlearning of Graph-Structured Data” (a.k.a Certified Graph Unlearning)
Eli Chien*, Chao Pan*, Olgica Milenkovic.
[ICLR 2023, NeurIPS 2022 GLFrontiers Workshop, arXiv (workshop version), code]
“Unlearning Nonlinear Graph Classifiers in the Limited Training Data Regime”
Chao Pan*, Eli Chien*, Olgica Milenkovic.
[TheWebConf 2023, arXiv, code]
General Graph, Hypergraph Neural Networks and Graph ML.
"PINA: Leveraging Side Information in eXtreme Multi-label Classification via Predicted Instance Neighborhood Aggregation,"
Eli Chien, Jiong Zhang, Cho-Jui Hsieh, Jyun-Yu Jiang, Wei-Cheng Chang, Olgica Milenkovic, Hsiang-Fu Yu
“Node Feature Extraction by Self-Supervised Multi-scale Neighborhood Prediction,”
Eli Chien, Wei-Cheng Chang, Cho-Jui Hsieh, Hsiang-Fu Yu, Jiong Zhang, Olgica Milenkovic, Inderjit S Dhillon.
[ICLR2022, arxiv, code, OGB leaderboard]
“You are AllSet: A Multiset Function Framework for Hypergraph Neural Networks,”
Eli Chien*, Jianhao Peng*, Chao Pan*, Olgica Milenkovic.
“Adaptive Universal Generalized PageRank Graph Neural Network,”
Eli Chien*, Jianhao Peng*, Pan Li, Olgica Milenkovic.
[ICLR2021, arXiv, code, slides]
"Multi-MotifGAN (MMGAN): Motif-targeted Graph Generation and Prediction,''
Anuththari Gamage, Eli Chien, Jianhao Peng, Olgica Milenkovic.
[ICASSP 2020, arXiv]
Non-Euclidean Space Learning (Hyperbolic, Product Spaces)
"Federated Classification in Hyperbolic Spaces via Secure Aggregation of Convex Hulls,"
Saurav Prakash, Jin Sima, Chao Pan, Eli Chien, Olgica Milenkovic
[TMLR]
"Provably Accurate and Scalable Linear Classifiers in Hyperbolic Spaces,"
Chao Pan*, Eli Chien*, Puoya Tabaghi,Jianhao Peng, Olgica Milenkovic.
[KAIS 2022, arXiv, ICDM 2021 conference version]
"HyperAid: Denoising in hyperbolic spaces for tree-fitting and hierarchical clustering,"
Eli Chien, Puoya Tabaghi, Olgica Milenkovic.
"Highly Scalable and Provably Accurate Classification in Poincare Balls,"
Eli Chien*, Chao Pan*, Puoya Tabaghi, Olgica Milenkovic.
[ICDM 2021 (Regular), arXiv(Long version), code]
Foundations of Machine Learning over Graphs and Hypergraphs
"Landing Probabilities of Random Walks for Seed-Set Expansion in Hypergraphs,''
Eli Chien*, Pan Li*, Olgica Milenkovic.
[ITW 2021, arXiv(Long version)]
"Active learning in the geometric block model,''
Eli Chien, Antonia Maria Tulino, Jaime Llorca.
[AAAI 2020, arXiv]
"Optimizing Generalized PageRank Methods for Seed-Expansion Community Detection,''
Pan Li*, Eli Chien*, Olgica Milenkovic.
"HS^2: Active Learning over Hypergraphs with pointwise and pairwise queries,''
I (Eli) Chien, Huozhi Zhou, Pan Li.
"On the Minimax Misclassification Ratio of Hypergraph Community Detection,''
I Chien*, Chung-Yi Lin*, I-Hsiang Wang.
[Transactions on Information Theory 2019, arxiv]
"Community detection in hypergraphs: Optimal statistical limit and efficient algorithms,''
I Chien, Chung-Yi Lin, I-Hsiang Wang.
"On the fundamental statistical limit of community detection in random hypergraphs,''
Chung-Yi Lin, I (Eli) Chien, I-Hsiang Wang.
[ISIT 2017, online access]
Statistical and Computational Methods
“Support Estimation with Sampling Artifacts and Errors,”
Eli Chien, Olgica Milenkovic, Angelia Nedich.
[ISIT 2021, arXiv (full)]
"Query K-means Clustering and the Double Dixie Cup Problem,''
I (Eli) Chien, Chao Pan, Olgica Milenkovic.
Biology Applications
"Small-sample estimation of the mutational support and distribution of SARS-CoV-2,"
Vishal Rana, Eli Chien, Jianhao Peng and Olgica Milenkovic
[TCBB 2022, Earlier version: medRxiv]
Others
"Representer Point Selection for Explaining Regularized High-dimensional Models,"
Che-Ping Tsai, Jiong Zhang, Hsiang-Fu Yu, Eli Chien, Cho-Jui Hsieh, Pradeep Kumar Ravikumar
[ICML 2023, arXiv]