Preprints & Technical reports
“Stochastic Gradient Langevin Unlearning,” Eli Chien, Haoyu Wang, Ziang Chen, Pan Li. [arXiv]
“Langevin Unlearning: A New Perspective of Noisy Gradient Descent for Machine Unlearning,” Eli Chien, Haoyu Wang, Ziang Chen, Pan Li. [arXiv, ICLR 2024 PrivML workshop (spotlight)]
“Linear Classifiers in Mixed Constant Curvature Spaces,” Puoya Tabaghi, Eli Chien, Chao Pan, Olgica Milenkovic. [arXiv]
“Multifaceted roles of cohesin in regulating transcriptional loops,” Minji Kim, Ping Wang, Patricia Clow, I (Eli) Chien, Xiaotao Wang, Jianhao Peng, Haoxi Chai, Xiyuan Liu, Byoungkoo Lee, Chew Yee Ngan, Feng Yue, Olgica Milenkovic, Jeffrey H Chuang, Chia-Lin Wei, Rafael Casellas, Albert Cheng, Yijun Ruan. [bioRxiv]
Highlighted Publications
Graph Differential Privacy and Decoupled Graph Convolution: We propose a new DP framework tailored for graph learning that can truly protect user data privacy. Most of the prior works merely focus on ensuring GNN weight is DP. However, the prediction of a node may directly leverage neighbor information at inference. Hence DP GNN weight is insufficient for user privacy protection. Furthermore, prior works focus on the indistinguishability of one edge or node, which can be either too loose or too restrictive as a privacy notion in practice. In reality, the privacy importance of node attributes and graph topology may be different, hence a smooth "interpolation" between edge and node level indistinguishability is desirable. Our unified k-neighbor-level notion of graph data adjacency provides a solution for this need. Surprisingly, our analysis shows that standard graph convolution design has several fundamental weaknesses, including failure to leverage the trade-off between utility and graph topology privacy defined by k. We propose decoupled graph convolution to alleviate all fundamental drawbacks with superior utility in practice.
Machine unlearning on graphs
[ICLR 2023] “Efficient Model Updates for Approximate Unlearning of Graph-Structured Data” (a.k.a Certified Graph Unlearning)
Eli Chien*, Chao Pan*, Olgica Milenkovic.
[ICLR 2023, NeurIPS 2022 GLFrontiers Workshop, arXiv (workshop version), code]
[TheWebConf 2023] “Unlearning Nonlinear Graph Classifiers in the Limited Training Data Regime”
Chao Pan*, Eli Chien*, Olgica Milenkovic.
Certified Graph Unlearning: We propose a series of works for graph unlearning with differential privacy types of guarantees. That is, an adversary cannot distinguish model parameters between training on a dataset before and after unlearning request, as their model parameter distributions are approximately the same. Our work is the first to tackle the approximate graph unlearning problem in various settings. We study three different scenarios including node feature, edge and node unlearning, which means one or a few of them are removed according to the unlearning request. Our studied downstream tasks include node (ICLR'23) and graph (TheWebConf'23) classification problems. We show that extending existing machine unlearning to graph is non-trivial in theory and essential in practice. Our methods demonstrate superior privacy-accuracy-complexity tradeoffs compared to retraining from scratch and prior unstructured graph unlearning approaches.
[ICLR 2022] Node Feature Extraction by Self-Supervised Multi-scale Neighborhood Prediction
Eli Chien, Wei-Cheng Chang, Cho-Jui Hsieh, Hsiang-Fu Yu, Jiong Zhang, Olgica Milenkovic, Inderjit S Dhillon.
[ICLR2022, arxiv, code, OGB leaderboard]
GIANT-XRT: We propose a new method that can greatly improve the performance of general graph learning tasks via better extracting node features from raw data. The key idea is to make use of the beneficial correlation of raw features (such as text) with graphs to guide learning of the encoder (such as Transformer) in a self-supervised learning fashion. We get a huge and uniform improvement against SOTA on OGB benchmark!
AllSet: It is about hypergraph neural networks, where we propose a unified view and connect it with recent develop of deep learning in set functions. We show that there is no need to worry about which hypergraph Laplacian to use in the design of hypergraph neural networks.
Full Publications (*equal contribution)
My name with ^ indicates the students (in bold font) are mainly advised by me when doing the corresponding project.
ML Privacy: Machine Unlearning and Differential Privacy
“Machine Unlearning of Pre-trained Large Language Models”
Jin Yao, Eli Chien, Minxin Du, Xinyao Niu, Tianhao Wang, Zezhou Cheng, Xiang Yue
“Breaking the Trilemma of Privacy, Utility, Efficiency via Controllable Machine Unlearning”
Zheyuan Liu*, Guangyao Dou*, Yijun Tian, Chunhui Zhang, Eli Chien^, Ziwei Zhu^
[TheWebConf 2024, arXiv]
“Differentially Private Decoupled Graph Convolutions for Multigranular Topology Protection”
Eli Chien*, Wei-Ning Chen*, Chao Pan*, Pan Li, Ayfer Özgür, Olgica Milenkovic.
“Efficient Model Updates for Approximate Unlearning of Graph-Structured Data” (a.k.a Certified Graph Unlearning)
Eli Chien*, Chao Pan*, Olgica Milenkovic.
[ICLR 2023, NeurIPS 2022 GLFrontiers Workshop, arXiv (workshop version), code]
“Unlearning Nonlinear Graph Classifiers in the Limited Training Data Regime”
Chao Pan*, Eli Chien*, Olgica Milenkovic.
[TheWebConf 2023, arXiv, code]
General Graph, Hypergraph Neural Networks and Graph ML.
"PINA: Leveraging Side Information in eXtreme Multi-label Classification via Predicted Instance Neighborhood Aggregation,"
Eli Chien, Jiong Zhang, Cho-Jui Hsieh, Jyun-Yu Jiang, Wei-Cheng Chang, Olgica Milenkovic, Hsiang-Fu Yu
“Node Feature Extraction by Self-Supervised Multi-scale Neighborhood Prediction,”
Eli Chien, Wei-Cheng Chang, Cho-Jui Hsieh, Hsiang-Fu Yu, Jiong Zhang, Olgica Milenkovic, Inderjit S Dhillon.
[ICLR2022, arxiv, code, OGB leaderboard]
“You are AllSet: A Multiset Function Framework for Hypergraph Neural Networks,”
Eli Chien*, Jianhao Peng*, Chao Pan*, Olgica Milenkovic.
“Adaptive Universal Generalized PageRank Graph Neural Network,”
Eli Chien*, Jianhao Peng*, Pan Li, Olgica Milenkovic.
[ICLR2021, arXiv, code, slides]
"Multi-MotifGAN (MMGAN): Motif-targeted Graph Generation and Prediction,''
Anuththari Gamage, Eli Chien, Jianhao Peng, Olgica Milenkovic.
[ICASSP 2020, arXiv]
Non-Euclidean Space Learning (Hyperbolic, Product Spaces)
"Federated Classification in Hyperbolic Spaces via Secure Aggregation of Convex Hulls,"
Saurav Prakash, Jin Sima, Chao Pan, Eli Chien, Olgica Milenkovic
[TMLR]
"Provably Accurate and Scalable Linear Classifiers in Hyperbolic Spaces,"
Chao Pan*, Eli Chien*, Puoya Tabaghi,Jianhao Peng, Olgica Milenkovic.
[KAIS 2022, arXiv, ICDM 2021 conference version]
"HyperAid: Denoising in hyperbolic spaces for tree-fitting and hierarchical clustering,"
Eli Chien, Puoya Tabaghi, Olgica Milenkovic.
"Highly Scalable and Provably Accurate Classification in Poincare Balls,"
Eli Chien*, Chao Pan*, Puoya Tabaghi, Olgica Milenkovic.
[ICDM 2021 (Regular), arXiv(Long version), code]
Foundations of Machine Learning over Graphs and Hypergraphs
"Landing Probabilities of Random Walks for Seed-Set Expansion in Hypergraphs,''
Eli Chien*, Pan Li*, Olgica Milenkovic.
[ITW 2021, arXiv(Long version)]
"Active learning in the geometric block model,''
Eli Chien, Antonia Maria Tulino, Jaime Llorca.
[AAAI 2020, arXiv]
"Optimizing Generalized PageRank Methods for Seed-Expansion Community Detection,''
Pan Li*, Eli Chien*, Olgica Milenkovic.
"HS^2: Active Learning over Hypergraphs with pointwise and pairwise queries,''
I (Eli) Chien, Huozhi Zhou, Pan Li.
"On the Minimax Misclassification Ratio of Hypergraph Community Detection,''
I Chien*, Chung-Yi Lin*, I-Hsiang Wang.
[Transactions on Information Theory 2019, arxiv]
"Community detection in hypergraphs: Optimal statistical limit and efficient algorithms,''
I Chien, Chung-Yi Lin, I-Hsiang Wang.
"On the fundamental statistical limit of community detection in random hypergraphs,''
Chung-Yi Lin, I (Eli) Chien, I-Hsiang Wang.
[ISIT 2017, online access]
Statistical and Computational Methods
“Support Estimation with Sampling Artifacts and Errors,”
Eli Chien, Olgica Milenkovic, Angelia Nedich.
[ISIT 2021, arXiv (full)]
"Query K-means Clustering and the Double Dixie Cup Problem,''
I (Eli) Chien, Chao Pan, Olgica Milenkovic.
Biology Applications
"Small-sample estimation of the mutational support and distribution of SARS-CoV-2,"
Vishal Rana, Eli Chien, Jianhao Peng and Olgica Milenkovic
[TCBB 2022, Earlier version: medRxiv]
Others
"Representer Point Selection for Explaining Regularized High-dimensional Models,"
Che-Ping Tsai, Jiong Zhang, Hsiang-Fu Yu, Eli Chien, Cho-Jui Hsieh, Pradeep Kumar Ravikumar
[ICML 2023, arXiv]