Journals
(PDF) S. Lee, T. Zhang, S. Prakash, Y. Niu, S. Avestimehr, Embracing Federated Learning: Enabling Weak Client Participation via Partial Model Training, IEEE Transactions on Mobile Computing, 2024 [SCIE Journal (JCR 9.2%, IF: 7.9)]
(PDF) Y. Niu, S. Prakash, S. Kundu, S. Lee, S. Avestimehr, Overcoming Resource Constraints in Federated Learning: Large Models Can Be Trained with only Weak Clients, Transactions on Machine Learning Research, 2023
(PDF) Y. Niu, Z. Fabian, S. Lee, M. Soltanolkotabi, and S. Avestimehr, mL-BFGS: A Momentum-based L-BFGS for Distributed Large-Scale Neural Network Optimization, Transactions on Machine Learning Research, 2023
(PDF) S. Lee, A. Sahu, C. He, and S. Avestimehr, Partial Model Averaging in Federated Learning: Performance Guarantees and Benefits, accepted by Elsevier Neurocomputing, 126647, 2023 [SCIE Journal (JCR 27.9%, IF: 6.0)]
(PDF) S. Lee, C. He, and S. Avestimehr, Achieving Small-Batch Accuracy with Large-Batch Scalability via Hessian-Aware Learning Rate Adjustment, Elsevier Neural Networks, 158, 1-14, 2023 [SCIE Journal (JCR 8.18%, IF: 9.657)]
(PDF) S. Lee, J. Jeon, and H. Lee, Probing Oxygen Vacancy Distribution in Oxide Heterostructure by Deep Learning-based Spectral Analysis of Current Noise, Applied Surface Science, 604, 154599, 2022 [SCIE Journal (JCR 2.5%, IF: 7.392)]
(PDF) S. Lee, J. Jeon, K. Eom, J. Jeon, K. Eom, C. Jeong, Y. Yang, J. Park, C. Eom, and H. Lee, Variance‐aware weight quantization of multi‐level resistive switching devices based on Pt/LaAlO3/SrTiO3 heterostructures, Scientific Reports, 12, 1-10, 2022 [SCIE Journal (JCR 25%, IF: 4.997)]
(PDF) S. Lee, Q. Kang, R. Al-Bahrani, A. Agrawal, A. Choudhary, and W. Liao, Improving Scalability of Parallel CNN Training by Adaptively Adjusting Parameter Update Frequency, Journal of Parallel and Distributed Computing, 2022 [SCIE Journal (JCR 15.91%, IF: 4.542)]
(PDF) S. Lee, K. Hou, K. Wang, S. Sehrish, M. Paterno, A. Agrawal, A. Choudhary, Q. Koziol, R. Ross, J. Kowalkowski, and W. Liao, A Case Study on Parallel HDF5 Dataset Concatenation for High-Energy Physics Data Analysis, Elsevier Parallel Computing, 2022 [SCIE Journal (JCR Q3, IF: 0.983)]
(PDF) S. Madireddy, J. Park, S. Lee, P. Balaprakash, S. Yoo, W. Liao, C. Hauck, M. Laiu, and R. Archibald, In Situ Compression Artifact Removal in Scientific Data Using Deep Transfer Learning, Machine Learning: Science and Technology, 2020 [SCIE Journal (JCR 24.55%, IF: 6.013)]
(PDF) Q. Kang, S. Lee, K. Hou, R. Ross, A. Agrawal, A. Choudhary, and W. Liao, Improving MPI Collective I/O Performance with Intra-node Request Aggregation, IEEE Transactions on Parallel and Distributed Systems Journal, 2020 [SCIE Journal (JCR 24.09%, IF: 3.757)]
Conference Proceedings
(PDF) S. Lee, Layer-Wise Adaptive Gradient Norm Penalizing Method for Efficient and Accurate Deep Learning, KDD, 2024 [BK-21 IF: 4]
(PDF) S. Lee, T. Zhang, and S. Avestimehr, Layer-wise Adaptive Model Aggregation for Scalable Federated Learning, AAAI, 2023 [Oral presentation] [BK-21 IF: 4]
(PDF) T. Zhang, T. Feng, S. Alam, S. Lee, M. Zhang, S. S. Narayanan, and S. Avestimehr, FedAudio: A Federated Learning Benchmark for Audio Tasks, IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2023
(PDF) K. Wang, S. Lee, J. Balewski, A. Sim, P. Nugent, A. Agrawal, A. Choudhary, K. Wu, and W. Liao, Using Multi-resolution Data to Accelerate Neural Network Training in Scientific Applications, International Symposium on Cluster, Cloud, and Internet Computing (CCGrid), 2022 [BK-21 IF: 1]
(PDF) K. Hou, Q. Kang, S. Lee, A. Agrawal, A. Choudhary, and W. Liao, Supporting Data Compression in PnetCDF, IEEE International Conference on BigData, 2021
(PDF) S. Lee, Q. Kang, K. Wang, J. Balewski, A. Sim, K. Wu, A. Agrawal, A. Choudhary, P. Nugent, and W. Liao, Asynchronous I/O Strategy for Large-Scale Deep Learning Applications, International Conference on High-Performance Computing, Data, and Analytics (HiPC), 2021 [BK-21 IF: 1]
(PDF) R. Al-bahrani, D. Jha, Q. Kang, S. Lee, Z. Yang, W. Liao, A. Agrawal, and A. Choudhary, SIGRNN: Synthetic minority Instances Generation in imbalanced datasets using a Recurrent Neural Network, International Conference on Pattern Recognition Applications and Methods, 2021
(PDF) S. Lee, A. Q. Kang, Agrawal, A. Choudhary, and W. Liao, Communication-Efficient Local Stochastic Gradient Descent for Scalable Deep Learning, IEEE International Conference on BigData, 2020
(PDF) Q. Kang, R. Ross, R. Latham, S. Lee, A. Agrawal, A. Choudhary, and W. Liao, Improving all-to-many personalized communication in two-phase I/O, The International Conference for High-Performance Computing, Networking, Storage, and Analysis (SC), 2020 [BK-21 IF: 3]
(PDF) Q. Kang, A. Sim, P. Nugent, S. Lee, W. Liao, A., A. Choudhary, and K. Wu, Predicting Resource Requirement in Intermediate Palomar Transient Factory Workflow, International Symposium on Cluster, Cloud, and Internet Computing (CCGrid), 2020 [BK-21 IF: 1]
(PDF) S. Lee, Q. Kang, S. Madireddy, P. Balaprakash, A. Agrawal, A. Choudhary, R. Archibald, and W. Liao, Improving Scalability of Parallel CNN Training by Adjusting Mini-Batch Size at Run-Time, IEEE International Conference on BigData, 2019
(PDF) S. Lee, D. Jha, A. Agrawal, A. Choudhary, and W. Liao, Parallelizing Deep Convolutional Neural Network Training by Exploiting the Overlapping of Computation and Communication (Best Paper Finalist), International Conference on High-Performance Computing, Data, and Analytics (HiPC), 2017 [BK-21 IF: 1]
(PDF) D. Palsetia, W. Hendrix, S. Lee, A. Agrawal, W. Liao, and A. Choudhary, Parallel Community Detection Algorithm Using a Data Partitioning Strategy with Pairwise Subdomain Duplication, International Supercomputing Conference (ISC), 2016
Workshop Papers
(PDF) T. Zhang, L. Gao, S. Lee, M. Zhang, and S. Avestimehr, TimelyFL: Heterogeneity-aware Asynchronous Federated Learning with Adaptive Partial Training, International Workshop on Federated Learning for Computer Vision held in conjunction with CVPR (FedVision), 2023
(PDF) Y. Niu, S. Prakash, S. Kundo, S. Lee, and Salman Avestimehr, Federated Learning of Large Model at the Edge via Principal Sub-Model Training, International Workshop on Federated Learning held in conjunction with NeurIPS (FL-NeurIPS), 2022
(PDF) S. Lee, A. Sahu, C. He, S. Avestimehr, Partial Model Averaging in Federated Learning: Performance Guarantees and Benefits, International Workshop on Trustable, Verifiable and Auditable Federated Learning in conjunction with AAAI, 2022
(PDF) Y. Niu, Z. Fabian, S. Lee, M. Soltanolkotabi, and S. Avestimehr, SLIM-QN: A Stochastic, Light, and Momentumized Quasi-Newton Optimizer for Deep Neural Networks, International Conference on Machine Learning (ICML) workshop, 2021
(PDF) S. Lee, A. Agrawal, P. Balaprakash, A. Choudhary, and W. Liao, Communication-Efficient Parallelization Strategy for Deep Convolutional Neural Network Training, In Workshop on Machine Learning in High-Performance Computing Environments, held in conjunction with International Conference for High-Performance Computing, Networking, Storage, and Analysis (SC), 2018
(PDF) S. Lee, W. Liao, A. Agrawal, N. Hardavellas, and A. Choudhary, Evaluation of K-Means Data Clustering Algorithm on Intel Xeon Phi, In Workshop on Advances in Software and Hardware for Big Data to Knowledge Discovery, held in conjunction with the IEEE International Conference on Bigdata, 2016
Preprints
Z. Tang, X. Chu, R. Y. Ran, S. Lee, S. Shi, Y. Zhang, Y. Wang, A. Q. Liang, S. Avestimehr, C. He, FedML Parrot: A Scalable Federated Learning System via Heterogeneity-Aware Scheduling on Sequential and Hierarchical Training. arXiv 2023
C. He, Z. Yang, E. Mushtaq, S. Lee, M. Soltanolkotabi, and S. Avestimehr, SSFL: Tackling Label Deficiency in Federated Learning via Personalized Self-Supervision, arXiv 2021
Thesis
(Ph.D. thesis) Scalable Parallelization Strategy for Large-Scale Deep Learning, Sunwoo Lee, Ph.D. Thesis, Northwestern University, 2020
(M.S. thesis) Component-based Design and Performance Analysis for Multiprocessor Embedded Real-Time Software, Sunwoo Lee, M.S. Thesis, Hanyang University, 2009