My research interests are in LLM systems, cloud computing, and mobile systems. I received my PhD in Computer Science through the Zhiyuan Honored PhD Program at Shanghai Jiao Tong University in 2025, advised by Prof. Haibing Guan. I was also a visiting scholar at Purdue University, advised by Prof. Y. Charlie Hu.

My recent research focuses on:

1. LLM inference systems on edge devices and servers, with a focus on energy-efficiency: CORE (MLSys 2026), SegMo (CVPR 2026)

2. I/O virtualization systems for public clouds, with a focus on efficiency, scalability, and manageability: HD-IOV (EuroSys 2024), Un-IOV (IEEE TC 2024), M3U (OSDI 2026)

3. Cloud systems for AI workloads, with a focus on serverless computing and in-network offloading: P4KVS (SIGMOD 2025), vCrypto (INFOCOM 2024), QKPT (IEEE TDSC 2021)

I received my BS in Computer Science and Technology through the IEEE Pilot Class at Shanghai Jiao Tong University in 2019, also advised by Prof. Haibing Guan.

Selected Publications

See Google Scholar for the complete list.

[1]. Inference First, Training Underneath: Layer-Granular LoRA Co-Serving with LayerLoom

• • Zongpu Zhang, Yishu Yin, Xuehai Qian

  • arXiv Preprint, 2026

[2]. Rethinking DVFS for Mobile LLMs: Unified Energy-Aware Scheduling with CORE

• • Zongpu Zhang*, Pranab Dash*, Y. Charlie Hu, Qiang Xu, Jian Li, Haibing Guan (* equal contribution)

  • The Ninth Annual Conference on Machine Learning and Systems (MLSys), 2026.

[3]. P4KVS: A Role-Replica Separation Offloading Method to Achieve In-Network Consistency for KV Stores Based on P4 Switches

• • Haojuan Li, Zongpu Zhang, Chenzhen Ye, Ruohan Tang, Jian Li, Haibing Guan, Qiaoling Wang, and Pengpeng Zhou

  • Proceedings of the ACM on Management of Data (SIGMOD), 2025.

[4]. HD-IOV: SW-HW Co-designed I/O Virtualization with Scalability and Flexibility for Hyper-Density Cloud

• • Zongpu Zhang, Jiangtao Chen, Banghao Ying, Yahui Cao, Lingyu Liu, Jian Li, Xin Zeng, Junyuan Wang, Weigang Li, Haibing Guan

  • The Nineteenth European Conference on Computer Systems (EuroSys), 2024.

[5]. Un-IOV: Achieving Bare-Metal Level I/O Virtualization Performance for Cloud Usage With Migratability, Scalability and Transparency

• • Zongpu Zhang, Chenbo Xia, Cunming Liang, Jian Li, Chen Yu, Tiwei Bie, Roberts Martin, Daly Dan, Xiao Wang, Yong Liu, Haibing Guan

  • IEEE Transactions on Computers (TC), 2024.

[6]. vCrypto: a Unified Para-Virtualization Framework for Heterogeneous Cryptographic Resources

• • Shuo Shi, Chao Zhang, Zongpu Zhang, Hubin Zhang, Xin Zeng, Weigang Li, Junyuan Wang, Xiantao Zhang, Yibin Shen, Jian Li, Haibing Guan

  • IEEE Conference on Computer Communications (INFOCOM), 2024.

[7]. QKPT: Securing Your Private Keys in Cloud With Performance, Scalability and Transparency

• • Zongpu Zhang, Hubin Zhang, Junyuan Wang, Xiaokang Hu, Jian Li, Wenqian Yu, Ping Yu, Weigang Li, Bo Cui, Guodong Zhu, Kapil Sood, Brian Will, Haibing Guan

  • IEEE Transactions on Dependable and Secure Computing (TDSC), 2021.

Under Review

TACT; LayerWeave; TIDE; PAD-MoE; SlideGraph; STAIR.

Collaborative Work

• VLM serving: SegMo (CVPR 2026).

• Virtualization and serverless systems: M3U (OSDI 2026), Ephemera (TACO 2025), SMore (TPDS 2025).

• LLM algorithm and application: MentalGPT (TAFFC 2026).

Earlier Work

Video object tracking and segmentation: TWS-OVS (MM 2018), HDDNN (TBD 2018), ACGAN (MM 2019), Object-guided (ICCV 2019).

Professional Experience

1. Visiting Scholar, Purdue University  2024-2026

   • Elmore Family School of Electrical and Computer Engineering. West Lafayette, IN, USA

   • Advisor: Prof. Y. Charlie Hu

   • AI-based system design for mobile LLM inferencing and distributed LLM serving.

2. Visiting Scholar, Purdue University  2023-2024

   • Department of Computer Science. West Lafayette, IN, USA

   • Advisor: Prof. Xuehai Qian

   • Optimizing distributed LLM serving systems with forward- and backward-fused pipeline parallelism.

3. Research Intern, Intel  2020-2023

   • Network Platforms Group (NPG). Shanghai, China

   • Advisor: Junyuan Wang and Xin Zeng

   • Contributed to the Intel QAT driver and QAT engine for Intel KPT and Intel S-IOV technologies.

4. Research Intern, SenseTime Research  Summer 2019

   • ADAS Team. Shanghai, China

   • Advisor: Guangliang Cheng 

   • Explored neural architecture search for visual recognition in autonomous driving.

5. Research Intern, Intel  Summer 2018

   • Data Center Group (DCG). Shanghai, China

   • Advisor: Jason Dai (Intel Fellow)

   • Prototyped a computer vision system using Intel BigDL for a distributed AI agent that plays FIFA. Media 

Professional Services

Journal Reviewer

• Computer Networks (International Journal of Computer and Telecommunications Networking)

• IEEE Transactions on Sustainable Computing

• IEEE Transactions on Mobile Computing

• Transactions on Machine Learning Research

Conference Program Committee

• MLSys 2026 Artifact Evaluation

Conference Reviewer

• INFOCOM 2027

Talks

• Feb 3, 2026, "Model-baesd RL for System Research"

  • Location: IntelliSys, Stevens Institute of Technology

• Feb 5, 2024, "I/O Virtualization for In-network Accelerators in Cloud -- Security, Efficiency, and Scalability"

  • Location: DSNL, Purdue University

• Jan 29, 2024, "Scalable I/O Virtualization for In-network Accelerators -- An example with Intel QAT accelerators".

  • Location: VT Security & Intelligence Lab, Virginia Tech

Awards

Student Awards

• 2019, Zhiyuan Honored Ph.D. Program (Top 2%), SJTU

• 2019, Excellent Bachelor’s Thesis Award (Top 1%), SJTU, Media (in zh-CN) 

• 2019, Zhang Xu Scholarship, SJTU

• 2019, Yang Yuanqing Scholarship, SJTU, Media (in zh-CN) 

• 2018, SenseTime Scholarship, Sensetime Inc.

• 2016,17,18, Academic Excellence Scholarship (three times), SJTU

Misc

• 2022, Winner (1st place) of the 2022 Fall NVIDIA DPU Hackathon in China, NVIDIA, Media (in zh-CN)