Research

AI / LLM Systems

Our group is dedicated to building fast and efficient AI systems. We are currently focusing on building fast and efficient LLM serving systems, which optimize resource management on commodity GPUs and accelerators (e.g., NPUs and PIMs). We are interested in multi-GPUs parallelism, disaggregating prefill and decoding, speculative decoding, test-time scaling, and reasoning.
Keywords: LLM, attention, parallelism

Operating Systems

Due to the challenge of scaling DRAM density, a new class of memory (e.g., CXL Memory Expander) has received a lot of attention to bridge the performance gap between DRAM and SSD. In this study, we revisit the design and implementation of memory management in state-of-the-art Linux for achieving high-performance, and also explore the advantages of leveraging hardware support for dealing with massive data.
Keywords: memory management, memory tiering, scheduling

Computer Architecture

We are interested in designing xPUs (CPUs, GPUs, and NPUs) for datacenter and cloud computing. In this study, we perform an in-depth performance analysis through cycle-accurate simulators (e.g., gem5 or GPGPU-sim) or leveraging HW performance counters on real systems, and then explore architectural solutions through those simulators or SW emulation.
Keywords: architecture, cycle-level simulation

Report abuse