Work in Progress

These are projects that I'm currently leading.

If you are interested in my current work, please feel free to reach me.  I will be happy to tell you the details and dive into discussions.

Embarcadero: In-memory pub/sub system for ML pipeline with Disaggregated memory

Recent applications comprise multiple distinct systems that each composed of cluster of compute nodes. For example, a simple streaming application requires 5 systems involved (Kafka, Zoo keeper, Hadoop, Storm, Cassandra). Thus, modern data center suffer tremendous computation, memory, energy waste for communications, so called data center taxes. This project aims to eliminate the data center taxes using disaggregated memory.

This work is in collaboration with VMware Research and Sk Hynix

On the continued Relevance of P2P Networking: Sharing Memory in the Cloud with Peer39

Applications scream for more memory while data center suffers from memory under utilization. Current rigid VM based cloud resource allocation creates this inescapable irony. In this project, we aim to share memory between cloud tenants over the rigid VM boundaries with P2P based solution Peer39. 

Boa: Scheduler-memory manager co-design in Distributed System [under submission to OSDI'24]

This work aims to optimize scheduling tasks in dynamic task graph systems. State-of-the-art task parallel frameworks prioritize parallelism without careful considerations of memory requirements. However, Out-of-Memory can lead to severe consequences that it may halt running tasks. While CPUs can be multiplexed, memory shortage cannot be overcome by systems. Prioritizing both memory and parallelism in dynamic task environment is a difficult challenge. We tackle this problem to better utilize cluster resources.

Log-structured Transactional User Space File System for Persistent Memory  [Will not go under submission with OptaneDC shutdown]

This work aims to match file system behavior to optimal access patterns for PM. I implemented a user space transactional file system that manipulates random and unaligned accesses to PM as aligned and sequential accesses with transaction abstraction. It sits between applications and file systems, accelerates write IO and provide full ACID transaction with repeatable reads.