Work in Progress
Work in Progress
These are projects that I'm currently leading.
If you are interested in my current work, please feel free to reach me. I will be happy to tell you the details and dive into discussions.
Publish-subscribe messaging systems are at the core of today’s data-driven clouds. These systems decouple the entities that produce and consume messages, providing a powerful communication abstraction that, in theory, scales to any number of producers and consumers. In practice, however, existing systems trade off high performance and efficiency for desirable properties such as durability and total ordering.
In this paper, we attribute this trade-off to the fact that current pub-sub systems are, like most of today’s distributed systems, built on top of point-to-point message-passing networks that limit the performance and flexibility of common tasks in pub-sub systems. To that end, we propose Embarcadero, a rack-scale publish-subscribe system that uses physically disaggregated memory, an upcoming hardware technology, to break this trade-off. Embarcadero provides high performance, strong correctness properties, and also allows for near-instantaneous elasticity and churn of broker nodes.
Byte-addressable disaggregated memory will soon be in data centers, aiming to increase memory size and bandwidth, improve memory utilization, and permit efficient sharing of data between compute nodes. Current proposals for disaggregated memory plan to provide hardware-based cache coherence similarly to local memory. However, we find that the cost of coherence becomes untenable as disaggregated memory scales to many nodes. We thus introduce a new model of coherence called federated coherence, which leverages the existing hardware coherence mechanisms within each node, without incurring the coherence costs across nodes. We explain how to use federated coherence to build strongly consistent application libraries that developers can use without concern for the weaker notion of coherence.
This work aims to optimize scheduling tasks in dynamic task graph systems. State-of-the-art task parallel frameworks prioritize parallelism without careful considerations of memory requirements. However, Out-of-Memory can lead to severe consequences that it may halt running tasks. While CPUs can be multiplexed, memory shortage cannot be overcome by systems. Prioritizing both memory and parallelism in dynamic task environment is a difficult challenge. We tackle this problem to better utilize cluster resources.
Current application trends put immense pressure on main memory. Many applications scream for more memory. Ironically, memory is severely underutilized at the data center level. We argue that fine-grained memory sharing among cloud applications can improve memory utilization, throughput, and cost. In this paper, we propose to borrow relevant ideas from peer-to-peer networks to bring elasticity to the rigid memory infrastructure of today’s data centers. Instead of sharing content, peers can share their free memory resources with other peers
Even though it is killed with the death of Intel Optane DC, this study poses an interesting view of using Persistent Memory with a new primitive, transactional file system!
This work aims to match file system behavior to optimal access patterns for PM. I implemented a user space transactional file system that manipulates random and unaligned accesses to PM as aligned and sequential accesses with transaction abstraction. It sits between applications and file systems, accelerates write IO and provide full ACID transaction with repeatable reads.