Skyhook architecture

Skyhook stores its data partitions in Ceph objects, and we leverage "programmable storage"1^,2, by developing extensions to Ceph object interfaces ("cls") with data processing, indexing, and repartitioning functions that can be performed directly within the storage system. SkyhookDM provides elasticity through both the storage system's inherent ability to dynamically add and remove data storage nodes in the cluster as well as the ability to dynamically repartition data directly within storage. Because our extensions enable storage to perform some processing, adding more nodes to the storage cluster provides increased processing power, enhancing both IO and Compute scalability.

Architectural overview

Below illustrates the high-level architecture of SkyhookDM with a data management application (e.g., PostgreSQL) on top of the storage system (e.g., Ceph), where a subset of the data management (or file access library) operations are expressed as capabilities with the storage system, allowing native operations to be "pushed down" directly into the storage system for execution.

Data management capabilities

Our custom storage extensions include functions for data and metadata management capabilities that enable elasticity and data processing. A key benefit of programmable storage is that functions can be customized for application specific processing. Although we have started with SQL processing, the methods and approach we are developing may also be useful for other applications where in-storage processing is beneficial. For instance, scientific data processing.

SkyhookDM has a 3-layer architecture, with the database on top, the external data access mechanism in the middle, and the storage layer on the bottom providing a clean separation of concerns and trust. We instantiate these layers with PostgreSQL database, PostgreSQL foreign data wrappers, and Ceph distributed object storage respectively. Each of these components are well known, mature, and trusted technologies which Skyhook builds upon and extends. We are actively developing SkyhookDM extensions within Ceph, please see our Github page for more information.

Ceph object characteristics and our SkyhookDM extensions

As depicted in the above figure, Ceph Object Storage Devices (OSDs) store a collection of objects, which in our case, each object contains a database partition. Ceph OSDs have two interesting properties that we develop for our database needs.

First, Ceph objects are classes that are extensible with user defined code. For SkyhookDM, we are developing a tabular data object class that includes functions such as filtering, repartitioning (object split/merge), and data indexing. Repartitioning is more abstract but can help with both elasticity and load balancing. Additionally, more complex user defined functions or transforms are also possible. While these all represent the class of object-based operations, further functionality can be added across objects such as batching and collection level indexing by composing and extending internal Ceph components into application specific services via programmability¹.

Second, each OSD has an internal indexing capability via an embedded Key-Value store (RocksDB) that is local to the device. This typically contains metadata regarding the collection of objects stored on that device. Additional metadata can be added to the KV store and queried at runtime, which can include indexing or other metadata such as counts or statistics related to object contents.

Object classes that are extensible with our user defined functions together with rich metadata storage and querying capability with the embedded KV store enable SkyhookDM to customize and leverage the storage system capabilities to improve scalability for data management .