Dynamic Data Objects is a library that makes it easy to model complex structures of data in memory. Then, you can easily serialize to/from it using a variety of data serialization techniques including: CBOR, JSON, MessagePack, BSON, UBJSON, Smile, and Amazon's ION (binary style only)
There are two main co-dependent aspects to Dynamic Data Objects that are at play here.
- In-Memory Object Model: DataObjects is a library of code that provides a few classes that can be used to model a dynamic hierarchical structure of data in memory. In concept, it's similar to a "variant" or "oleVariant" data type that's available in some programming languages. A dataObject can hold a value in memory and that value would be one of many possible data types such as a signed integer, a double floating point number, a string, etc. It also has a few different types of collections like arrays or maps (key-value collections). DataObjects supports two key-value collections types: Sparse Arrays (a key-value map where the key is an integer), and a Frame (a key-value map where the key is a string. These are sometimes called dictionaries, maps, etc.)
- Serialization: The primary purpose here is to take a DataObject that may hold a complex hierarchy of data in memory and serialize it into a stream of bytes that can be sent over a network, saved to a file, etc. It could be used for server-to-server communications, client-to-server communication, data or preference persistence, etc. Ultimately, however, the DataObject can be serialized to a stream and another DataObject elsewhere can read that stream and built an exact in-memory clone of the original. In most cases, this is done without any data loss or data translation. However, some of the serializations supported, such as JSON, do a very poor job of modeling data, so when using certain serializations, there must be some translations taking place. For example, a GUID data value will be serialized in JSON as a string because JSON doesn't have native support for the GUID data type.
Of course, there are many trade-offs that need to be dealt with such as memory consumption, CPU processing and stream size, etc. that are at conflict with one another with any library like this. There are also other considerations such as complexity, dependencies, incompatibilities, code size, portability, etc. that have to be considered. So, I first defined a few design principles that I will follow as a guide when coding the library. See the other page listing the design principles.
This is a Delphi library. I am making sure it works correctly for Win32 and Win64 right now. Eventually I'll make sure it works for Android, IOS and Linux too but not until I think the first version is solid enough to start putting time into testing with these other targets.
The only thing you need to do use the core of data objects is to include "dataObjects" in a uses clause.
For each serialization you want to use, just include the source code for it in a uses clause.
That's it. No dlls needed. No other dependencies.
I do have a few additional utilities included in the source such as a Save-As dialog that understands the registry of serializers that are included in a project.
I seriously doubt this code will work in other pascal based environments such as Lazarus/free pascal, but maybe.
Since I use Delphi most of the time, my primary use for DataObjects will be within applications written in Delphi. The most common use cases that I use this library for are:
- syncing structured data models between servers
- Pulling data from MongoDB using BSON,
- Publishing HTTP services to browser javascript clients using JSON
- Persisting complex preferences for an application instead of using an *.ini file or the registry.
- Syncing structured data models and calling remote procedure calls between a client and a server.
- Saving logs that have a bit more complex data than just text. That way I can mine the logs later if I want to extract good data and do analytics on something.