A binary file can be perfectly intact and still be difficult to open. Its owner may remember where each record begins but no longer know which serializer, type code or index layout produced it. The missing information is the recipe for interpreting the record's bytes, rather than the record's value alone.

Self-describing storage retains an agreed description alongside the stored data. Sphere 10 Framework's source implementation applies this idea to ObjectSpaces and ObjectStreams. It keeps construction metadata with the stream so supported stores can reconstruct the normal configuration before loading their records.

Contents

Data Needs an Interpretation

Consider a record with a numeric identifier, a title and a price. A serializer needs to know member order and representation. An index needs to know the key layout and tree specification. A container needs to know its allocation structure. The same byte sequence can be read incorrectly if one of those assumptions changes.

A supplied-schema store keeps that recipe outside the stored bytes, usually in application code or a versioned manifest. That is a legitimate design in which the application owns the schema and provides it each time. A self-describing store puts supported parts of the recipe in its metadata region instead. The reader still uses the Framework's ordinary loading path after reconstructing that configuration.

What the Framework Retains

The ObjectSpace stores one serializer factory for its dimensions. Each dimension's ObjectStream describes its item type and relevant index settings. Declarative registrations can retain supported serializer layouts and type codes. This is more informative than writing only an informal version number at the front of a file.

The source API makes that choice explicit:

var builder = new ObjectSpaceBuilder()
    .UseMemory()
    .StoreMetaData();
// Register the dimensions and their indexes before Build().

This configuration fragment selects metadata retention. The complete self-describing store tutorial supplies the registered model and reconstruction example.

Capacity Is Not a Migration Plan

Metadata has an allocated capacity and a used length. Reserving a larger capacity can leave room within that allocation, but it does not promise that every later schema edit can be applied to an existing store. In particular, adding dimensions, changing serializers or changing an index specification is not automatically a supported migration.

The supplied-schema path also remains important for legacy stores. Enabling metadata in a new builder does not retroactively add it to old bytes. Existing files still need their original compatible interpretation unless the application performs a separately supported conversion.

Keep the Remaining Contract Explicit

The reader supplies the original byte order because opening does not detect every encoding. File ownership, application permissions, supported type registrations and the decision to accept a store remain application concerns. A useful acceptance test is to create the store, close the original owner, reconstruct from a raw stream and query a record. Separately test unsupported or mismatched input.

This capability is particularly useful for tooling, portable storage containers and long-lived engineering data. It reduces the amount of construction detail a caller must repeat while keeping the storage model inspectable. Read the self-describing storage guide for the exact source contract, and begin with the complete embedded-store tutorial if you are adopting the Framework today.

Continue the Series

How to Build a Self-Describing Object Store

How to Store Serializer Definitions Alongside Your Data

For the broader Sphere 10 development toolkit, see the Sphere10 Framework overview .