Contents

Prepare the Example

Use the .NET 10 SDK and Sphere10 Framework 3.1.3. In a new console project, add the core package:

dotnet new console --framework net10.0 --name ClusteredDocumentBundle
cd ClusteredDocumentBundle
dotnet add package Sphere10.Framework --version 3.1.3

Replace the generated source with the named files below, or download the complete example at the end. The package supplies the storage, serialization and collection mechanisms, while the small model and workflow belong to this application. Use a new writable output directory. The sample refuses to overwrite an existing store rather than silently clearing earlier data.

Decide What Each Logical Stream Means

The example uses stream zero for a manifest, stream one for a UTF-8 document and stream two for a three-byte attachment. The JSON manifest records the two payload positions. Those meanings are application policy because the generic container does not know what a document, filename or thumbnail is.

This separation is valuable when a storage format has several independently accessed parts. A record collection can keep its item data, indexes and supporting metadata in related logical streams. A document application can use a manifest to connect typed metadata with larger binary content. The same container mechanism supports those layouts without making the container itself a complete document-management system.

Write the Three Payloads

Save the complete example as Program.cs:

using System;
using System.IO;
using System.Text;
using Sphere10.Framework;

namespace ArticleSample;

public static class Program {
	public static void Main(string[] args) {
		var directory = args.Length == 0 ? Path.Combine(Environment.CurrentDirectory, "sample-data") : args[0];
		Directory.CreateDirectory(directory);
		var path = Path.Combine(directory, "documents.s10");
		if (File.Exists(path))
			throw new InvalidOperationException("Choose a fresh output directory for this container example.");
		using (var file = new FileStream(path, FileMode.CreateNew, FileAccess.ReadWrite)) {
			using var streams = new ClusteredStreams(file, clusterSize: 256, autoLoad: true);
			using (var manifest = streams.Add())
				manifest.Write(Encoding.UTF8.GetBytes("""{"document":1,"attachment":2}"""));
			using (var document = streams.Add())
				document.Write(Encoding.UTF8.GetBytes("Stocktake approved."));
			using (var attachment = streams.Add())
				attachment.Write(new byte[] { 0x10, 0x20, 0x30 });
		}
		using (var file = new FileStream(path, FileMode.Open, FileAccess.ReadWrite)) {
			using var streams = new ClusteredStreams(file, clusterSize: 256, autoLoad: true);
			var manifest = ReadText(streams, 0);
			var document = ReadText(streams, 1);
			using var attachment = streams.OpenRead(2);
			if (streams.Count != 3 || document != "Stocktake approved." || attachment.Length != 3)
				throw new InvalidOperationException("Reopened logical streams differ from the saved container.");
			Console.WriteLine($"Logical streams: {streams.Count}");
			Console.WriteLine($"Manifest: {manifest}");
			Console.WriteLine($"Document: {document}");
			Console.WriteLine($"Attachment bytes: {attachment.Length}");
			Console.WriteLine("File reopen check: passed");
		}
	}

	private static string ReadText(ClusteredStreams streams, long index) {
		using var stream = streams.OpenRead(index);
		using var reader = new StreamReader(stream, Encoding.UTF8, leaveOpen: true);
		return reader.ReadToEnd();
	}
}

Add() returns a writable logical stream. Closing each child before continuing finalizes that stream's descriptor updates and releases its access resources. The outer FileStream remains owned by the caller in the surrounding scope. On reopen, the application supplies the same cluster size and byte-order contract used to create this file.

The UTF-8 encoding is explicit because textual payloads still need a byte contract. The binary attachment contains three literal bytes so the example can demonstrate a binary payload without introducing an unrelated document format. A real application would write the original attachment bytes and retain its actual media type and length in the manifest.

Close the File and Prove Reopen

Run the application with a fresh output directory:

dotnet run -c Release -- ./fresh-document-bundle

The expected output is:

Logical streams: 3
Manifest: {"document":1,"attachment":2}
Document: Stocktake approved.
Attachment bytes: 3
File reopen check: passed

The second block opens the physical file again and checks its logical stream count, document text and attachment length. That proves the demonstration is not simply reading from the first open stream's current position. ReadText opens a selected child and uses a temporary reader while retaining clear ownership of both wrappers.

For a larger payload, consume the child stream incrementally instead of replacing the example with a whole-file read. The application can choose its copy buffer and decoding rules according to the payload. A stream abstraction enables incremental reads, while process-wide memory use and throughput depend on the complete workload.

Keep Container Policy Separate from Application Policy

Clustered streams solve allocation and access within a backing stream. They do not automatically supply filenames, directory permissions, multi-process coordination, an encryption policy or transaction approval. This example uses a normal file-backed stream without wrapping the three child writes in an atomic business transaction.

When atomic acceptance is required, compose the appropriate transactional storage layer or use the file-backed ObjectSpace example, which makes commit and rollback explicit. When integrity is required, select the suitable hash/Merkle mechanism and define where its trusted checkpoint comes from. Those mechanisms can share a storage foundation without becoming interchangeable claims.

Before evolving a bundle format, version its manifest and record the settings needed to reopen old data. Define behavior for unknown payload types and damaged manifests. Keep the original bytes of binary attachments intact unless a transformation is part of the application's explicit contract. The value of a composable container is that those choices remain visible in the application rather than being hidden inside an ad hoc file layout.

Continue Building

The complete sample project includes the project file, source, expected console output and MIT license notice. Code excerpts on this page are supplied under that same license. Keep the package version with your application and retain the sample's observable checks when adapting it because compilation alone does not prove persistence or correct query results.

sample.zip

Sphere10 Framework lets you compose this capability with the rest of your application rather than adopt an unrelated application shell. Start with the working example, change one schema or workload decision at a time, and verify the resulting behavior with your own records.

ObjectSpaces and stream-backed collections build on this foundation. Continue with the embedded catalogue when the payloads should become typed records with declared member indexes.

Useful references:

• ClusteredStreams implementation
• Core storage guide
• Sphere10 Framework on NuGet
• Explore Sphere10 Framework

Continue the Series

How to Build an Embedded Object Database in C#

How to Build a Self-Describing Object Store