A cache is easy to start with a dictionary and hard to finish. Once it holds objects, something has to decide when an entry is too old, when the source behind it has changed, how much the cache may hold, and who frees a value that is replaced or evicted.

TCacheBase<TKey, TValue> in Sphere 10 Pascal Framework treats those decisions as policies chosen at construction. A derived cache supplies only how to fetch a value, how large it is and, optionally, how to tell that it is stale. This article walks through the lifecycle of a cached value and the policy that governs each stage.

1. A first look

Reading through the cache is an indexed lookup by key. The sample's ReadText function takes a cache of TEntry objects keyed by string and returns the text of one entry.

function ReadText(ACache: TEntryCache; const AKey: String): String;
var
  Entry: TEntry;
begin
  Entry := ACache[AKey];
  Result := Entry.Text;
end;

There is no try..finally and no Free, because the cache owns the object under idpFree, the dispose policy that a later section describes. The reference is borrowed. Use it immediately and copy out any values that must survive a later cache operation, since a replacement, an eviction or the destruction of the cache frees the object.

2. The lifecycle of a cached value

stateDiagram-v2
    direction TB
    [*] --> Cached: first read, Fetch
    Cached --> Cached: read, hit
    Cached --> Due: expired or stale
    Cached --> Due: Invalidate
    Due --> Cached: next read, Fetch
    Cached --> Removed: reap, Remove, Flush
    Due --> Removed: reap, Remove, Flush
    Removed --> [*]: dispose policy

3. Give the cache its fetching and freshness rules

A cache class passes its policies to the inherited constructor and overrides up to three methods. The sample caches objects of a class named TEntry, keyed by string and loaded from a TSource object that the cache borrows.

constructor TEntryCache.Create(ASource: TSource);
begin
  inherited Create(nil, idpFree, crpOldest, epSinceFetchedTime, nvpCacheNormally, 4096, TTimeSpan.FromMinutes(1), 0.1, svpCheckStaleOnDemand);
  FSource := ASource;
end;

function TEntryCache.Fetch(const AKey: String): TEntry;
begin
  Result := FSource.Load(AKey);
end;

function TEntryCache.EstimateSize(const AValue: TEntry): SizeInt;
begin
  Result := AValue.InstanceSize + (Length(AValue.Key) + Length(AValue.Text)) * SizeOf(Char);
end;

function TEntryCache.CheckStaleness(const AKey: String; AItem: TCachedItem<TEntry>): Boolean;
begin
  Result := AItem.Value.Revision <> FSource.Revision;
end;

The constructor arguments after the owner map onto the lifecycle as follows.

• Dispose policy (idpFree). The cache frees a value when its entry is replaced or removed. idpNone leaves values with the caller.
• Reap policy (crpOldest). Which entries are evicted first when space is needed. Other choices include crpLeastUsed, crpLargest, crpIdle and crpNone.
• Expiration policy (epSinceFetchedTime). Age is measured from the fetch. epSinceLastAccessedTime gives a sliding lifetime and epNone disables expiry.
• Null value policy (nvpCacheNormally). Whether a nil result is cached, returned without caching, or raised as an error.
• Maximum capacity (4096). The budget, in the units that EstimateSize returns.
• Expiration duration (one minute). The age limit used by the expiration policy.
• Reap level (0.1). The fraction of capacity the cache aims to release when it has to make room.
• Stale value policy (svpCheckStaleOnDemand). Calls CheckStaleness on each read. The default, svpAssumeNeverStale, never calls it.

EstimateSize is a budgeting figure chosen by the cache author. It is compared with the maximum capacity and is not a measurement of process memory.

When a subclass is more than the job needs, TActionCache<TKey, TValue> takes the fetch and size functions as callbacks instead.

This retained screenshot shows the repository's Delphi/VCL cache demonstration. The output in this article is from the separate executable sample.

4. Read through the cache

classDiagram
    direction TB
    class TCacheBase {
      +Get()
      +Invalidate()
      #Fetch()*
      #EstimateSize()*
    }
    class TCachedItem~TValue~ {
      +Value
      +FetchedOn
      +LastAccessedOn
      +Size
    }
    class TEntryCache {
      #Fetch()
      #EstimateSize()
      #CheckStaleness()
    }
    TCacheBase <|-- TEntryCache
    TCacheBase *-- TCachedItem~TValue~

5. Expiry, staleness and invalidation

These three mechanisms answer different questions, and all of them take effect on the next read of the key.

A staleness check runs on every hit, so it should be cheap. The sample compares an integer revision stored in the value with the source's current revision, which avoids loading anything to find out that nothing changed.

      Cache.Invalidate('A');

Invalidate marks one entry without removing it. Remove deletes an entry at once, and Flush empties the cache. The ItemFetching, ItemFetched, ItemRemoving and ItemRemoved events report each transition to one handler.

6. Limits

The sample prints the number of fetches and of live value objects after each step.

Two reads: fetches=1 live=1
After source change: fetches=2 live=1
After Invalidate: fetches=3 live=1
After cache destroyed: live=0

The cache unit also contains bulk-fetch and preloaded cache classes that share these policies.

7. Get the Framework and run the example

Get Sphere 10 Pascal Framework and follow its package instructions . Use the units directly, or build the optional Lazarus or Delphi runtime package.

The sample was compiled and run on Windows with Free Pascal 3.2.0, against source revision 2cc6558 . Its Build.ps1 accepts the Framework, compiler, Lazarus and output paths.

Download the complete runnable sample.

For the broader Sphere 10 development toolkit, see the Sphere10 Framework overview .

Author: Herman Schoenfeld