Clicking Next should not always mean loading the same catalogue again. If a provider returns a stable dataset, fetching it repeatedly wastes work before the user even changes the query. Filtering and sorting that entire dataset for every page can add another unnecessary pass.

Sphere 10 Pascal Framework separates these responsibilities in TCustomDataSource<T>. Opt-in snapshot caching retains the raw values and a filtered, sorted query view. Page navigation then converts only the selected rows into the grid's table representation. We can demonstrate the distinction with a counted source instead of relying on a vague claim that caching makes everything faster.

Contents

Count the Work Before Optimising It

Our provider supplies the integers one through one hundred and increments FetchCount each time FetchAll runs. The count is deliberately observable. A correct second page should contain different records while leaving the provider count at one. Explicit invalidation should cause the next page request to rebuild the backing snapshot.

The column schema exposes a numeric Value field. A descending sort gives an unmistakable result in which the first ten-record page starts at one hundred, and the second starts at ninety. Checking those values proves that we did not accidentally take a page first and sort it afterwards.

Enable Snapshot Reuse

Set SnapshotCacheEnabled on the source before fetching. Pass the source through IDataSource, which exposes the paging operation, and construct TPageFetchParams from a page index, positive page size, filter criteria and operand. The example uses one sort criterion. Search criteria can share the same query description when needed.

The component remains explicitly owned by the surrounding try/finally. Holding an IDataSource reference is not a replacement for knowing who destroys the component. We clear the interface variable before freeing its owner so the example's lifetime boundary remains visible.

program ArticleSample;
{$IFDEF FPC}{$mode Delphi}{$H+}{$ENDIF}
uses Classes, SysUtils, Variants, Generics.Collections, UCommon.Data;
type
  TNumberSource = class(TCustomDataSource<Integer>)
  public
    FetchCount: Integer;
  protected
    function GetColumns: TDataColumns; override;
    procedure FetchAll(const AContainer: TList<Integer>); override;
    function GetItemField({$IFDEF FPC}constref{$ELSE}const{$ENDIF} AItem: Integer; const ABindingName: String): Variant; override;
  end;

function TNumberSource.GetColumns: TDataColumns;
begin
  Result := TDataColumns.Create(TDataColumn.From('Value', True));
end;

procedure TNumberSource.FetchAll(const AContainer: TList<Integer>);
var I: Integer;
begin
  Inc(FetchCount);
  for I := 1 to 100 do AContainer.Add(I);
end;

function TNumberSource.GetItemField({$IFDEF FPC}constref{$ELSE}const{$ENDIF} AItem: Integer; const ABindingName: String): Variant;
begin
  if ABindingName <> 'Value' then raise EDataTable.Create('Unknown binding');
  Result := AItem;
end;

var
  Source: TNumberSource;
  Provider: IDataSource;
  Params: TPageFetchParams;
  Table: TDataTable;
  Page: TPageFetchResult;
  Filters: TFilterCriteria;
begin
  Source := TNumberSource.Create(nil);
  try
    Source.SnapshotCacheEnabled := True;
    Provider := Source;
    SetLength(Filters, 1);
    Filters[0] := Default(TColumnFilter);
    Filters[0].ColumnName := 'Value';
    Filters[0].Filter := vgfSortable;
    Filters[0].Sort := sdDescending;
    Params := TPageFetchParams.Create(0, 10, Filters, foAnd);
    Page := Provider.FetchPage(Params, Table);
    if Integer(Table.Rows[0].Value) <> 100 then raise Exception.Create('Sort failed');
    Params.PageIndex := 1;
    Page := Provider.FetchPage(Params, Table);
    if Integer(Table.Rows[0].Value) <> 90 then raise Exception.Create('Page failed');
    if Source.FetchCount <> 1 then raise Exception.Create('Snapshot was fetched again');
    WriteLn('First page starts at 100; second page starts at 90; fetches=1');
    Source.InvalidateSnapshot;
    Page := Provider.FetchPage(Params, Table);
    if Source.FetchCount <> 2 then raise Exception.Create('Invalidation failed');
    WriteLn('After invalidation: fetches=2');
    Provider := nil;
  finally
    Source.Free;
  end;
end.

Observe the Two Different Refreshes

First page starts at 100; second page starts at 90; fetches=1
After invalidation: fetches=2

Changing PageIndex from zero to one reuses the current query snapshot. Calling InvalidateSnapshot is different because it marks the backing values as unsuitable for the next fetch. The following request rebuilds them, which is why the provider count becomes two.

This makes invalidation an application decision. After an import, accepted edit or remote revision change, invalidate the source whose data actually changed. Do not disable caching throughout the application merely because one catalogue has a different refresh policy. Equally, do not keep a snapshot indefinitely when the business process requires current values.

Understand What Is Still Being Loaded

The default first fetch still uses FetchAll. Snapshot caching removes repeated work, but does not turn an arbitrary source into a database-native paging engine. A million-row source that cannot reasonably materialise its result needs a different provider implementation, with FetchPage retrieving the requested query page and reporting coherent totals.

Rows containing object references need particular care. Snapshot reuse borrows those objects without deep-copying them or taking over their lifetime. Keep them alive while the cached snapshot and active fetches can use them. Prefer immutable value snapshots when that matches the application.

Choose Useful Acceptance Checks

Test a changed page, a changed sort, a changed search and an explicit invalidation. Include an empty result and a page near the end. A provider can pass a simple first-page test while returning wrong counts or reusing a query snapshot after its criteria changed.

Count conversions as well as fetches if hydration is expensive. The provider count measures how often the backing dataset was obtained, while conversion counts measure how many visible rows were projected into the table representation. They describe different work and should not be presented as interchangeable benchmark results.

The same principle is useful for local reference lists, generated reports and cached service responses. Use the Framework to retain the mechanics of filtering, sorting and paging, while your application defines when a dataset is fresh enough. The data-source implementation and complete public recipe provide the next level of detail.

Get the Framework and Run the Example

Get Sphere 10 Pascal Framework and follow its package instructions . Use the units directly, or build the optional Lazarus .lpk or Delphi runtime package. Pascal uses these source/package workflows. The C# Framework is distributed separately through NuGet.

The attached application sample was compiled and executed with Free Pascal 3.2.0 on Windows against this public source revision , with range and overflow checks. Its Build.ps1 accepts the Framework, compiler, Lazarus and output paths so the example does not need machine-specific paths. Lazarus UI examples require the LCL Win32 widgetset. A successful FPC run does not claim a new Delphi build. Follow the repository's separate Delphi/VCL project and resource instructions there.

Download the complete runnable sample.

Continue the Series

How to Build a Grid That Adapts to Its Window

How to Design a Cache Around the Value’s Lifecycle

How to Keep Old Background Results Out of a New View

For the broader Sphere 10 development toolkit, see the Sphere10 Framework overview .