A long-running process often needs to report the count, range, mean and deviation of a measurement such as latency or throughput. Computing those figures from a list means keeping every reading for as long as the summary is wanted, even though nothing ever looks at an individual reading again.

The Statistics class in Sphere10 Framework is a running accumulator for this case. Each reading updates a handful of fields and is then discarded, and the summary values are computed from those fields on demand. This article covers how to feed it, which values it exposes, how it behaves when empty or reset, and what a summary without history cannot tell you.

1. A first look

Construct the accumulator with a history capacity of zero, call AddDatum for each reading, and read the summary from its properties.

var statistics = new Statistics(0);
foreach (var reading in ReadValues(cycles))
	if (double.IsFinite(reading) && reading >= 0)
		statistics.AddDatum(reading);

Console.WriteLine($"Readings accepted: {statistics.SampleCount}");
Console.WriteLine($"Range: {statistics.Minimum} to {statistics.Maximum}");
Console.WriteLine($"Mean: {statistics.Mean.ToString("F2", CultureInfo.InvariantCulture)}");
Console.WriteLine($"Population standard deviation: {statistics.PopulationStandardDeviation.ToString("F2", CultureInfo.InvariantCulture)}");

The loop keeps none of the readings it passes to AddDatum, and the four values printed afterwards come from the accumulator alone.

2. How the accumulator works

stateDiagram-v2
    direction TB
    [*] --> Empty
    Empty --> Accumulating: AddDatum
    Accumulating --> Accumulating: AddDatum
    Accumulating --> Empty: Reset
    Empty: Empty
    Empty: SampleCount is 0
    Empty: Mean is NaN
    Accumulating: Accumulating
    Accumulating: count, sums, min, max
    Accumulating: Mean = Sum / SampleCount

3. Feed readings to the accumulator

The constructor argument in the first look is the capacity of an optional trend history. Zero states that no readings are kept, which is also what the parameterless constructor does.

AddDatum accepts any double and does not validate it, so the filter belongs to the caller, as it did in the first look's loop. A NaN or an infinity would enter the sums and turn every later mean into NaN. A reading that could not be taken should be skipped rather than recorded as zero, while a measured zero is a legitimate value. Keep one accumulator for each metric, and keep every reading in it in the same unit.

Two further methods change the accumulated state.

• AddDatum(datum, numTimes) records the same value several times in one call.
• RemoveDatum(datum) subtracts a value from the count and the sums. It is available only when the history capacity is zero, and it cannot restore an earlier minimum or maximum.

4. Read the summary

The summary is a set of read-only properties, four of which the first look printed.

The class also exposes GeometricMean, HarmonicMean and the variation coefficients, which are maintained from a running product and a running sum of reciprocals.

An empty accumulator needs a guard before it is displayed. Mean and the deviations return NaN when SampleCount is zero, and Minimum and Maximum are zero, which is their initial value and not an observation. Check SampleCount > 0 and show the summary as unavailable otherwise.

5. Reset and ownership

Reset() clears every field and starts a new interval, so the summary always describes the readings accepted since the last reset.

statistics.Reset();
Require(statistics.SampleCount == 0 && double.IsNaN(statistics.Mean), "Reset did not clear the accumulator.");
Console.WriteLine($"Count after reset: {statistics.SampleCount}");

The object that owns the measurement should own the accumulator. A statistic held by a screen is lost when the screen is recreated, whereas one held by the service that produces the readings survives for as long as the service does.

6. Keep the summary's meaning precise

A summary without history answers a fixed set of questions. The following need the readings themselves, or a different sampling design.

• Percentiles and medians. They cannot be derived from a count and two sums.
• Windows over time. There is no way to ask for the last five minutes after the fact. Reset at the start of each interval instead.
• Time weighting. The mean is an average over readings. When readings arrive at irregular intervals, it is not an average over time.

SampleCount is a 32-bit unsigned integer, and the sums are doubles with finite precision. A producer that runs for a very long time should reset on a schedule, and readings should be expressed in a unit that keeps their magnitudes moderate.

7. Get the package and run the sample

Add the package to a .NET 10 project.

dotnet add package Sphere10.Framework

Statistics ships in the Sphere10.Framework NuGet package. Download the sample project. It targets .NET 10 and Sphere10 Framework. The program feeds 62,000 generated readings to a Statistics(0) instance, skipping the unusable ones. The readings are the integers 10 to 40 in equal numbers, so the program checks each summary value against its known result and throws if one differs.

dotnet run -c Release

Readings accepted: 62000
Range: 10 to 40
Mean: 25.00
Population standard deviation: 8.94
Count after reset: 0

• Live statistics guide
• Statistics accumulator API

Download the complete NuGet sample project. The archive contains the project, its model files and run instructions. Keep the pinned package version when reproducing the example.

The Sphere10 Framework overview describes the rest of the toolkit.

Author: Herman Schoenfeld