Eliminate Flaky CI Tests with Deterministic Clock & Seeded Random

The hidden source of flakiness

A test that passes locally but fails in CI is usually blamed on “environment differences”. The real culprit is often an implicit dependency on the system clock, on a global random source, or on the order in which asynchronous network calls are scheduled. Those dependencies introduce nondeterminism that the test suite never controls. The result is flaky CI runs, wasted developer time, and a loss of confidence in the test harness.

Locking the clock

Node’s Date.now() and process.hrtime() read the host’s wall‑clock and high‑resolution timer. When a test calls them directly, the exact timestamps become part of the test’s observable behaviour. If a test asserts on a timestamp or uses it to generate a filename, the same test can produce different artefacts on each CI worker.

The fix is to replace the native source with a deterministic stub. SHPBL’s reproducible‑build settings expose a clock module that can be required at the start of the suite:

import { deterministicClock } from '@shpbl/sdk/clock';

deterministicClock.setBase(1609459200000); // 2021‑01‑01T00:00:00Z
global.Date = deterministicClock.Date;
global.performance.now = deterministicClock.now;

All subsequent calls to new Date() or performance.now() return values derived from the base timestamp plus a monotonic increment. The same test run on any runner will see identical timestamps, eliminating the hidden source of variance.

Seeding random generators

The built‑in Math.random() draws from an OS‑provided entropy pool. Its output changes on every invocation, and any code that relies on it – directly or through a library – will behave differently across CI agents. The usual workaround (“set a seed once”) is rarely applied because Math.random() does not expose a seed API.

SHPBL supplies a drop‑in replacement that mirrors the ECMAScript interface while allowing an explicit seed:

import { seededRandom } from '@shpbl/sdk/random';

seededRandom.seed(0xdeadbeef);
global.Math.random = seededRandom.random;

With the seed fixed, the sequence of pseudo‑random numbers is repeatable. Tests that generate UUIDs, shuffle arrays, or simulate probabilistic behaviour become deterministic. The seed value can be stored in a version‑controlled file so that the same seed is used for every CI build.

SHPBL’s reproducible‑build settings

SHPBL’s library is model‑independent: no component calls a model at runtime, so there is no external variance to inherit. The same build artefacts are computed rather than generated, and published checksums act as a root of trust. By importing the clock and random shims from the @shpbl/sdk, you lock both time and entropy to values that are part of the source tree. The harness at [/harness] records a verdict for each artefact – CERTIFIED, PROVISIONAL, INCONCLUSIVE, or FAILED – so you can see immediately whether the test suite remains reproducible after a change. For a deeper look at why model independence matters, see the pillar post [/blog/why-model-independence-is-an-engineering-posture].

What to do on Monday

1. Add @shpbl/sdk to your dev dependencies.
2. Create a test/setup.ts file that imports the deterministic clock and seeded random modules and registers them on the global object.
3. Commit a file seed.txt containing the chosen seed value.
4. Update your CI configuration to run node -r ts-node/register test/setup.ts before the test runner starts.

Run the suite locally, push the change, and watch the CI job produce the same output as your machine. If the harness reports a PROVISIONAL verdict, you have a reproducible build that still needs a correctness check – the next step is to add a certification run. This small change removes three common sources of flakiness and gives you deterministic tests that you can rely on day after day.