Deterministic Network Mocks to Eliminate Flaky Integration Tests

The flaky test problem

Your CI pipeline fails intermittently. One run passes, the next one blows up on a call to an external payment gateway. The stack trace shows a timeout, a 502, or a different JSON shape. The same test code, the same source revision, but a different outcome. This is the classic flaky‑test symptom caused by nondeterministic network behaviour. Random latency, transient server errors, and time‑driven retries all inject variance into the test run. When the variance reaches the CI surface, confidence evaporates. Developers start ignoring test failures, and the build becomes a waste of time.

The root cause is not the test framework. It is the reliance on live network interactions during integration testing. Those interactions are outside your control, and they inherit the external service’s timing and error profile. The result is a test suite that cannot be reproduced reliably, and a build process that cannot be declared deterministic.

Deterministic network mocking

Replace the live call with a deterministic mock layer. The mock must provide two guarantees:

1. Fixed response payloads – the same JSON object for a given request, regardless of when the test runs.
2. Predictable timing – either immediate resolution or a configurable, reproducible delay.

Implement the mock as a thin wrapper around the HTTP client used by your code. The wrapper inspects the request URL and method, matches it against a static map, and returns a pre‑constructed response object. If a delay is required, the wrapper uses a deterministic time provider (for example, a virtual clock seeded from the test run’s hash). Because the clock is seeded, every worker in a distributed CI job sees the same progression of “time”.

// tiny deterministic mock for fetch
type MockSpec = { url: string; method: string; body: any; delayMs?: number };
const mockCatalog: MockSpec[] = [
  {
    url: 'https://api.payments.example/charge',
    method: 'POST',
    body: { status: 'succeeded', id: 'ch_123' },
    delayMs: 42,
  },
];

export async function fetchMock(input: RequestInfo, init?: RequestInit) {
  const { url, method = 'GET' } = typeof input === 'string' ? { url: input } : input;
  const spec = mockCatalog.find(m => m.url === url && m.method === method);
  if (!spec) throw new Error(`No mock defined for ${method} ${url}`);

  const delay = spec.delayMs ?? 0;
  // deterministic sleep – uses a seeded virtual clock
  await deterministicSleep(delay);
  return new Response(JSON.stringify(spec.body), { status: 200 });
}

The deterministicSleep function draws its delay from a virtual clock that is seeded at the start of the test run. All workers share the same seed, so the observed delay is identical across machines. The mock never reaches the network, eliminating latency spikes and server‑side errors.

Because the mock is a pure JavaScript module, it can be bundled with the test artefacts. The resulting test binary is fully self‑contained, which aligns with the reproducible‑build principle described in the pillar post /blog/why-model-independence-is-an-engineering-posture. You own the logic outright; there is no hidden variance from an external model or service.

Integrating the mock into CI

1. Gate the mock behind a feature flag – keep the production code path unchanged. In CI, enable the flag so that the mock replaces the real client. This keeps the production deployment safe while giving the test suite deterministic behaviour.

1. Seal the mock catalogue – store the mock definitions in version‑controlled JSON files. Commit the files alongside the code that consumes them. The CI runner checks out the repository, loads the catalogue, and runs the tests. Because the catalogue is part of the source tree, the build artefact contains everything needed to reproduce the run. This mirrors the practice of sealed checksums for repeatable artefacts.

1. Validate the mock against the real service – run a one‑off verification job that hits the live endpoint, records the response, and diffs it against the stored mock. If the diff is non‑trivial, flag the change for review. This step ensures the mock stays faithful to the contract without introducing flaky behaviour into the regular CI flow.

1. Treat the mock as part of the test harness – the same harness that records certification verdicts (CERTIFIED, PROVISIONAL, etc.) can be extended to report “MOCK‑PASS”. The verdict does not claim the external service is stable; it only guarantees that the test suite behaved reproducibly under the deterministic mock.

By moving the network interaction into a deterministic layer, you convert a flaky‑test symptom into a reproducible CI job. The test suite now runs the same way on every worker, every day, every branch. The build artefacts become repeatable, and the engineering posture aligns with model independence: the code you ship is fully owned, fully testable, and free from external variance.

What to do on Monday

1. Identify one integration test that hits an external HTTP endpoint.
2. Extract the request details (URL, method, headers) and the expected response payload.
3. Create a small mock catalogue file containing those details.
4. Replace the live fetch call in the test with the fetchMock wrapper shown above.
5. Enable the mock flag in your CI configuration and run the suite.

If the test passes consistently across three consecutive CI runs, you have eliminated that source of flakiness. Replicate the pattern for other external calls, and your CI will evolve from flaky to deterministic, supporting reproducible builds without sacrificing realistic behaviour.