Use Harness Verdicts as Feature‑Flag Gates to Ensure Release Safety

Green test suites give a false sense of safety

Most teams treat a passing CI run as the final seal of approval. The failure mode is clear: a green test suite does not guarantee functional correctness. Tests exercise only the paths they know about. Uncovered edge cases remain hidden. When a feature ships behind a flag, the flag can be toggled on in production before the code has been exercised by a realistic harness. The result is a user‑visible bug that could have been caught earlier.

Harness verdicts expose the missing rung

The certification harness records four outcomes: CERTIFIED, PROVISIONAL, INCONCLUSIVE, and FAILED. The middle two are the most useful for release safety. A PROVISIONAL verdict means the artifact ran reproducibly, but the harness could not yet assert full correctness. A CERTIFIED verdict means the harness exercised the contract and all checks passed. Treating these outcomes as binary “pass/fail” discards valuable nuance. Instead, map them to the feature‑flag lifecycle.

When a build produces a PROVISIONAL result, the harness has verified that the code compiles, loads, and executes without runtime errors on the supplied inputs. It has not yet proven that the behaviour matches the specification. This is the exact point where a feature flag should stay off. Only after a CERTIFIED verdict should the flag be allowed to flip on for all users.

Using verdicts as the flag gate

Implement the flag check as a small runtime guard that queries the latest harness verdict for the artifact. The guard can be a thin wrapper around the TypeScript client (@shpbl/sdk) or a direct HTTP call. The logic is straightforward:

import { getVerdict } from '@shpbl/sdk';

export async function isFeatureEnabled(featureId: string): Promise<boolean> {
  const verdict = await getVerdict(featureId);
  return verdict === 'CERTIFIED';
}

When the function returns true, the feature flag is considered safe to enable. If the verdict is PROVISIONAL, the flag remains false. This pattern forces the release pipeline to wait for a CERTIFIED outcome before exposing the feature to users.

The approach works with any deployment model. In a CI pipeline, the same check can block a promotion step. In production, the guard prevents accidental toggles by ops teams. Because the verdict is stored centrally, the gate is consistent across environments.

Aligning with the broader certification story

The four‑verdict model is described in detail in the pillar post /blog/the-four-verdicts-between-green-tests-and-correct-code. That article explains why the ladder of correctness matters. By treating PROVISIONAL as a “feature‑flag gate” and CERTIFIED as the “go‑live” signal, you close the gap between compilation and correctness without adding new tooling.

The live‑run view on /harness shows the verdicts in real time. Watching the harness execute gives you confidence that the artefact’s contract has been exercised. It also provides a clear audit trail for compliance teams.

What to do on Monday

1. Add a call to getVerdict (or the equivalent API endpoint) in the code that decides whether a flag is on.
2. Update your CI configuration to fail the promotion step if the latest verdict is not CERTIFIED.
3. Deploy the change to a staging environment and verify that the guard blocks the feature until the harness reports CERTIFIED.

These three steps turn the abstract notion of “half‑tested code” into a concrete, enforceable rule. After Monday, your feature flags will be backed by a measurable correctness level, not just a green test suite.