Using PROVISIONAL Harness Verdicts as a Canary Gate

Most canary rollouts fail not because traffic routing broke, but because the decision to route traffic was arbitrary.

A developer merges a branch after unit tests pass, opens an administrative panel, and sets a feature flag to 5%. If the error budget does not collapse within thirty minutes, they push it to 25%, then 100%. This is deployment by survival. It treats the absence of an immediate crash as proof that the implementation is correct.

There is a measurable distance between code that passes a test suite and code that is safe for full production volume. In The Four Verdicts Between Green Tests and Correct Code, we outline the taxonomy that sits across that gap: CERTIFIED, PROVISIONAL, INCONCLUSIVE, and FAILED.

If you want rollouts that do not rely on intuition, you tie your deployment pipeline directly to those rungs. Specifically, you use a PROVISIONAL verdict as the mechanical precondition for a canary release.

What PROVISIONAL Actually Asserts

A PROVISIONAL verdict does not mean "we ran out of time to test this." It represents a concrete technical state: the artifact executes repeatably, builds without non-deterministic side effects, and satisfies baseline invariant checks on standard inputs. However, its broader operational contract—its behavior under saturation, malformed boundary inputs, or stateful drift—has not yet been asserted by the test gauntlet.

+------------------+     +-------------------+     +-----------------+
|   PROVISIONAL    | --> |   Canary Gate     | --> |   CERTIFIED     |
| (Baseline valid, |     | (1-5% traffic,    |     | (Full rollout,  |
|  contract open)  |     |  telemetry wired) |     |  invariants met)|
+------------------+     +-------------------+     +-----------------+

Treating code as binary—either ready for all users or ready for none—forces teams into bad trade-offs. You either stall shipping while engineers hand-craft mocks for every edge case, or you push unverified artifacts directly to the entire fleet.

Using PROVISIONAL as a formal feature flag gate resolves this tension. It gives the artifact permission to run in production, but restricts its blast radius to a canary cohort while live telemetry closes the verification gap.

Connecting Harness Metadata to Flag Evaluation

To make code certification practical, the verdict must travel with the build artifact as structured metadata rather than living in a continuous integration log. Using harness verdicts as feature-flag gates ensures the application runtime resolves exposure dynamically from that metadata.

When the application evaluates whether to enable a new code path for an incoming request, it inspects the artifact's current certification status:

type HarnessVerdict = 'CERTIFIED' | 'PROVISIONAL' | 'INCONCLUSIVE' | 'FAILED';

interface ReleaseManifest {
  artifactId: string;
  verdict: HarnessVerdict;
  canaryThresholdPct: number;
}

function resolveFeatureExposure(
  manifest: ReleaseManifest, 
  userHash: number
): boolean {
  switch (manifest.verdict) {
    case 'CERTIFIED':
      return true;
    case 'PROVISIONAL':
      // Confine execution to the canary window
      return (userHash % 100) < manifest.canaryThresholdPct;
    case 'INCONCLUSIVE':
    case 'FAILED':
    default:
      return false;
  }
}

Under this model, manual intervention is removed from the initial rollout phase. If an automated run produces a PROVISIONAL verdict, the feature can deploy straight to production, but the runtime limits its execution strictly to the configured canary slice (for example, 1% to 5% of traffic).

If the certification harness reports INCONCLUSIVE—meaning the test runner could not exercise the input contract—the runtime keeps the flag closed. It does not matter that the code compiled cleanly; unexercised contracts do not enter canary pools.

Upgrading from Canary to Full Promotion

A canary release gated by a PROVISIONAL verdict is not meant to sit in that state indefinitely. The canary window exists to collect execution traces and telemetry that validate the invariant contracts the harness could not prove in isolation.

The promotion lifecycle works in three deliberate steps:

1. Canary Execution: The PROVISIONAL artifact handles a restricted traffic partition. Telemetry gathers real-world input shapes, performance distributions, and unexpected exit codes.
2. Contract Synthesis: The recorded execution traces are fed back into your local verification suite or CI pipeline to confirm that invariants held across the canary sample.
3. Verdict Promotion: Once the contract passes the complete assertion suite, the build artifact is marked CERTIFIED. The release manifest updates, and the feature flag logic automatically widens exposure to 100% without code changes or manual parameter tweaking.

If the canary encounters an unhandled invariant violation, the verdict drops to FAILED, and the flag evaluator immediately cuts traffic back to zero.

What to Do on Monday

Do not overhaul your entire deployment pipeline at once. Start with your next non-trivial feature rollout:

1. Stop using binary flags for new logic. Add a metadata field to your feature flag configuration for verdict alongside the percentage rollout.
2. Assign PROVISIONAL explicitly. If a pull request has green unit tests but has not run against production-mirrored traffic or integration invariants, classify its artifact as PROVISIONAL.
3. Cap PROVISIONAL exposure at 5%. Enforce in your flag routing layer that no artifact marked PROVISIONAL can be assigned higher traffic.
4. Define the promotion condition. Write down the exact telemetry metrics or test run required to flip that verdict to CERTIFIED before you deploy the canary.