Using PROVISIONAL Verdicts to Distinguish Correctness Levels

Most continuous integration pipelines communicate in a single bit: zero for success, non-zero for failure.

When a build passes, a package is minted, and tests exit cleanly, the artifact enters the deployment pipeline with a single implicit label: safe.

This is where the defect hides. Exit code zero tells you only that nothing blew up within the narrow bounds of the executed script. It confirms that the code compiles, the syntax is valid, and whichever specific assertions were written did not fail. It does not tell you whether the component’s invariants were actually exercised or whether anyone verified boundary conditions.

By flattening the evaluation into a binary pass, teams conflate basic executability with functional correctness. The pipeline claims victory at the lowest rung of the ladder, obscuring the actual safety margin of the artifact.

The Gap Between Executable and Correct

To build dependable systems, you have to separate "the artifact runs reproducibly" from "the artifact does what it claims under full contract bounds."

In a disciplined verification model, an artifact moves through explicit states rather than a boolean flag. As we outline in our guide on the four verdicts between green tests and correct code, a binary pass/fail is insufficient for automated gating.

A complete verdict ladder uses distinct classifications:

1. FAILED: The component violated a declared invariant or threw an unhandled fault under valid inputs.
2. INCONCLUSIVE: The test harness could not construct valid inputs or satisfy environmental prerequisites. This is a harness limit, not a component defect.
3. PROVISIONAL: The component builds cleanly and executes reproducibly, but its contract correctness has not yet been asserted against exhaustive invariants.
4. CERTIFIED: The component has been exercised against its formal contract, boundary cases, and state invariants, with passing proofs or test suites.

The most critical rung in day-to-day development is the PROVISIONAL verdict. It marks the transition from static viability to dynamic assurance.

What a PROVISIONAL Verdict Captures

Consider a sliding-window rate limiter written for an HTTP edge gateway.

export class SlidingWindowLimiter {
  private timestamps: number[] = [];

  constructor(
    private readonly windowMs: number,
    private readonly maxRequests: number
  ) {}

  public allow(now: number): boolean {
    this.timestamps = this.timestamps.filter(t => now - t < this.windowMs);
    if (this.timestamps.length < this.maxRequests) {
      this.timestamps.push(now);
      return true;
    }
    return false;
  }
}

A standard test runs: instantiate the class, submit five requests under a limit of ten, assert that all return true. The test suite passes. The build tool exits with code zero.

Is this code verified? No. Under real conditions, this.timestamps.filter on every invocation will degrade under load. More critically, if timestamps arrive out of order from distributed clocks, the filter logic fails silently.

Marking this artifact as CERTIFIED is dishonest. The harness has only verified that the happy path executes without syntax errors.

Marking it PROVISIONAL is honest. It asserts:

• The module compiles without warnings under strict type checks.
• The exported interface matches the consumer's expected signature.
• Basic smoke tests demonstrate structural viability.
• Correctness remains unproven. Property tests for monotonicity, scale limits, and concurrency have not yet been evaluated.

A provisional verdict protects downstream consumers from false confidence. It states clearly that while the code is well-formed, its behavioral bounds remain unverified.

Gating Downstream Work with Explicit Rungs

Once you treat PROVISIONAL as a first-class status in your metadata, you can write precise downstream policies instead of relying on all-or-nothing deployments.

A deployment orchestrator should treat rungs differently:

• Production Traffic: Requires CERTIFIED. If a dependency holds only a PROVISIONAL verdict, the release pipeline rejects promotion to tier-1 infrastructure.
• Internal Canaries and Shadow Environments: Can accept PROVISIONAL artifacts. You can route mirrored traffic through a provisional component to observe behavior without exposing user-facing transactions to unproven invariants. (See our analysis on using CERTIFIED and PROVISIONAL verdicts for safe canaries).
• Local Development: Accepts PROVISIONAL or even INCONCLUSIVE builds so engineers are not blocked while full verification suites run asynchronously.

This stops the common anti-pattern where engineers weaken assertions or skip long-running test suites just to get a green light in CI. Under a multi-verdict system, skipping the invariant suite does not yield a passing build; it yields a PROVISIONAL artifact that cannot leave the staging sandbox.

What to Do on Monday

You do not need to overhaul your entire infrastructure to stop conflating compilation with verification.

1. Audit your test reports. Identify which suites merely assert "function runs without throwing" versus suites that check boundary properties and invariant states.
2. Add a metadata tag to build manifests. Replace status: "passed" with an explicit state: PROVISIONAL for builds that have passed only smoke and unit tests, and CERTIFIED only when full property and contract suites execute.
3. Update your deployment gate. Configure your production deployment tool to fail if any newly published internal package lacks a CERTIFIED tag in its manifest.

Naming your verification rung honestly prevents simple executability from masquerading as system safety.