Block Release Promotion on PROVISIONAL Verdicts

Green CI runs mask missing guarantees

A build that finishes with “all tests passed” still leaves the release vulnerable. The tests have merely shown that the harness could execute the artifact; they have not proved the artifact meets its functional contract. In practice this shows up as a release that ships without the behaviour a downstream consumer expects, leading to hot‑fixes, rollback, and lost confidence. The failure mode is treating a passing CI run as a de‑facto certification.

A PROVISIONAL verdict is not a warning flag

When the certification harness reports PROVISIONAL, the artifact has been executed and reproduced, but the harness could not assert correctness. The verdict distinguishes two things: reproducibility (the artifact can be built and run) and correctness (the harness could not verify the output against the specification). This is a precise failure mode – a missing correctness guarantee – not a generic “something went wrong”.

A correct posture is to treat the PROVISIONAL label as a hard stop in the release pipeline. The pipeline should reject promotion to any downstream environment until the artifact moves to CERTIFIED or the missing checks are addressed. Ignoring the label treats the provisional state as an informational note, which is exactly what leads to the “green but broken” releases that waste engineering time.

The cost of letting provisional artifacts through

When a provisional artifact reaches production, the most common symptom is a silent contract breach: a downstream service receives malformed data, a feature flag toggles unexpectedly, or a performance SLA is missed. The root cause is that the harness could not exercise the full input contract – a limitation of the test harness, not a defect in the artifact.

If the release gating logic respects the provisional verdict, the failure is caught early, in the same CI run that produced the verdict. The team can then either:

• augment the harness inputs to cover the missing contract paths, or
• add targeted unit or integration tests that provide the missing evidence.

Both actions raise the verdict to CERTIFIED and give a concrete, repeatable guarantee that the artifact does what it claims.

How SHPBL enforces the gate

SHPBL’s certification harness runs every artifact through the same CERTIFIED / PROVISIONAL / INCONCLUSIVE / FAILED pipeline. A provisional result is recorded in the harness report and automatically blocks promotion in any CI configuration that respects the harness outcome. The library’s typed TypeScript client (@shpbl/sdk) and the plain HTTP API expose the verdict status, so a simple check in your CI script can abort the release if the verdict is not CERTIFIED. The harness itself is documented at [/harness] and the four‑verdict model is explained in the pillar post [/blog/the-four-verdicts-between-green-tests-and-correct-code].

What to do on Monday

Add a single step to your CI definition that queries the SHPBL harness for the latest verdict of the artifact you are about to release. If the response is anything other than CERTIFIED, fail the job and raise a ticket to extend the harness inputs or add missing tests. This change turns a provisional verdict from a passive label into an active release gate, protecting your downstream users without altering any existing test suites.