CI Integration of Harness Verdicts to Block Unsafe Deploys

Green Tests Are Not a Safety Net

A CI run that ends with “All tests passed” is often treated as a guarantee. The failure mode is clear: the artifact that will be deployed has never been exercised by the certification harness. The test suite may cover only a fraction of the behaviour that the harness checks. When a change passes the unit tests but triggers a FAILED verdict in the harness, the deployment still proceeds. The result is a production incident that could have been blocked earlier.

Treat the Harness Verdict as a CI Gate

The harness produces one of four explicit outcomes: CERTIFIED, PROVISIONAL, INCONCLUSIVE, or FAILED. Those outcomes are not decorative labels; they are the only reliable signal that the artifact satisfies its contract.

A CI pipeline should query the harness after the build step and abort the deployment if the verdict does not meet the policy. A minimal policy might be:

Allow deployment only when the verdict is CERTIFIED or PROVISIONAL.

Anything else stops the pipeline. This turns the harness into a true gate instead of a badge that sits on the side of the build log.

Example: GitHub Actions Integration

jobs:
  build-and-test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Build artifact
        run: npm run build
      - name: Run test suite
        run: npm test
      - name: Submit to harness
        id: harness
        run: |
          curl -s -X POST https://api.shpbl.com/harness \
            -H "Authorization: Bearer ${{ secrets.SHPBL_TOKEN }}" \
            -F artifact=@dist/my-lib.tgz > result.json
          echo "verdict=$(jq -r .verdict result.json)" >> $GITHUB_OUTPUT
      - name: Enforce CI gate
        if: steps.harness.outputs.verdict != 'CERTIFIED' && steps.harness.outputs.verdict != 'PROVISIONAL'
        run: |
          echo "Deployment blocked – harness verdict: ${{ steps.harness.outputs.verdict }}"
          exit 1

The step Enforce CI gate reads the verdict directly from the harness response. If the verdict is INCONCLUSIVE or FAILED, the job exits with a non‑zero status, preventing any subsequent deploy step from running.

Policy Choices and Trade‑offs

A PROVISIONAL verdict means the harness could execute the artifact but has not yet asserted full correctness. It is useful when the artifact is new or when the certification pipeline is still being populated with test vectors. Blocking on PROVISIONAL is a conservative stance; allowing it through requires an explicit policy decision.

A CERTIFIED verdict gives the strongest guarantee: the harness exercised the artifact against the full matrix of 40 primitives and recorded a successful run. Deployments that require the highest safety level should require this verdict.

INCONCLUSIVE indicates a limitation of the harness, not a defect in the artifact. Treat it as a separate policy decision: you may allow it if the missing coverage is understood and acceptable, or you may block until the harness is extended.

FAILED always blocks. The harness detected a contract violation; proceeding would be unsafe.

Choosing the right mix depends on your risk tolerance and the maturity of your test matrix. The key point is that the decision is explicit, not an implicit assumption that a green test suite is sufficient.

Reporting and Auditing

When the gate blocks a deployment, surface the verdict and the harness log in the CI output. This provides immediate feedback to the developer and creates an audit trail for compliance teams. The SHPBL harness includes a reproducible run record; the archive checksum can be verified against the published root of trust. Linking to the run (/harness) lets anyone replay the exact conditions that produced the verdict.

How SHPBL Makes This Straightforward

SHPBL supplies a model‑independent harness that returns the four standard verdicts. The same endpoint is reachable via the HTTP API, the typed TypeScript client (@shpbl/sdk), or the offline file‑based edition. Because the artifact is never generated by a model, the result is repeatable and verifiable. The certification harness records a verdict for each submission, so you can embed the call directly in any CI system and enforce a policy that matches your safety requirements. See the live run at /harness for an example of the verdict payload.

Next Steps for Monday

1. Add a harness submission step to your CI pipeline, using the API or SDK.
2. Define a policy that permits only CERTIFIED or PROVISIONAL verdicts

for production releases.

1. Enable the CI gate to abort on any other verdict and surface the harness

log in the build output.

Implementing these three changes turns the harness verdict into a real deployment safety mechanism, closing the gap between compilation and correctness that many teams overlook.