Turn INCONCLUSIVE Harness Results into Targeted Test Coverage

Green tests are a mirage

A test suite that reports all passed does not guarantee the artifact does anything useful. The suite only shows that the code can be exercised under the inputs you supplied. When the certification harness returns INCONCLUSIVE, the runner could not satisfy the artifact’s input contract. That is not a failure of the code; it is a failure of the test set. Ignoring the signal leaves a gap between “it compiles” and “it is correct”.

The verdict tells you what you did not provide

The harness records four outcomes. INCONCLUSIVE appears when the harness cannot drive the artifact far enough to observe a definitive behaviour. The log typically contains two clues: a missing environment variable and a rejected file format. Those clues are the contract edges the artifact expects.

For example, an artifact that parses a CSV file may emit INCONCLUSIVE because the harness supplied an empty file. The harness log will show “input file size = 0 bytes – cannot infer schema”. The missing contract is “non‑empty CSV with a header row”.

Treat the log as a contract checklist. Each missing element is a concrete test input you must add. The checklist is not a to‑do list of vague “more tests”. It is a set of required inputs that the harness could not fabricate.

Turn the missing contract into a test case

1. Capture the exact expectation – Re‑run the artifact with the harness in verbose mode. Note the error message and the point where execution stopped. In the CSV example, the message “expected at least one header row” pinpoints the contract.

1. Create the minimal satisfying input – Build the smallest file that meets the contract. A one‑line header followed by a single data row is enough to prove the parser can advance beyond the contract check.

1. Add the input to the harness configuration – The harness accepts a JSON manifest describing inputs. Insert the new file path and any required environment variables.

1. Rerun the harness – If the artifact now proceeds to its functional checks, the verdict will upgrade to PROVISIONAL (reproducible, correctness not yet asserted) or CERTIFIED (if all downstream checks also pass).

The same pattern works for missing environment variables, network endpoints, or feature flags. The key is to isolate a single missing contract, satisfy it, and observe the verdict shift.

A concrete walk‑through

Suppose you have an artifact transformer that reads a JSON payload from an environment variable PAYLOAD. Your current harness configuration only sets PAYLOAD=''. The harness returns INCONCLUSIVE with the log entry “PAYLOAD is empty – cannot deserialize”.

Step 1 – Run the harness with --debug. The log confirms the deserializer threw EmptyInputError.

Step 2 – Craft the smallest valid JSON, e.g. { "id": 1 }.

Step 3 – Update the manifest:

{
  "env": { "PAYLOAD": "{\"id\":1}" }
}

Step 4 – Rerun. The harness now reaches the transformation logic and reports PROVISIONAL because the output matches the expected schema.

From here you can add further inputs that explore edge cases (missing fields, large arrays) and eventually achieve CERTIFIED once all functional checks succeed.

Monday’s concrete step

1. Pull the latest harness run that produced an INCONCLUSIVE verdict.
2. Open the log and write down the first missing contract it mentions.
3. Create a minimal input that satisfies that contract.
4. Add the input to the harness manifest and rerun.

If the verdict upgrades, you have expanded your test coverage with a single, targeted change. If it stays INCONCLUSIVE, repeat the process for the next missing contract.

The process is repeatable and cheap: each iteration costs only a few minutes of editing and a harness run. Over a sprint you can convert a suite of INCONCLUSIVE outcomes into a robust set of PROVISIONAL and CERTIFIED artefacts, closing the gap highlighted in the pillar post /blog/the-four-verdicts-between-green-tests-and-correct-code.

By treating the harness as a contract auditor rather than a binary pass/fail gate, you turn ambiguous signals into actionable test expansion. The result is a test suite that does more than stay green – it proves the artifact can handle the inputs it really expects.