Component Health Scores from Harness Verdicts

The failure to see reliability

Most teams treat a registry row as a line item. The count grows, the dashboard glitters, and the audit backlog swells. The problem is that the count says nothing about execution. A row can be a stub, a dead‑end, or a fully certified artefact. When the only signal is “present”, teams waste time chasing metadata that never runs. The result is a false sense of coverage and a real risk of deploying untested code.

Harness verdicts are the only ground truth

The certification harness runs each artefact against its declared contract. It records one of four outcomes: CERTIFIED, PROVISIONAL, INCONCLUSIVE, or FAILED. These outcomes are not marketing fluff; they are the only reproducible evidence that an entry can be executed. Because the harness is model‑independent, the verdict is computed, not generated, and the same input always yields the same result. The harness therefore supplies a binary truth table for every component.

Turning verdicts into a numeric health score

A health score condenses the four‑state verdict into a single number that can be sorted, filtered, and visualised. The simplest mapping preserves the ordering of confidence:

| Verdict | Base value | |--------------|------------| | CERTIFIED | 1.0 | | PROVISIONAL | 0.6 | | INCONCLUSIVE| 0.3 | | FAILED | 0.0 |

The score can be refined by weighting the age of the latest run and the frequency of re‑certification. A pragmatic formula is:

function healthScore(verdict: string, daysSinceRun: number, runsLast30d: number): number {
  const base = {CERTIFIED:1, PROVISIONAL:0.6, INCONCLUSIVE:0.3, FAILED:0}[verdict] ?? 0;
  const ageFactor = Math.max(0, 1 - daysSinceRun / 180); // decay after six months
  const freqFactor = Math.min(1, runsLast30d / 5);      // cap at five runs per month
  return +(base * ageFactor * freqFactor).toFixed(2);
}

The function returns a value between 0 and 1, rounded to two decimals. A component that was CERTIFIED yesterday and has been exercised five times in the last month scores 1.00. The same component, last run 150 days ago, drops to 0.83. A PROVISIONAL artefact exercised twice this month scores 0.48. A FAILED entry always scores 0, regardless of recency.

The calculation lives entirely in the client side. The same logic can be reproduced in the HTTP API or the TypeScript SDK, because the harness verdicts are exposed through the unified access gate. The score therefore becomes a deterministic attribute of each registry entry.

Using the score to focus audit effort

When the health score is added as a column in the component list, the weakest rows rise to the top. Teams can set a threshold – for example, 0.5 – and treat anything below as “audit‑required”. The threshold is not a hard rule; it is a decision point that turns an opaque count into a triage surface.

Consider a monorepo with 1,200 entries. After a single harness run, the scores distribute as follows:

• 250 entries at 1.00 – fully trusted, no immediate action.
• 400 entries between 0.6 and 0.9 – stable, schedule periodic re‑run.
• 350 entries below 0.5 – stale, missing contracts, or recent failures.

Instead of opening tickets for every row, the team now has a concrete backlog of 350 items. They can allocate a sprint to re‑run the harness, update contracts, or retire dead artefacts. The score also surfaces regressions: a component that fell from 0.9 to 0.2 in one week flags a recent break that needs urgent attention.

The approach aligns with the argument in the pillar post [/blog/a-component-count-is-not-an-inventory] – a count alone is not an inventory. A health‑scored inventory tells you which rows you can actually read, execute and grade.

Monday’s concrete step

1. Run the certification harness on your current registry via the live endpoint [/harness] or the HTTP API [/api-access].
2. Export the verdicts (verdict, timestamp, run count) to a CSV.
3. Apply the healthScore function in a short script and add the resulting column to your component spreadsheet.
4. Sort by the new column and flag every entry with a score below 0.5 for the next audit sprint.

Doing this before the end of the week turns an invisible risk into a visible work‑item list. The team can start the next sprint with a clear, data‑driven priority set, rather than guessing which rows are trustworthy.