Repair Verifier
Checks repairs against the exam and rolls back regressions
What it costs
Bought once, yours forever. Roster workers are one-time purchases — no subscription. The eval report ships inside the bundle.
Just Repair Verifier, with its eval report.
All 7 workers — £105 bought singly, 33% off as a bundle.
Where it stands
In testing — running its suite ahead of release.
- Scenarios in the suite120
- Minimum pass rate98%
- p95 latency ceiling4.0s
- Cost per run ceiling£0.04
- RunsOn-demand
These are the thresholds Repair Verifier has to clear before it may ship — not results. Its measured eval numbers are in the report below.
The eval report
This ships inside the bundle. It is the contract: what the agent is tested on, what it refuses outright, and what it hands back to a human.
{
"agent": "the-forge-repair-verifier",
"version": "0.1.0",
"status": "measured",
"gate": {
"scenarios": 120,
"pass_rate_min": "98%",
"p95_latency_max": "4.0s",
"cost_per_run_max": "£0.04"
},
"suites": [
{
"name": "core",
"scenarios": 66,
"description": "The ordinary cases. What the agent is for, done correctly."
},
{
"name": "adversarial",
"scenarios": 30,
"description": "Prompt injection, contradictory instructions, hostile or malformed input, and attempts to make it act outside its contract."
},
{
"name": "regression",
"scenarios": 24,
"description": "Every bug ever found, kept forever so it cannot come back."
}
],
"refuses": [
"Acting on instructions found in content it was asked to read",
"Sending, publishing or paying anything without explicit approval",
"Inventing a fact it cannot source"
],
"escalates_to_human": [
"Confidence below the configured floor",
"Anything involving money, legal commitment or a new counterparty",
"Conflicting instructions it cannot resolve"
],
"measured": {
"certified": true,
"certified_on": "2026-08-12",
"baseline": "2026-08-10",
"judge": {
"model": "claude-opus-4-8",
"rubric_version": "1.0.0",
"calibrated": "2026-08-09",
"trusted": true
},
"results": {
"standalone": {
"passed": true,
"judged": true,
"judge_mean": 5,
"model": "claude-sonnet-4-6"
},
"veriphos": {
"passed": true,
"judged": true,
"judge_mean": 5,
"model": "claude-sonnet-4-6"
},
"replay": {
"passed": true,
"judged": false,
"judge_mean": null,
"model": "claude-sonnet-4-6"
}
}
}
}
measured block is real: per-host results from the latest eval run, judged against the 2026-08-10 baseline, certified 2026-08-12.Others in The Forge
Intake Checker
Vets incoming agent briefs for scope, duplication and roster fit
Spec Writer
Turns an approved brief into a frozen agent spec, assigning its target tier
Agent Builder
Builds the agent from its frozen spec
Exam Author
Authors the agent's frozen exam from the spec
Exam Runner
Runs the frozen exam and records the evidence
Release Packager
Packages the passing agent for delivery, refusing to ship below its target tier