Reproduction agent for flaky bugs
Some bugs only show up sometimes, so nobody can prove they are fixed. Verdict runs your test command again and again under conditions you approve, a tighter timeout, a slower upstream, no response at all, until it finds the combination that breaks it.
You get back the exact command, how often it failed, the commit range it points at and a regression plan to stop it returning. Nothing is written to your repository without your approval.
750 ms budget / no response10 of 10 generated recordsVERDICT_SCENARIO=SNAPSHOT_REGISTRATION VERDICT_REQUEST_BUDGET_MS=750 VERDICT_UPSTREAM_BEHAVIOR=NO_RESPONSE pnpm --filter @truefoundry/trueforge-core test -- tests/core/sandbox/daytonaSnapshotRegistration.test.ts --runInBandEach cell is one setting of the knobs you approved, run several times. The count in the corner is how many of those runs reproduced the bug. Select a cell and you see the command that produced it, the exit code and the commit it ran against. The bundled case is a fixture, and its numbers are generated rather than executed.
A result you cannot check is a rumour. Verdict keeps the command, the environment and every run record attached to the claim they produced, and it marks what it could not establish rather than quietly leaving it out.
Inspect the full evidence traildaytona-stalled-endpoint, 10 of 10 runsdaytona-responsive-endpoint, 0 of 10 runs@truefoundry/trueforge-core@0.1.4Three stages, each one allowed to claim less than the evidence it collected.
Runs your command across every condition you approved and counts how many attempts reproduced the bug. Conditions that half worked stay on the board as partial.
A condition that reproduces, or an honest missTakes the condition that reproduced and narrows it to the commits that could have caused it, replaying the same command across history where the code still builds.
A commit range, with its limits statedTurns the confirmed evidence into a regression-proof plan: the test name, the fixture and the assertion that should fail. It does not write the test, and a draft pull request is only ever opened by a workflow you approved.
A regression plan, held for your approvalA run that finds nothing still costs what you agreed and still tells you what it tried.
You define the knobs and the budget. The agent cannot widen its own experiment, so a run that finds nothing costs what you agreed to spend.
Every accepted observation carries its command, environment, exit code and commit. The conclusion is derived from those records rather than asserted alongside them.
Partial and unresolved conditions stay on the board. A search that half worked reports as a search that half worked.
The same records always produce the same verdict. The reducers live in a shared package and are covered by the test suite.
The trusted host validates the exact repository, workflow and ref before a dispatch is offered. The default is to stay read-only.
Each case exports a readable report and a machine-readable bundle, so the trail survives outside the interface.
Until you approve a dispatch, Verdict holds four read-only GitHub tools and nothing else. When it does propose a write, you see the exact repository, workflow and files first. The default is to stay read-only.
No. Verdict stops at a reviewable claim. It proposes a regression test and a publication manifest, and a maintainer decides whether anything is written.
That is a result, and it is reported as one. The matrix keeps not-reproduced, partial and unresolved conditions visible instead of collapsing them into a single confident answer.
No, and it says so on every screen. The case workspace you can click through is a fixture with generated numbers. The reproduction above it is not: it was executed against @truefoundry/trueforge-core@0.1.4, and the repository carries the record and a verifier that recomputes its hash. Keeping those two apart is the same discipline the product applies to its own findings.
A GitHub issue, a maintainer-approved command, the condition knobs worth trying and a run budget. Repository access stays on four read tools until you approve a dispatch.