Amazon takes deductions from vendor payments: shortages, price claims, chargebacks. Many are wrong, and nobody checks them in time. This engine reconciles every deduction and payment against the vendor's own records and says, for each one, whether it can be disputed, why, and which file row proves it.
Recoverable now versus lost because the dispute window closed before anyone looked. Hover a bar for the numbers.
Every claim has a deadline. The work queue puts claims whose dispute window closes within 30 days first, then ranks by expected value: amount times an assumed win rate for that kind of evidence (configurable priors, to be replaced by a vendor's own dispute history).
Most deductions are valid and the engine says so. A claim is only made when the vendor's records prove it; when they can't, the finding says what is missing instead of guessing.
The largest claims for the selected vendor, each with the rule that decided it, the evidence, and the exact file and row it comes from.
The generator records the right answer for every deduction it creates; the engine never sees it. This run used a seed that was never used during development.
The same underlying world, with more and more proofs of delivery missing and references truncated or mistyped, scored against what really happened rather than what the files can prove. Precision is the share of claims that are true; recall is the share of real money recovered.
Claude drafts each letter from the facts of one finding (an offline template stands in when no API key is set). A deterministic verifier then requires every amount, document number and date in the letter to appear in those facts; a letter that fails is rejected. To prove the verifier works, each letter is re-checked with one error injected at a time.
A perfect score on its own benchmark proves consistency, not truth. The value of the benchmark showed when it failed:
94.95 % precision, $88,757 in false claims → match payments by PO when that PO has a single invoice → 100 %, $0
1.234,50 was read as 1.2345 with no error; 31.12.2025 dates were rejected.98.01 % recall, $2,669 missed → detect the decimal separator per column, per file → 100 %
MJPYNQ05 into MJPYNQ5, so the typo MJPYNQ0V stopped matching. Found on the holdout run.5 wrong verdicts hidden by a rounded "100.00 %" → fuzzy match on raw characters, print exact mismatch counts → 0 mismatches
QQUOGOZZ passed.968 of 972 injected errors caught → any token with a digit, a hyphen or 8+ capitals must come from the facts → all caught