The measuring is the part that usually gets skipped.
deflection-audit Live
Your AI vendor reports 85% deflection. This audits what that number survives. It re-judges every AI-closed conversation from the transcript alone, then recomputes the rate crediting only closures that hold up. On a 40-conversation labeled set, 80% claimed came out to 55% audited.
skill-regression-harness
Agent skills are prompt-matched, so a model upgrade can stop one from firing without an error anywhere. Static checks plus live dispatch tests of 23 frozen prompts, run under a tool denylist so skills that reach Salesforce and Gmail can't touch either. 21 of 23 correct on the live run.
agent-ops-bench
Does a second agent pay for itself? Generalist, parallel specialists, and generalist plus QA reviewer, measured on accuracy, cost per ticket, and latency. Specialists took routing from 83% to 97% and left severity flat; the QA layer lifted severity from 70% to 80% at $0.60 per net correction.
claude-eval-kit
The eval machinery from these projects, pulled out into something reusable. Blind classification, confusion matrices, ordinal miss-direction splits, and confusable-pattern trap audits. It's the scoring layer under deflection-audit and the skill harness.
claude-triage-simulator
An LLM triage classifier with a real eval harness. Severity and routing for IoT support tickets, scored against a 60-ticket labeled set with under- versus over-triage analysis. It caught all 12 critical tickets.