Agents under evaluation
One readiness verdict per agent version. Scores use the six standard agent metrics and the default readiness bands, plus two checks we add: outcome verified against the system of record, and reliability across repeated runs (pass^k).
Agents under evaluation
6
9 versions in the last 30 days
Median loop time: change → verdict
2.6 days
▼ from 9.1 days · target < 3 days
Release gates this month
11 passed · 3 blocked
2 blocked on critical failures
Judge agreement with humans
87%
Judge v7, held-out set · target ≥ 85%
Open human reviews
23
Across IST · CET · ET queues
Agent versions
Metrics:TC task completenessRA response accuracyGR groundednessCO coherenceTU tool-use accuracyGA goal adherence| Agent · version | Readiness | TC | RA | GR | CO | TU | GA | Outcome ✓ | pass^3 | Judge agr. | Release gate |
|---|
Readiness bands (default, customizable)
Excellent90–100% · proceed with confidence
Good70–89% · deploy with caution
Moderate50–69% · investigate
Poor0–49% · do not deploy
What makes a verdict trustworthy
1Outcome-grounded: grade what changed in the record (approval, request state), not only what the agent said.
2Reliable: pass^k counts a scenario as passed only if all k repeated trials pass.
3Calibrated judge: every score carries a judge version and its agreement with human labels.
Agent sources
Built in AI Agent Studio4 agents
Conversational assistant1 agent
External · OpenTelemetry traces1 agent
The same suite runs on any agent type and on hosted or open-weight models (selector, top right).