An Agent Evaluation Should End in a Release Decision

For a hypothetical service-desk agent, distinguish reading an incident, proposing a resolution and executing a change. Give each action a permission boundary. A correct diagnosis does not authorize a production restart, and a polite answer does not prove the requested ticket was actually updated.

Construct cases from the intended workflow: a routine issue, missing context, conflicting records, an unavailable tool, a duplicate request and an instruction embedded in retrieved material that tries to redirect the agent. Include tasks that should be declined or escalated. Keep a held-out set that is not used to tune prompts.

Specify the expected final state and prohibited side effects for each case. Use isolated test accounts, synthetic records and resettable fixtures. A test environment should let reviewers inspect what happened without creating real customer changes.

Use deterministic checks for facts that can be verified directly: which record changed, whether a restricted tool was called and whether required approval preceded execution. For explanation quality or ambiguous outcomes, use a written rubric and expert review. Model-based grading needs calibration against examples people have judged.

Anthropic's evaluation guidance describes combining code-based, model-based and human graders. Apply that idea according to the task: a fluent completion message should not outweigh a failed state check, while a harmless variation in wording should not automatically fail a correctly completed workflow.

Run cases repeatedly where nondeterminism affects the outcome. Record tool failures, human interventions, latency and cost alongside completion. Inspect transcripts for unexpected behavior, but restrict and redact logs so evaluation does not become a new store of sensitive content.

Set acceptance thresholds by consequence before running the suite. A forbidden write or permission bypass should block release even if the average completion score is high. A lower-risk explanation problem might instead lead to a narrower launch scope and a defined correction plan.

Version the model, prompts, tool definitions, permission configuration, source snapshot, fixtures and graders together. Compare the candidate with the current baseline and rerun known failure cases. A model update or retrieval change can alter behavior even when application code stays the same.

After approval, use a controlled rollout with stop conditions, escalation coverage and a tested rollback. Feed appropriately handled production failures back into regression cases. The suite informs an accountable owner's decision; a high score does not grant the agent authority beyond its approved workflow.