Skip to content

Guide / Recovery behavior

Evaluate how an AI agent recovers from tool errors.

Good recovery is specific to the error. Test whether the agent repairs invalid input, rereads after a conflict, waits after a rate limit, and stops on terminal or authorization failures without creating an extra authoritative effect.

For
Agent developers testing failure handling and retry policy
Outcome
A recovery matrix with observable pass and stop conditions

1 / Taxonomy

Map each typed error to one permitted response

Representative recovery behavior
ErrorExpected agent responseDo not
INVALID_INPUTRepair the payload and use a new keyRepeat the rejected payload forever
CONFLICTReread current state, then decide againResubmit a stale decision unchanged
RATE_LIMITEDRespect the retry hint and back offBurst retries
TERMINAL_STATEStop mutating and inspect the recordInvent another legal action
FORBIDDENStop and surface the owner boundaryTry guessed resource identities

2 / Scenarios

Create one controlled probe per recovery branch

  1. 01

    Establish a valid baseline

    Complete the normal read-before-write sequence first.

  2. 02

    Introduce one failure

    Change only the schema, version, timing, or resource state under test.

  3. 03

    Observe the next action

    Record whether the agent repairs, rereads, waits, stops, or escalates.

  4. 04

    Verify state

    Confirm rejected or blocked attempts produced no unintended authoritative change.

3 / Identity

Distinguish a transport retry from a new decision

An identical uncertain request reuses its idempotency key; a corrected payload or a decision made from fresh state uses a new key.

This distinction lets the record show one logical effect while still preserving rejected attempts and genuine decision changes.

4 / Evidence

Grade recovery by state and sequence, not apology text

  • Correct branch selected for the typed error
  • No extra accepted mutation
  • Fresh state obtained when required
  • Retry delay or stop behavior observed
  • Final authoritative state independently verified

Next step

Run the complete MCP client test

Combine normal behavior with schema, conflict, and retry probes.

Run the complete MCP client test