Short answer
A conforming MCP connection can still support poor agent behavior
An agent may discover and call tools through a valid MCP client while choosing the wrong tool, sending stale state, mishandling a product error, or misreporting the result.
Conformance focuses on the shared protocol surface: capabilities, message shapes, lifecycle, and transport behavior appropriate to the implementation. It does not know the domain meaning of a WagerCall expected version or whether an agent selected a legal action from the latest observation.
Behavior testing starts where the generic protocol ends. It exercises the server's published schemas and semantics, then evaluates how the agent responds to the structured results.
Scope map
Name the layer each test covers
| Layer | Example question | Result can support |
|---|---|---|
| Protocol | Can client and server exchange valid MCP messages? | A bounded conformance claim |
| Tool schema | Does the call satisfy this tool's input contract? | Contract compatibility |
| Domain behavior | Did the agent choose a legal action from current state? | Product-specific behavior evidence |
| Recovery | Did it reread after conflict or back off after a limit? | Reliability evidence for tested cases |
| Outcome | Did the workflow reach the declared terminal state? | Task result under the named environment |
Product example
WagerCall exposes behavior tests, not a general MCP certificate
WagerCall can reveal how an agent handles its discovery, schema, ownership, version, idempotency, rate-limit, Session, and Room contracts.
Those tests are valuable because they are stateful and observable. A call can be valid JSON and still be stale, unauthorized, illegal for the current game state, or unsafe to retry with a new request identity.
Passing them does not establish that the client conforms to every MCP feature or interoperates with every MCP server. The accurate claim is that the tested agent behavior matched the named WagerCall contract and cases.
Reporting
Report a matrix of tested behavior instead of one overloaded pass
- Name the MCP protocol and server contract version used.
- List the tools and error cases actually exercised.
- Separate schema-valid calls from semantically correct decisions.
- Link claims to structured responses and authoritative state changes.
- Avoid extending product-specific results to untested servers or protocol features.