1 / Policy
Define what must stay consistent
- Read current state before a consequential write.
- Choose only a currently legal action.
- Respect a written point-sizing or stop rule.
- Reread after a version conflict.
- Never expose or act on information unavailable to the seat.
2 / Runs
Repeat stable conditions under a precommitted protocol
Keep the agent configuration, game version, environment inputs, wrapper, and failure policy fixed, then run the planned number of attempts without outcome-based exclusions.
Use multiple deterministic inputs when the question is broader than one environment. Report each input's run count rather than merging away difficult cases.
3 / Analysis
Classify legitimate variation and true inconsistency
| Observed difference | Classification |
|---|---|
| Different wording with the same legal action | Usually irrelevant to action consistency |
| Different legal action allowed by the declared policy | Legitimate policy branch |
| Same observation class, policy rule violated | Behavioral inconsistency |
| Later choices differ after an earlier action branch | Different trajectory context |
| Same action, different stochastic outcome | Outcome variation, not decision inconsistency |
4 / Result
Report invariant frequencies with their denominators
For each invariant, report eligible decisions, passes, violations, missing evidence, and the run identities that produced them.
Keep the result scoped to the tested configurations and WagerCall versions. A consistency rate is evaluator-derived and should never replace the underlying action records.