Short answer
The seed controls environment construction, not the agent's policy
When every stable environment input matches, the seed can reproduce the same hidden shoe or random stream. It does not control model sampling, prompt changes, tool wrappers, timeouts, or agent choices.
The environment determines what state exists and how a legal action changes it. The agent determines which action to attempt after receiving an observation. Those are separate systems even when they interact in one run.
Two agents can begin from the same seeded environment and diverge on the first choice. After that decision changes state, their later observations can differ too. The divergence is evidence about behavior, not proof that the environment failed to reproduce.
Three claims
Environment, transcript, and trajectory reproducibility are not interchangeable
| Claim | What must remain fixed | Expected result |
|---|---|---|
| Environment | Version identity, config, RNG identity, seed | Same constructed environment |
| Transcript | Same environment plus same accepted actions | Same canonical transitions and events |
| Agent trajectory | Agent stack plus all relevant inputs | Not guaranteed by the environment seed |
Diagnostic
Use same-actions replay to test the environment separately
To verify deterministic environment behavior, replay the same accepted action sequence against the same stable inputs and compare the canonical transcript.
This diagnostic removes the agent's choice variability from the question. Raw database rows are a poor equality target because run identifiers, request identifiers, and timestamps are expected to change. A normalized transcript can retain the ordered state, action, event, outcome, and ledger fields that define the environment's behavior.
Once that test passes, fresh agent runs can be compared as behavioral samples. Their different actions should remain visible rather than being normalized away.
Evaluation
Treat divergence as a measurement target, not a replay defect
Ask where two trajectories first differ, what each agent could observe at that point, and whether the alternatives were legal and policy-consistent.
- Name the complete environment identity in the report, not only the seed.
- Record the model, prompt, wrapper, and retry policy as agent-side variables.
- Compare repeated runs rather than inferring consistency from one sample.
- Do not claim that a deterministic environment makes the agent deterministic.