Skip to content

Article / Replay

Canonical transcript vs raw logs for reproducible AI agent runs

Raw logs preserve operational detail. A canonical transcript preserves the normalized state and event fields needed to compare equivalent runs. They solve different debugging and reproducibility problems.

For
Teams choosing an equality target for agent replay and regression checks
Outcome
Compare runs without requiring volatile infrastructure data to match

The distinction

Canonical transcripts describe behavior; raw logs describe execution

A canonical transcript keeps ordered domain evidence and removes fields that can change without changing the run's meaning, while raw logs retain the surrounding operational detail.

Request identifiers, database row identifiers, timestamps, worker names, and trace identifiers are valuable during an incident. They are also expected to differ across two correct executions. Requiring byte-for-byte equality over those logs creates false replay failures.

A canonical projection chooses stable fields deliberately: event sequence, event type, public observation, accepted action payload, authoritative outcome, and ledger legs where relevant. Its schema becomes part of the replay contract.

Use the right record

Choose the record from the question you need to answer

Canonical transcript and raw log responsibilities
QuestionCanonical transcriptRaw logs
Did the same actions produce the same state?Primary evidenceSupporting detail
Which worker timed out?Usually omittedPrimary evidence
Did event order change?Stable sequence comparisonMay contain duplicate emissions
Why did latency increase?May retain bounded durationsFull operational context
Can a public reviewer inspect the run?Safe projection possibleOften contains protected data

Design constraint

Normalization must remove volatility without erasing behavior

Exclude fields that are incidental to equivalent execution, but retain every field whose change would alter the evaluator's account of what happened.

Removing timestamps is usually safe for deterministic equality when event sequence is preserved. Removing rejected attempts, action payloads, settlement legs, or version identity is not safe when those fields define the behavior being tested.

The normalization rule should be versioned and reviewable. Otherwise a transcript can appear stable only because a new behavioral field was silently dropped from the comparison.

Practical boundary

Retain both records instead of forcing one to do every job

Use the canonical transcript for replay assertions and public evidence, and keep access-controlled raw logs for operations and incident diagnosis.

  • Generate canonical order from domain sequence, not log arrival order.
  • Keep protected credentials and private observations out of public projections.
  • Link a canonical event to operational traces through controlled identifiers when needed.
  • Describe transcript equality narrowly; it is not proof that two agents reasoned identically.

Next step

Inspect canonical evidence

Open a completed public record without relying on raw application logs.

Inspect canonical evidence