state-tracking · activeTracking state through a long task
Models lose track of the current state of entities, files, or an environment after many steps.
Also called: entity tracking, world-model consistency, stale state
Tags: autonomous-agent, coding-agent, context, data-analysis
A capable model maintains an accurate picture of the current state of what it is working on, updating it after every action, so that later decisions rest on what is true now rather than what was true earlier.
Claims
- mechanismsingle paperModels that predict next steps well can still hold an incoherent implicit world model, which fails when the task deviates from familiar traces.
- mechanismsingle paperOnly models with substantial code pretraining track entity state through a sequence of operations, and all degrade as the sequence lengthens.
- observationsingle paperIn partially observable text environments (ALFWorld, ScienceWorld), letting an LLM agent query an externally maintained state estimate that keeps an explicit distribution over unobserved object locations raises task success more than querying a deterministic memory of what has already been observed, and the gain shrinks to near zero on a frontier model that already nearly saturates the benchmark.
Techniques
- Checkpoint task state to a fileprocessWrite task id, status, completed steps and artifact paths to a small file after every meaningful step, and read it at session start.
- Re-read before editingprocessForce the agent to read the current version of a file immediately before changing it.
Capabilities are a way of carving up the subject, and carvings are arguable. Say so if this one is wrong — especially a proposed one, which a pipeline added because several papers used the same framing, not because anyone decided it was right.