WorldTrace: Addressable Memory for Video World Models
Abstract
We study visual persistence in autoregressive video world models. The KV cache accumulates a growing visual memory but, once rollouts extend beyond the training horizon, the model can no longer reliably address stored content. We show that compression alone cannot restore recall: temporal positional encodings go out of distribution, so attention cannot retrieve compressed memory regardless of what is stored in it. We propose WorldTrace, a training-free framework that assigns each summary slot a fixed, in-distribution position by slot rank, and explore two retention approaches: WT-Field (canonical-key averaging, +15.5% TempSSIM) for temporal coherence and WT-Landmark (frozen verbatim traces, PAC = 0.989 at 256x training horizon) for episodic recall.