Velox: Spatial Memory Speeds Up Video World Models

Background and Context

Video world models turn actions and camera motion into long visual rollouts. That makes video a natural interface for interactive simulation. Interactivity demands fast generation. Visual consistency demands that the model remember what it has already drawn. Chunked video diffusion models, the current mainstream design, keep this history in a KV cache that grows with every chunk.

The result is a simple and costly pattern. The longer the rollout, the larger the cache, and the slower each later chunk becomes. A paper on Velox targets exactly this bottleneck. Velox is a training-free attention router. It picks context at region level, using frustum-slab correspondences derived from streaming geometry.

The core idea is that spatial memory can tell the model which past chunks matter for the region it is drawing now. Not every historical chunk needs to join the computation. The authors report that at the 48th chunk, attention runs 4.36x faster and denoising runs 2.43x faster. On the full 158-case WBench navigation set, the API-free average score stays within 0.15 of the dense LingBot-Fast-Cam baseline.

Deep Analysis

The method works in a few steps. First, Velox represents each past chunk as a camera-frustum slab with a depth interval. The slab records roughly which region of three-dimensional space that chunk covered. Second, the depth of the current chunk is obtained through recency-priority multiwarp, which warps historical depth into the current view with the most recent history taking priority. Third, the overlap between slabs softly modulates the block importance that the model already computes in each layer. This is modulation, not replacement, so the model's own judgment is kept. Fourth, regions that history does not cover receive no geometric bias, and the model's native routing applies. Geometry intervenes only where it is reliable. Where it is not, the system falls back to the original behavior.

Why does Velox use token-block granularity for routing? Routing granularity in Velox matches FlashAttention-style tiled execution. Because the router decides at the same block size that the kernel processes, the kernel can truly skip unneeded blocks. That cuts the part of the attention cost that grows with rollout length. A coarser or finer unit would leave the saved work unrealized on the GPU. Slab construction also runs asynchronously with denoising on the same H200. The geometry cost is hidden inside denoising time, so it adds no extra delay to generation. The whole method needs no training and leaves model weights unchanged. The two speed figures differ for a clear reason. Denoising contains computation beyond attention that does not grow with history, so the end-to-end gain of 2.43x is smaller than the attention-only gain of 4.36x. The 48th chunk is a revealing observation point, because the history is long there and dense attention is under the most strain.

Industry Impact

Many long-context accelerators rely on generic sparsification or compression. Velox instead uses structure that is specific to world models: scenes have 3D geometry, cameras have poses, and past content has a definite spatial relationship to the present view. Deciding where to look by spatial memory has firmer grounds than choosing by attention score or recency alone.

The engineering profile is also friendly. The method is training-free, leaves weights untouched, and aligns with tiled attention kernels. A video world model that is already trained may gain faster long rollouts at low cost. For teams building real-time interactive simulation, game-like scenes, or embodied-agent training environments, a rollout that slows as it lengthens is a practical barrier. Velox is aimed at that barrier.

The asynchronous design offers a second lesson. An auxiliary module that computes geometry beside the main model can have its cost hidden by pipelining. Designers of other add-on routers can reuse that pattern. The quality result, a gap of at most 0.15 on the API-free average over 158 navigation cases, suggests that skipping many past blocks costs little in navigation performance.

Outlook

Several limits deserve attention. The source summary gives no ablation details. The separate contributions of frustum slabs, multiwarp, and soft modulation therefore cannot be judged here, and readers should consult the paper or its project page. The speed figures were also measured on an H200 with asynchronous slab construction. Other hardware may show different gains.

The method also depends on depth from streaming geometry. How robust Velox is when that depth is wrong remains an open question. So does its behavior in rollouts much longer than 48 chunks. These are natural directions for follow-up work.

Still, the direction is clear. As video world models move toward interactive use, memory cost per chunk becomes a first-order concern. Velox shows that geometry already present in the pipeline can steer attention, and that doing so can cut latency at the 48th chunk by more than half in denoising while keeping navigation scores close to the dense baseline.