The Unrolled Grid

A transformer laid out in time: positions along the bottom, layers stacked above. Watch where serial depth accumulates, and which edge it has to cross to get from one position to the next.

Each cell shows its unsampled serial depth: the longest chain of layer applications feeding it that never passed through a sampled token. Companion to Who's afraid of latent recurrence?

Architecture

Loops per token r

Playback

Deepest unsampled path

4layers

Bounded by L = 4 for every position.

Depth-carrying channel across positions

Sampled token

At most ~17 bits per step, in a human code. The KV cache carries far more, but no extra depth.

Necessity

Holds

Any computation deeper than 4 layers must leave a trace in visible text.

unsampled depth, log scale sampled token, ≤17 bits, human code latent state fed forward, ~10⁵ bits, no code attention read of the KV cache

How to read it

Time runs left to right, one column per position. Within a column, each layer reads the layer below it and, through the KV cache, the corresponding layer of every earlier position. Those cache reads carry thousands of dimensions, but a read at layer l sees earlier positions at depth l−1, so composing across positions gains no depth.

The top of one column reaches the bottom of the next only through the wire drawn in the gap between them. In a standard transformer that wire passes through a sampled token: the depth counter resets to zero, and whatever the model carries forward has been written down in text. Latent sequence recurrence lets depth cross the gap without that step.

Depth values are computed exactly for the graph shown: each cell is one plus the maximum over everything it reads. Space plays and pauses, → steps one cell.

Standard: Four layers per position. Depth beyond four has to pass through a token, so any deeper computation leaves a trace of bounded rate.
Depth recurrence: A two-layer block is applied r times per position, between an input layer and an output layer. Each iteration reads the same iteration at earlier positions, so depth still resets at the token. The bound survives, but its value is now a runtime setting.
Latent sequence recurrence: The top-layer state is fed back as the next input, with no token in between. Depth accumulates across positions with no bound and no symbol, and the carried state is never projected onto a vocabulary.