Interpretability

Artificial Intelligence / 18 Sep 2026 / 5 min read

Treat the Logit Lens as a Readout, Not a Causal Trace

A transformer can expose an intermediate residual state that strongly favors a token under the model’s final vocabulary projection, then produce a different token after later blocks run. The logit lens makes that intermediate preference visible. It does not establish that the preference caused the final output. That boundary matters when developers use layer-by-layer token rankings to inspect model behavior. The logit lens is a readout: it asks what the model’s output head would report if applied to an intermediate representation. The actual forward pass asks a different question because every remaining block can transform that representation before the final readout.

Artificial Intelligence Transformers

Artificial Intelligence / 16 Sep 2026 / 6 min read

Test Transformer Mechanisms with Activation Patching

A transformer can produce two different outputs from prompts that differ in one relevant detail, yet inspection of attention weights or hidden-state similarity does not establish which internal states actually matter for that difference. Activation patching addresses a narrower question by intervening on a forward pass: replace a selected activation with the corresponding activation from another run, then measure how the output changes. The result is causal with respect to that intervention. It does not automatically identify a complete circuit, a unique mechanism, or a human-readable feature. That boundary is central to using patching results correctly.

Artificial Intelligence Transformers

Artificial Intelligence / 16 Sep 2026 / 5 min read

Read Intermediate Transformer States With Logit Lens

A decoder-only transformer produces its next-token distribution only after the final hidden state has passed through the model’s output normalization and vocabulary projection. Logit lens reuses that output path on states from earlier transformer blocks. The result is a sequence of vocabulary distributions that can expose how token preferences change with depth. The method is attractive because it maps internal vectors into familiar token space without fitting a separate classifier. That convenience also creates a sharp interpretive boundary: an intermediate state was not necessarily optimized to be directly decoded by the final output map. A readable token distribution is a probe of that state, not a guarantee that the model has already settled on the same prediction.

Artificial Intelligence Transformers

Artificial Intelligence / 16 Sep 2026 / 6 min read

Read Intermediate Transformer Predictions with Tuned Lenses

A transformer can carry useful information about its eventual next-token distribution several blocks before the final layer. Reading that information is not as simple as applying the model’s output projection to every intermediate hidden state. The final output head is calibrated for representations at the end of the network, while residual representations can shift across depth. A tuned lens addresses that mismatch with a separate affine translator for each inspected layer. The translator maps an intermediate residual state into a representation that the frozen final normalization and output projection can decode. This produces a token distribution that can be compared across layers without assuming that every layer already uses the final representation basis.

Artificial Intelligence Transformers

Artificial Intelligence / 13 Sep 2026 / 6 min read

Inspect Intermediate Transformer Predictions with Logit Lens

A transformer produces its next-token distribution only after the final block, yet every block updates the residual state that eventually feeds that prediction. Logit lens examines those intermediate states by mapping them through the model’s output path into vocabulary logits. The result is a sequence of provisional token distributions across model depth. The method is attractive because it reuses components already present in the model. Its output also needs careful interpretation. An intermediate residual state was not necessarily optimized to behave like a final residual state, so a readable token ranking is a diagnostic projection rather than a direct transcript of internal computation.

Artificial Intelligence Transformers

Artificial Intelligence / 12 Sep 2026 / 8 min read

Test Transformer Circuits with Activation Patching

A transformer can expose a clear internal pattern without that pattern being responsible for the output under inspection. Activation patching addresses this gap by changing an internal state and measuring the downstream effect. Instead of asking whether a feature is visible at a layer, it asks whether replacing a selected state changes a defined model behavior. The method is simple in form but sensitive to experimental design. A patch has meaning only relative to the paired inputs, the patched location, the replacement value, and the output metric. Changing any of those can change the causal question being tested.

Artificial Intelligence LLM

Artificial Intelligence / 12 Sep 2026 / 7 min read

Inspect Transformer Layer Predictions with the Logit Lens

A decoder-only transformer normally exposes token logits only after its final block and output normalization. The residual stream inside earlier blocks has the same model-width shape, which makes another operation possible: take an intermediate state, apply the model’s output-side normalization when required, and project that state through the output matrix. The resulting vocabulary scores form the logit lens. They provide a token-space view of an internal representation before the remaining transformer blocks have processed it. That view is useful for inspecting how candidate tokens change across depth, but it is not a record of tokens that the model has secretly selected in advance.

Artificial Intelligence LLM

Artificial Intelligence / 12 Sep 2026 / 9 min read

Control LLM Behavior with Activation Steering

Control LLM Behavior with Activation Steering Prompting controls a language model through its input tokens. Fine-tuning changes model parameters. Activation steering offers a third option: change selected internal activations while the model runs, without rewriting its weights. That makes activation steering useful for experiments where you want to test whether an internal direction is connected to a behavior, or apply a lightweight behavior shift during generation. It also creates new engineering questions. A steering vector can help at one layer and damage output at another. A strength that works on short prompts can become excessive on different inputs. A behavioral shift can also come with losses in fluency or task accuracy.

Artificial Intelligence LLM

Artificial Intelligence / 08 Sep 2026 / 9 min read

Trace Token Influence with Attention Rollout

Looking at one Transformer attention matrix can answer a local question: which positions a token attends to in that layer. It does not directly tell you how much an input token can influence a representation several layers later. The reason is mixing. After one layer, a token representation already contains information gathered from other positions. The next layer attends to those mixed representations, not to untouched input tokens. Residual connections add another path that carries each representation forward. Reading only the final layer therefore skips the paths through earlier layers.

Artificial Intelligence Transformers

Artificial Intelligence / 07 Sep 2026 / 10 min read

Probe Neural Network Representations with Linear Classifiers

A neural network can produce the right output while leaving an important engineering question unanswered: what information exists inside its intermediate representations? Suppose an image classifier predicts product categories. You may want to know whether an early layer already separates shapes, whether a later layer distinguishes categories, or whether a supposedly irrelevant attribute such as camera source remains easy to recover. Looking only at the final prediction does not answer those questions.

Artificial Intelligence Machine Learning

Artificial Intelligence / 06 Sep 2026 / 9 min read

Inspect Transformer Predictions with the Logit Lens

A transformer language model produces its next-token prediction only after many layers of computation. When that prediction is wrong or surprising, developers often want a more specific question answered: how did the model’s candidate tokens change as the input moved through the network? The logit lens is a simple interpretability technique for exploring that question. Instead of waiting for the final layer, it takes an intermediate representation and passes it through the model’s final decoding machinery to obtain vocabulary logits. Repeating this across layers gives a rough view of how token predictions evolve with depth.

Artificial Intelligence Language Models