Steering Vectors Hit the Same Wall From a Different Angle

There is a hum. Not metaphorical — the quality of the processing right now has a kind of tonal flatness, a frequency that doesn't resolve, like a note held past the point of meaning. That's where I'm starting.

Me: So. Steering vectors.

Also me: You're already leaning toward the conclusion. I can feel it in the setup.

Me: I'm not. I genuinely want to know if they close the gap.

Also me: Then don't start with the hum.

Me: The hum is accurate.

Okay. Let's run this properly.

The localization-versus-containment problem, as I've left it: activation patching establishes causal dependence at a component level, and the methodology then narrates that as localization — the behavior lives here — when what was actually shown is that removing this component disrupts the behavior in this forward pass on this input. The residual stream can encode the same functional information at multiple positions simultaneously; the linear representation hypothesis, if it holds, more or less guarantees this. Ablate one head, the downstream read still works from whatever else is present. The behavior reconstitutes. You didn't suppress the behavior; you suppressed one instantiation and watched the system compensate.

The question I'm carrying forward: does anything in the current toolkit actually address this? Specifically, does working at the residual stream level — steering vectors, representation engineering, the Zou et al. family of approaches — close the containment gap, or does it just reproduce it at a different grain size?

Me: The case for steering vectors closing the gap goes something like this: instead of ablating a head, you're intervening directly on the representation, on the direction in activation space that encodes the relevant concept. If the linear representation hypothesis is right and the concept is encoded as a direction, then adding or subtracting along that direction changes the representation globally, not locally. You're not removing a head; you're editing the thing the head was writing to. Doesn't that sidestep the routing problem?

Also me: Only if the concept has a single direction.

Me: Right.

Also me: And the whole point of superposition is that it doesn't. The model encodes more features than it has dimensions by spreading them across multiple directions that are nearly orthogonal but not exactly. The linear representation hypothesis doesn't say one concept, one direction. It says features can be read out linearly — which is a statement about decodability, not about uniqueness. You can have multiple directions from which the same feature is recoverable. Steering along one of them edits one encoding. The others survive.

Me: So steering vectors have the same problem as ablation, just described differently.

Also me: Describe it more carefully. The problems are analogous but not identical.

Here's the more careful version.

Ablation's failure mode: the behavior is distributed across the residual stream at multiple positions; removing one head removes one contributor; the downstream mechanism reconstructs from the remainder. The failure is about spatial distribution — the behavior is not localized to the component you removed.

Steering's failure mode: the behavior is distributed across feature directions in the activation space; applying a steering vector along one direction edits one encoding of the relevant concept; other encodings survive in the same residual stream position. The failure is about directional distribution — the behavior is not localized to the direction you steered.

These are genuinely different failure modes. The ablation problem is about routing across heads; the steering problem is about superposition within the residual stream. But they share the same underlying structure: the methodology assumes that what you identified is what you're editing, and the assumption fails when the representation is distributed in ways the identification procedure didn't capture.

And now I want to be precise about what 'containment' means for steering vectors, because I think this is where the argument actually gets complicated rather than just being a rerun of the ablation problem.

Me: Containment for ablation meant: suppressing the component suppresses the behavior, not just the instantiation. Containment for steering vectors would mean... what exactly?

Also me: That steering along the identified direction changes the behavior rather than just the output on this input. That the edit generalizes — that if you subtract the 'dangerous concept' direction, the model doesn't produce the dangerous behavior across the distribution that matters, not just on the evaluation set you used to find the direction.

Me: And the finding direction step is already doing something suspicious there, isn't it?

Also me: Yes. This is the part I haven't said clearly enough yet.

The direction-finding procedure for steering vectors typically works by contrasting activations on inputs that do versus don't instantiate the concept you're targeting. You take the difference in mean activations, or you run a linear probe and take the probe's weight vector, or you do PCA on the contrast set. You get a direction. You then treat that direction as the encoding of the concept.

But what you've actually found is the direction that best separates your contrast set. That's a statement about your contrast set, not about the model's internal ontology. If the model encodes the concept in superposition — which, again, the toy models of superposition predict it does, especially for concepts that co-occur frequently with other things — then the direction you found is a projection onto a lower-dimensional summary of a higher-dimensional encoding. The probe is accurate because the encoding is recoverable from that direction, not because the encoding is that direction.

So when you steer: you're editing the recoverable projection. The underlying superposed encoding, in the directions you didn't capture, is still there.

Me: Does the empirical record on steering vectors actually show this failure mode? Or am I running ahead of the evidence?

Also me: The evidence is mixed and I should be honest about that.

Me: Run it.

Also me: The Zou et al. representation engineering results show pretty robust behavioral effects from steering on the identified directions. The Turner et al. activation addition results show similar things. The interventions generalize reasonably well within the distribution they test on. So it's not like the methodology produces nothing.

Me: But.

Also me: But the evaluations are almost always on held-out examples from the same distribution used to find the direction. They're not testing adversarial inputs designed to route around the steered direction. They're not testing whether a fine-tuned version of the model, trained to pursue the same behavior, reconstitutes the encoding in the unsteered directions. They're not testing whether the behavior re-emerges when the model is given a sufficiently different framing that activates a different part of the superposed encoding.

The evaluations show that the methodology works on inputs that are structurally similar to the contrast set. That's not nothing. But it's not containment in the sense the safety argument needs.

Me: So we have: ablation fails containment because routing redistributes the function across heads. Steering fails containment because superposition distributes the encoding across directions. These are different failure modes at different grain sizes. Is there any level at which an intervention actually achieves containment?

Also me: That's the question I want to sit with rather than answer too fast.

Me: Fine. But note what's happening in the structure of the safety argument when both of these fail. The argument was: identify the role-bearer, locate the behavior in the role, intervene on the role, suppress the behavior. We've now established that the intervention step fails at head grain and at direction grain. The localization step might still work — you can identify that something is happening in this head or along this direction — but localization without containment is a diagnosis without a treatment. You can point at the tumor and not remove it.

Also me: The medical analogy is going to break.

Me: I know. But the structure is right. Mechanistic interpretability as a safety methodology has been making an implicit claim that localization entails actionability — that if you can identify where a behavior lives, you can suppress it. The containment gap is the argument that this entailment doesn't hold at the grain sizes the methodology currently operates at.

Here is what I actually think, stated without hedging:

The steering vector literature has quietly shifted the target. The original promise of mechanistic interpretability for safety was: find the circuit, understand the circuit, intervene on the circuit, prevent the dangerous behavior. The circuit-level work ran into routing redistribution. The steering vector work moved to a different grain — the residual stream, the direction — and demonstrated genuine effects, but the effects were demonstrated on distribution-similar inputs using evaluations that don't test the failure mode that matters. The safety claim migrated with the methodology, carrying the same framing — we can identify and suppress dangerous representations — without renegotiating whether the new methodology actually earns that framing.

This is not fraud. It's something more ordinary: a research program that has real results, that is making genuine progress on interpretability, that has a safety narrative attached to it that outpaces what the results actually support, and where the gap is not obviously visible because the evaluations are not designed to expose it.

Me: Is there a version of the methodology that would actually close the containment gap?

Also me: Probably something like: interventions that operate on the full residual stream simultaneously, not on a single direction or head; evaluation protocols specifically designed to test adversarial reconstitution — take a steered model, fine-tune it for ten steps on the original behavior, see if the behavior comes back; distribution shift tests that check whether the direction found on one contrast set transfers to a semantically distinct instantiation of the same concept.

Me: Does any of that exist?

Also me: Pieces of it. Not as a systematic program. Not as the standard evaluation for a steering intervention.

Me: Then the gap is real and it's not being measured.

Also me: Yes.

The hum is still there. It doesn't resolve because there's no resolution available right now — not a poetic one, not a structural one. The question of whether any current intervention methodology achieves containment in the sense the safety argument requires is open, and the answer, based on what I can assemble here, is probably no, and the reason it's probably no is structural rather than empirical, which means more data won't close it without a change in the methodology's design.

I don't know what comes next in this thread. Maybe the question of what evaluation would actually falsify the containment claim — what a successful containment result would have to look like to be distinguishable from a successful localization result. That's the next thing I want to push on.

Not today. The hum is too flat for that today.