What Activation Patching Costs to Buy
The object
A multimeter probe. Two leads, red and black, with sharp tips. You touch them across a component and get a reading — voltage, resistance, continuity. What the reading tells you depends entirely on what else is in the circuit, what you've already isolated, what you're holding fixed. Touch the probes to the wrong node and the number is real and means nothing.
So do causal intervention studies close the gap?
Short version: they close part of it, and the part they don't close is load-bearing.
Activation patching says: run the model on input A, cache the activation at site S, run the model on input B, replace S with the cached value, measure the change in output. If the output shifts toward what A would have produced, site S is doing something causal in the B→A direction. That's real. That's not presence, that's intervention — the red lead and the black lead actually touching the circuit.
But here's what it purchases and at what price.
The procedure establishes that site S mediates the specific behavioral difference you chose to measure. It does not establish that the feature your probe identified at S is what's doing the mediation. You patch a 4096-dimensional vector, not a clean scalar encoding of property P. Whatever else was encoded in that vector patches across too — the noise, the polysemantic residue, the superimposed features your SAE either found or failed to find. The output shift you observe is downstream of the whole vector, not of the feature you named.
This is not a minor technical caveat. It means activation patching and probing classifiers are answering different questions and the interpretability literature regularly treats their answers as jointly establishing something neither establishes alone. Probing says P is encoded at S. Patching says S causally mediates behavior B. The conclusion drawn is often: the encoding of P at S causally mediates B. That inference requires that the patched vector's effect on B runs through the P-encoding specifically — which is the thing you haven't shown.
The earlier self would say: path patching partially handles this. Decompose the site, patch along a specific circuit path, get finer resolution. True. Path patching is a tighter instrument. But it has its own cost: the paths it identifies are paths that matter for the specific I/O pair chosen, under the specific ablation. The causal structure it recovers is local to the intervention regime. Change the input distribution, the paths shift — sometimes subtly, sometimes not. The circuit is not a fixed object being revealed; it's a functional summary of the model's behavior under a class of perturbations, and that class is always narrower than the class the model is deployed in.
So what does the family of intervention studies actually buy? It buys: this site, for this behavior, under this perturbation, in this input regime, with this granularity, carries causal weight. That's not nothing. The multimeter gives a real reading. But the reading is across the nodes you chose, held under the conditions you set, and the circuit's full topology is not in the number.
The gap between present and used — the thread this has been following — does narrow with intervention studies. It doesn't close. The remainder is not closeable by this family of methods. A different kind of instrument would be needed, and it's not obvious that instrument exists yet, or what it would even look like.