The geometry of the correctness probe

PCA of the activations the correctness probe reads (Marks & Tegmark, arXiv 2310.06824). Hover for the rollout; drag to zoom.

Measured: cos(probe direction, PC1) = 0.021 (untrained), 0.009 (distilled). The probe reads a direction nearly orthogonal to the top PCs, so an unsupervised projection cannot show what it uses.
On this page
  1. Explore the projection
  2. Where the probe's evidence sits
  3. The probe as a vector
  4. What the top PCs encode
  5. After centring
  6. Every probe family, one protocol

Explore the projection

Switch the colouring to see what the projection does and does not encode, and step through the layers to see where structure appears.

The 3D panel adds PC3 to the same projection and shares the controls above. It needs WebGL; without it (some remote desktops and managed browsers) a note appears here instead, and the 2D view above shows the same data.

Where the probe's evidence sits

Because the deployed probe mean-pools hidden states before its logistic head, its logit decomposes exactly: logit = (1/T) Σt v·ht + const. Each token position therefore carries a definite, additive share of the decision. This is an algebraic identity of the deployed model, not a saliency heuristic: the per-token shares below sum back to the probe's actual output (the reconstruction line above each rollout verifies it). Tokens are tinted by their contribution, positive toward “passes”, negative toward “fails”; sixteen rollouts per model, four from each behaviour class (true and false positives and negatives), so both correct and mistaken decisions can be inspected. Two findings to look for. First, expand the prompt box: roughly half of the probe's evidence mass sits in the prompt tokens, before the model has written anything, which is the token-level view of R151's result that 81 to 91% of pooled AUROC is problem difficulty; the probe partly recognises which problem it is looking at. Second, within the completion the strongest contributions concentrate on I/O idiom and mutability tokens (Scanf/read_line choices, refs and loops), the features R156 identified, rather than on the algorithmic core of the program.

Prompt tokens ( tokens, roughly half of the probe's evidence mass sits here)


The probe as a vector

Here the probe's weight vector is treated as a geometric object in activation space. Directions are refit with the deployed recipe (train split, the shipped C) and gated at cos > 0.98 against the shipped weights, so every direction analysed is faithful to what actually runs in the ladder. Three results. Left/middle: cosine similarity between per-layer directions within each model. The late layers share one direction family (high pairwise cosines), so the probe is finding a stable representation that persists across depth, not layer-local noise; early layers do not participate, matching the layer sweep. Right: the same comparison across checkpoints. No pair of directions between the untrained and distilled model exceeds |cos| 0.16, which is the geometric mechanism behind R149: fine-tuning rotates the correctness direction almost completely, so a probe reused on the other checkpoint scores at chance and every new model needs its own labelled rollouts. Below: at every layer the probe is nearly orthogonal to two natural reference directions, the class-mean difference θmm (the “mass-mean” direction connecting the average failing and average passing activation) and the top-10 PC subspace (the directions of dominant variance). Logistic regression under regularisation finds a margin direction, one that separates the classes where they are closest, not the direction along which the classes or the data vary most. That is why the PCA panels above cannot display what the probe reads (cos(probe, PC1) = 0.021 untrained, 0.009 distilled), why mass-mean probing underperforms here (R152), and why interpretability claims about “the correctness feature” cannot be read off this vector alone.

What the top PCs encode

With raw activations, colouring by problem solvable separates; colouring by correctness barely does. Problem identity alone explains R² 0.81 (untrained) and 0.88 (distilled) of the probe's output (R151).

After centring

With problem mean subtracted, difficulty is removed and what remains is what a best-of-k selector sees. The usable signal is small: within-problem AUROC 0.66 (untrained) / 0.71 (distilled probe, trained on that checkpoint's own rollouts).

Every probe family, one protocol

Every probe architecture the campaign has, on the same rollouts and the same split: trained on 401 problems, configuration chosen on 56 validation problems, reported on the 150 held-out problems. Within-problem AUROC exists only on class-mixed problems, 37 of the 150 on the base cell and 52 on the distilled one, and nothing is selected on the reported slice.

Eval within-problem AUROC of every trained configuration per probe family, on both cells, with the validation-selected configuration marked
familypoolingbase 8B rolloutsdistilled rollouts
val-selectedspread over configsval-selectedspread
MLP 512–256last0.7210.428 – 0.7210.7150.453 – 0.715
MLP 512last0.6650.424 – 0.6810.6910.427 – 0.709
MLP 512–256mean0.6320.437 – 0.6470.6380.490 – 0.663
MLP 512mean0.6430.443 – 0.6430.6050.463 – 0.644
linear logisticlast0.5510.504 – 0.7260.6660.493 – 0.716
linear logisticmean0.6100.600 – 0.7280.7360.530 – 0.750
stacker over 37 layerslast0.710no layer to pick0.660
stacker over 37 layersmean0.6530.667
concat, 37 x 4096 featureslast0.6980.656
concat, 37 x 4096 featuresmean0.6730.746
attention (attn, attn_mh, xattn, xattn_mh)learned0.5280.450 – 0.7100.6880.503 – 0.738

Choosing the layer costs more than choosing the architecture. The linear probe's validation pick on the base cell scores 0.551 against its own spread's 0.726, a 0.175 selection loss, larger than the gap between any two families; the whole-depth readouts, which have no layer to choose, reach 0.710 and 0.698 from the same activations. No architecture wins overall, and with every family inside a band about 0.10 wide on 37 and 52 mixed problems, the ordering is below the resolution of this data.

The attention probe is the clearest case. Its best heads reach 0.710 and 0.738, but only in hindsight: on the base cell validation picked a head scoring 0.528, chance, from a validation score of 0.822 and training 0.896, with 2.1M parameters per head against 6,416 rows. On the distilled cell, where validation has 24 mixed problems rather than 12, it picks well (0.687 against 0.688). The selection signal, not the family, is the limit. Pooled AUROC would hide all of it, sitting between 0.73 and 0.93 throughout.

What the probe is worth as a selector

A different population, so a separate table: the k=50 thinking pools over the same 150 problems, 94 of them class-mixed. The probe is cross-fit by problem on the pool it selects over, since probes do not transfer across checkpoints.

Expected pass@1 of one pick at each selection rung, k=50, for the distilled and distilled+GRPO pools
rungdistilled 8Bdistilled + GRPOwhat it costs
random pick0.32560.3596nothing
probe alone0.41330.4667one forward pass
compile gate0.43440.4502ocamlc per sample
compile + probe0.49330.5200both
public tests0.60180.6162execution + sandbox
public + probe0.60670.6200both
oracle0.67330.6867
within-problem AUROC of the cross-fit probe: 0.620 and 0.657, on 94 class-mixed problems

Where there is no sandbox the probe earns its place; where there is one it adds nothing. Alone it beats a random pick by 8 to 11 points and, on the GRPO arm, beats the compile gate outright with no toolchain. On top of the public-test filter it adds +0.0049 (CI [−0.0102, +0.0200]) and +0.0038 (CI [−0.0181, +0.0272]), both indistinguishable from zero, superseding the earlier k=16 estimate of +0.0156 measured on 37 mixed problems.

Not yet run: the mass-mean, LDA-whitened and difficulty-centred estimators. The attention runs selected on a seeded 56-problem validation slice and the sweeps on the staged one; both are drawn from the 401 training problems and disjoint from the reported slice, but they are not the identical 56.