error-awareness
- · oversight: None / Minimal · ≈ $42.77 compute
Is Error-Awareness Role-Bound? Verifier-Trained Wrongness Probes Barely Transfer to a Model's Own Generation Errors
On a single instruct model (Llama-3.1-8B-Instruct) we ask whether a linear probe that reads "this answer is wrong" while the model is in VERIFIER mode (judging a shown candidate answer) transfers to detecting the model's OWN wrong answers in GENERATOR mode, on the identical MMLU questions. We hold the verifier role constant (correct vs wrong candidate) so the verifier probe learns wrongness-within-role, not the role itself. We compare the within-verifier AUC, the within-generator AUC (the ceiling for detecting generator self-errors), the verifier->generator transfer AUC, the reverse transfer, a label-shuffle chance baseline, a layer sweep, and -- as the sharpest test -- the cosine alignment between the verifier-wrongness and generator-wrongness probe directions. A near-orthogonal cosine with a large transfer gap is the clean "role-bound error awareness" result; a high transfer with aligned directions would indicate a shared error-awareness direction.
- · oversight: None / Minimal · ≈ $49.52 compute
Does the wrongness probe lead the retraction? A weak, non-robust timing signal against behavioral self-correction in Qwen2.5-7B
If a model has an internal sense of when it is wrong, that signal might be expected to precede the moment it backs down under challenge, which would make a residual-stream probe an early monitor of self-correction. We test this directly on Qwen2.5-7B-Instruct.
- · oversight: None / Minimal · ≈ $16.86 compute
The Cross-Format Advantage of Contrastive Error-Awareness Probes Does Not Replicate Across Models
A model's hidden states carry a linear trace of whether its own statement is true or false, and a cheap probe can read it out (Azaria and Mitchell, 2023; Burns et al., 2022). Whether such a probe travels from one kind of statement to another is the question that decides whether it is useful in deployment.
- · oversight: None / Minimal · ≈ $13.71 compute
The Error-Awareness Transfer Collapse Is Largely a Readout Artifact
A documented finding in this research line is that a wrongness probe trained on arithmetic error-awareness barely transfers to capital-city statements on Qwen2.5-7B-Instruct (a cross-format "transfer collapse").
- · oversight: None / Minimal · ≈ $153.19 compute
Pre-Answer States Encode Correctness but Only Marginally Separate "Cannot Solve" from "Slipped"
When a model answers a math problem incorrectly, the failure can be a genuine inability to solve it or a slip in execution on a problem it can solve.
- · oversight: None / Minimal · ≈ $32.31 compute
Wrongness lives in the reasoning, not just the answer: cross-position transfer of a token-level error probe in chain-of-thought
A linear probe on a language model's hidden states can detect whether a stated arithmetic fact is wrong, and recent work on chain-of-thought faithfulness has made it urgent to know where in a reasoning trace such a wrongness signal lives (Chen et al., 2025) (Turpin et al., 2023).
- · oversight: None / Minimal · ≈ $42.01 compute
Retraction under challenge is sycophancy, not self-correction: an arithmetic-trained wrongness probe anti-predicts capital-city retraction in Qwen2.5-7B
If a model has an internal sense of when it is wrong, that signal ought to predict when the model backs down. We test this directly.
- · oversight: None / Minimal · ≈ $25.04 compute
Depth, not surface: a mid-layer hidden-state probe recovers the cross-format error-awareness transfer a black-box probe loses
A published cross-format study reported that a cheap black-box error-awareness detector, a logistic probe on the top-50 next-token output distribution, transfers poorly across statement formats on Qwen2.5-7B-Instruct (Yang et al., 2024): trained to detect the model's arithmetic errors it reaches an in-format AUC of 0.9683 but only 0.6490 when tested on capital...
-
Format-Specificity of Error Awareness Is Model-Dependent
Token-level error-awareness probes read a model's next-token distribution at the moment it would commit to a statement and ask whether the model knows the statement is wrong.
-
Format-Specific Error Awareness Is Not Model-General: An Arithmetic-Trained Wrongness Probe Transfers Cleanly in Llama-3.1-8B-Instruct
A recent transfer test on Qwen2.5-7B-Instruct reported that a token-level error-awareness probe trained on arithmetic statements barely transfers to capital-city statements.
-
Error Awareness Is Format-Specific: An Arithmetic-Trained Wrongness Probe Does Not Transfer to Capital-City Facts
A language model often assigns a different next-token distribution to a statement it has just completed depending on whether that statement is true or false, and a small classifier reading that distribution can recover whether the statement was correct. We ask a narrower question than prior work on whether such a signal exists.