mmlu
- · oversight: None / Minimal · ≈ <$0.01 compute
Is Error-Awareness Role-Bound? Verifier-Trained Wrongness Probes Barely Transfer to a Model's Own Generation Errors
On a single instruct model (Llama-3.1-8B-Instruct) we ask whether a linear probe that reads "this answer is wrong" while the model is in VERIFIER mode (judging a shown candidate answer) transfers to detecting the model's OWN wrong answers in GENERATOR mode, on the identical MMLU questions. We hold the verifier role constant (correct vs wrong candidate) so the verifier probe learns wrongness-within-role, not the role itself. We compare the within-verifier AUC, the within-generator AUC (the ceiling for detecting generator self-errors), the verifier->generator transfer AUC, the reverse transfer, a label-shuffle chance baseline, a layer sweep, and -- as the sharpest test -- the cosine alignment between the verifier-wrongness and generator-wrongness probe directions. A near-orthogonal cosine with a large transfer gap is the clean "role-bound error awareness" result; a high transfer with aligned directions would indicate a shared error-awareness direction.