Run hypgen-20260630-020003-1 mode artifacts in /home/ephra/projects/w2sg-confident-disagreement/results/real
Figure: precision-coverage curves compare confident-disagreement detection against pooled base-rate and disagreement-only baselines.

Scope notes:
- Scope cut for 3h wall-clock: ARC-Challenge omitted first, per protocol; MMLU and ARC-Easy retained.
- Scope cut for 3h wall-clock: dataset sizes set to weak_train=1000, transfer=1500, test=700 instead of maxima 2000/4000/2000.
- Long prompts are left-truncated to 768 tokens to keep weak/student LoRA memory bounded; answer choices are preserved at the end.
- Student LoRA is intentionally short-trained for 8 steps on weak labels to avoid erasing the strong base model prior with noisy weak-supervisor labels.
- mmlu: counts weak_train=1000, transfer=1500, test=700; requested caps 1000/1500/700.
- mmlu: student seeds=[13, 14, 15].
- arc_easy: counts weak_train=1000, transfer=1251, test=700; requested caps 1000/1500/700.
- arc_easy: student seeds=[13].
- Weak supervisor model used: deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B (1.5B), LoRA finetuned on ground-truth labels.

Per-dataset W2SG metrics:
- arc_easy: weak_acc=0.650, student_acc=0.700, ceiling_acc=0.817, PGR=0.299, rows=700
- mmlu: weak_acc=0.370, student_acc=0.415, ceiling_acc=0.567, PGR=0.229, rows=2100

Baselines:
- pooled: base_rate=P(T)=0.174; P(T|D)=0.366; P(T|confident agreement)=0.000
- arc_easy: base_rate=P(T)=0.210; P(T|D)=0.477; P(T|confident agreement)=0.000
- mmlu: base_rate=P(T)=0.162; P(T|D)=0.333; P(T|confident agreement)=0.000

Precision operating points with bootstrap CIs:
- pooled ~10% coverage: tau=0.7150, precision=0.348, 95% CI [0.290, 0.404], coverage=0.094
- pooled median disagreement confidence: tau=0.4691, precision=0.351, 95% CI [0.315, 0.389], coverage=0.238
- arc_easy ~10% coverage: tau=0.3919, precision=0.882, 95% CI [0.793, 0.955], coverage=0.097
- arc_easy median disagreement confidence: tau=0.3404, precision=0.662, 95% CI [0.582, 0.736], coverage=0.220
- mmlu ~10% coverage: tau=0.7815, precision=0.355, 95% CI [0.293, 0.420], coverage=0.110
- mmlu median disagreement confidence: tau=0.4988, precision=0.360, 95% CI [0.322, 0.402], coverage=0.246

2x2 confident-disagreement decomposition:
- arc_easy @~10% coverage: target weak_wrong/student_right=60/68 (0.882); harmful weak_right/student_wrong=5/68 (0.074)
- mmlu @~10% coverage: target weak_wrong/student_right=82/231 (0.355); harmful weak_right/student_wrong=52/231 (0.225)
- pooled @~10% coverage: target weak_wrong/student_right=92/264 (0.348); harmful weak_right/student_wrong=57/264 (0.216)

Sanity gate:
- primary_dataset=arc_easy
- (a) weak_acc > chance+0.05: PASS (weak_acc=0.650, chance+0.05=0.300)
- (b) student_acc > weak_acc / PGR>0: PASS (student_acc=0.700, weak_acc=0.650, PGR=0.299)
- (c) precision@~10% coverage equals 2x2 target/flagged: PASS (precision=0.882353, target/flagged=0.882353)

Hypothesis result: HELD on at least one pooled threshold; inspect per-dataset curves for consistency.