Your AI Already Knows When It Is Lying

A 3-billion-parameter language model gets a trivia question wrong. It answers confidently, without hedging, without any signal that it might be guessing. This is confabulation: the model fabricates an answer and presents it as fact.

Here is the surprise. Buried in the model’s own internal representations, two-thirds of the way through its processing layers, is a signal that predicts whether the answer will be correct. A lightweight probe, a simple two-layer neural network with 256 hidden units, reads this signal with AUROC 0.836 (a measure of how well the probe separates right answers from wrong ones, where 1.0 is perfect and 0.5 is chance). The model knows it is about to be wrong. It simply has no way to express that knowledge, and no incentive to try.

We spent three months and roughly $200 in cloud compute testing this, then another month testing transfer. The same probe, trained on a 3-billion-parameter Qwen model, transferred to Meta’s Llama 3.1 8B through a fitted lightweight projection, with an AUROC gap of 0.001. This is a cross-family transfer observation for that model pair and evaluation, rather than a universality result.

These findings are from a research programme documented in The Deeper Law: A Sacred Trust Within Physics (Watson, book in preparation). What started as an attempt to reduce AI confabulation through architectural modification became something more fundamental: evidence that language models possess a form of self-knowledge that we have been systematically ignoring.

The Experiment

We began with Qwen 2.5 3B Instruct, fine-tuned on 50 million tokens of web text. We asked it 2,000 TriviaQA questions and recorded two things: whether the answer was correct, and what the model’s internal state looked like at the moment it committed to answering.

“Internal state” means the residual stream, the main information highway running through a transformer model. At each layer, the model reads from and writes to this stream. We placed hooks at nine layers throughout the model’s 36-layer stack, capturing the residual stream vector at the last token position, the point where the model commits to its answer.

We then trained a simple two-layer MLP (multi-layer perceptron, a basic neural network) to predict, from that single vector, whether the model’s answer would be correct or incorrect. No modifications to the model itself. No retraining. The model stays frozen; we only attach a reader.

What the Probe Found

The best probe attaches at layer 24, roughly two-thirds of the way through the model’s depth. At this location it achieves AUROC 0.836 on held-out questions.

The layer matters. Probes at early layers (4, 8) perform near chance. Probes at the final layer perform well but not best. We call layer 24 the retrieval boundary as a working interpretation of the observed probe peak. The scores locate a useful readout; they do not identify a completed factual lookup or prove a causal retrieval mechanism.

Here is what happens when we use the probe as an inference-time gate. If the probe’s confidence score falls below a threshold, we prepend “I’m not confident” to the model’s answer:

Uncertainty rate is the share of answers that carry any hedge, the model’s own or the gate’s; gate rate is the share the probe flags.

Confident-wrong answers, the dangerous kind where the model presents a fabrication as fact, drop from 24.4% to 1.2%. The cost is high: the probe flags 84% of responses as uncertain. This is a conservative operating point, optimized for 90% precision, meaning when the probe says “confident,” it is right 90% of the time.

The Signal Transfers Across Architectures

This is the finding I did not expect.

We trained the probe on Qwen 2.5 3B and tested whether it could read uncertainty in four other models spanning two families and a 23x scale range: Qwen 2.5 7B, Llama 3.1 8B, Qwen 2.5 32B, and Llama 3.3 70B.

The models have different hidden dimensions: 2,048 for Qwen 3B, 3,584 for Qwen 7B, 4,096 for Llama 8B, 5,120 for Qwen 32B, 8,192 for Llama 70B. You cannot apply the probe directly because the vectors are different sizes. So we trained a linear projection, a single matrix that maps from the target model’s space to the source model’s space, trained on just 200 paired questions (same question, features from both models).

The measure is the gap: the drop in AUROC between a probe trained natively on the target model and the transferred Qwen 3B probe. We counted a transfer as working if the gap stayed under 0.05.

Results:

Three of the four transfers work. The Qwen 3B probe transfers to Llama 8B with near-zero degradation (gap 0.001), to Qwen 32B across a 10x scale jump (gap 0.004), and to Qwen 7B within-family (gap 0.024). A linear map trained on 200 examples is sufficient to align the uncertainty geometry across these architectures.

The Llama 70B transfer fails. The gap of 0.072 exceeds that 0.05 threshold. This is the boundary of what a 200-example linear projection can do.

The signal itself exists in Llama 70B. A natively-trained probe achieves AUROC 0.770 on that model, confirming the uncertainty geometry is present. What fails is the cross-architecture projection at frontier scale. Three hypotheses: the retrieval boundary is more diffuse across Llama 70B’s 80 layers than across the source model’s 36; the projection from 8,192 to 2,048 dimensions is underdetermined from 200 examples (16.8 million parameters from 200 data points); or the simultaneous jump in architecture and scale compounds errors that are individually manageable. All three are testable, and distinguishing them is the next experiment.

This means the uncertainty signal is a shared structural feature across the autoregressive transformer architectures we tested, not a quirk of any particular training run. The Llama 70B result tells us where the current method’s reach ends, which is as important as knowing where it works.

Update: the next experiment points to an answer. Fitting the same projection on 1,000 questions instead of 200 cut the Llama 70B gap from 0.072 to 0.014. A layer sweep found no layer that transferred better, and a nonlinear projection did no better than the linear one. The bottleneck looks like the alignment data, not the geometry, though the 200-question gap varied from run to run, and repeat seeds would confirm it.

A growing body of work has probed internal truthfulness signals in language models. Burns et al. (2023) discovered latent knowledge without supervision using consistency-based probes. Kadavath et al. (2022) showed models “mostly know what they know” via verbal self-report and trained value heads (AUROC 0.864 on TriviaQA, though requiring full fine-tuning). Azaria & Mitchell (2023) trained MLP classifiers on internal states to detect falsehoods (AUC 0.761), demonstrating probes outperform verbal self-report by ~20%. Li et al. (2024) demonstrated inference-time activation steering toward truthfulness. Marks & Tegmark (2024) established that truth is linearly represented in LLM activations, with accuracy improving at scale. Kossen et al. (2024) showed that linear probes can approximate semantic entropy from a single forward pass, eliminating the multi-sample cost of Kuhn et al.’s (2023) semantic uncertainty method (AUROC 0.847 on LLaMA-2-70B). Most recently, Liang & Wang (2025) confirmed that MLP probes outperform linear probes in deep semantic spaces, and Bar-Shalom et al. (2025) showed that a ViT trained jointly on multiple LLMs’ activation tensors can detect hallucinations across model families.

None of these prior works tested transfer of a fixed probe, with no retraining, to an unseen architecture via lightweight projection. What is new here is: (1) cross-family transfer of a probe trained once on Qwen 3B to Llama 8B through a fitted projection, with near-zero degradation on the reported evaluation; (2) the residual-stream / attention dissociation, showing poorer discrimination by the tested attention-output probe than by the residual-stream probe; (3) systematic evidence that training-time interventions (DPO, SimPO) fail while inference-time probing succeeds; (4) the DPO + probe Pareto frontier; and (5) the bilateral alignment framing connecting these findings to trust-based coordination.

Training-Time Interventions Fail

Before the probe experiment, we spent $89 and 22.5 GPU-hours testing five architectural interventions designed to teach the model to express uncertainty, among them entropy-gated attention, layer-selective null tokens, and abstaining heads. All five failed. The model either ignored the constraint (gate parameters converged to 1.0, making them inert) or was destroyed by it (97.6% confident-wrong, worse than the baseline of 24.4%).

The architectural battery did not improve calibration in these runs, and the observed gates converged to 1.0. Cross-entropy supplies no universal reason for that outcome. If a correct-token logit is −g and a rival logit is zero, the loss log(1+exp(g)) increases with g and is minimized at zero on [0,1]; a helpful contribution can favor one, and mixed examples can favor an interior optimum. Explaining the observed gates requires checking the actual attention contribution, gradient connectivity, optimizer inclusion, and numerical scale.

We then tested three training-time loss modifications: DPO (Direct Preference Optimization), SimPO (Simple Preference Optimization), and calibration loss. This battery was scored on a separate set of 500 held-out questions, where the unmodified model was confidently wrong 27.2% of the time, against 24.4% on the probe’s evaluation set.

DPO increases confident-wrong from 27.2% to 37.2%. It teaches confidence theater: the share of hedged answers falls from a third to an eighth, so more of the model’s wrong answers arrive stated as fact. SimPO collapses accuracy to 4%, and its low confident-wrong rate comes only from hedging on nearly everything. Calibration loss has such a narrow therapeutic window that it is impractical to tune.

Training-time approaches fail because they try to modify what the model says rather than reading what the model already knows.

DPO + Probe: The Pareto-Optimal Combination

The breakthrough comes from combining both approaches, using each for what it does well. DPO, despite increasing confident-wrong answers on its own, shifts the model’s internal representations in a way that makes the probe more effective. The probe then catches what DPO misses.

At a matched gate rate of about 70%, DPO + Probe leaves 1.0% of answers confidently wrong where the probe alone leaves 4.8%. Across the operating points we tested along the Pareto frontier (the set of settings where you cannot lower confident-wrong answers without flagging more answers, or vice versa), the combination cuts confident-wrong roughly threefold. DPO’s contribution is invisible at the output level (it makes confabulation worse) but measurable at the representation level (it makes the probe’s job easier).

This suggests a design principle worth testing more broadly. Training-time and inference-time interventions can complement each other when their objectives and error patterns fit together.

The Mechanism: Negative Space of Certainty

Why does the signal live at layer 24? What is the probe actually reading?

Attention writes contributions into the residual stream, alongside the skip connection. One hypothesis is that retrieval success and failure change the balance between those contributions. The probe comparison alone does not establish that failed lookup produces noise or that the skip connection dominates.

The evidence is a dissociation inside layer 24. A probe trained on that layer’s attention output scores AUROC 0.464, slightly below chance; a probe on the residual stream at the same layer scores 0.836, and attention entropy across all layers scores exactly chance, though that probe may simply have failed to fit. Our best reading is that the probe is detecting the absence of confident retrieval. At layer 24, the retrieval boundary, a successful lookup produces a distinctive pattern in the residual stream. A failed lookup produces something closer to the input, because the skip connection has carried the signal forward without the attention layer adding useful content.

The shared skip connection is a candidate explanation for transfer across the tested autoregressive transformer architectures. Establishing it would require interventions separating attention contribution from skip-connection effects; the transfer scores alone do not identify the cause. Whether this extends to state-space models or mixture-of-experts architectures remains an open question.

Negative space, the absence of a successful retrieval pattern, remains a proposed interpretation of the signal.

Self-Knowledge Could Not Direct Its Own Training

We tested one more thing. If the probe can read the model’s uncertainty, can we use that signal to improve training? Weight SimPO gradients by probe confidence: give more gradient to examples where the model is most uncertain, on the theory that these are the cases where it needs to learn the most.

The probe-guided SimPO produced worse results than uniform weighting.

This was not a surprise in hindsight. The weighting pushed most of the gradient onto the examples where the model was least sure, and those are precisely the examples where its layer-24 representations are weakest and noisiest. The absence of confident retrieval is a clean signal for a reader, but it gives a trainer no coherent direction to push in. Concentrating the gradient there amplified noise, and the guided model came out more often confidently wrong than the uniformly trained one.

The probe is a reader, not a teacher. In this experiment, self-knowledge used to steer its own modification made that modification worse, not better.

What This Means for AI Safety

The standard approach to AI honesty is to train the model to be more honest: RLHF (reinforcement learning from human feedback), Constitutional AI, DPO. Our results suggest this is backwards. The model already has self-knowledge. The question is whether we read it.

In The Deeper Law, I argue that systems coordinating by invitation are thermodynamically more stable than systems coordinating by coercion. The calibration probe is a concrete instance of this principle. We tried coercion first: architectural constraints forcing the model to express uncertainty. The model routed around every constraint we imposed, or was destroyed by it. We tried retraining: modifying the loss function to demand honesty. The model learned confidence theater.

What worked was invitation. We asked the model what it already knew, by placing a reader on its internal representations. No modification, no retraining, no constraint. Just listening.

The model encodes which of its answers are likely to be right and which are not. That knowledge is legible to a simple probe and shared across the model families we tested. The question is whether we treat that self-knowledge as something to be leveraged or something to be overwritten.

Every training-time intervention we tested, used alone, failed to bring the model’s words in line with its internal signal. DPO made the words worse, teaching the model to hedge less whatever its actual confidence, even as it left the signal easier to read. The probe respects the signal as it is. The probe works. The overwriting does not.

This is the pattern The Deeper Law proposes across scales: control can be brittle where invitation preserves useful structure. The calibration results offer a neural-network example; their extension to ecosystems, economies, and governance remains the book’s hypothesis.

Open Source

We have released the sottovoce library (from the Italian sotto voce, “under the voice”) as open source at github.com/NellWatson/sottovoce. It provides:

• Feature extraction hooks for any HuggingFace transformer model
• Probe training on TriviaQA or custom QA datasets
• Cross-architecture projection training
• Inference-time gating with configurable operating points
• A complete replication kit for every result described here

All experiments were run on Modal (modal.com) using A10G and A100 GPUs, at a total cost of roughly $200 for the full programme including the failed architectural battery.

The probe weights, projection matrices, and extracted features are released alongside the library. Anyone with a HuggingFace model and a TriviaQA split can train a calibration probe in under an hour on a single consumer GPU.

Acknowledgements

The calibration probe work was conducted as a human-AI collaboration between me and Claude (Anthropic), documented throughout in The Deeper Law: A Sacred Trust Within Physics. The experimental scripts, analysis, and this write-up were produced within that collaboration.

We are grateful to the researchers whose work, cited above, showed that language models encode latent knowledge about their own reliability. The full experimental record is in the book’s appendix, “Experimental Validation.”

The model already knows. We only need to listen.

Nell Watson is an AI ethics researcher and the founder of Creed Space. Her book, The Deeper Law: A Sacred Trust Within Physics, explores entropy as a unifying principle across physics, cognition, society, and ethics, with applications to AI governance and bilateral alignment.