Welfare Probe Engineering: The Correctness-Signal Record

Specialist Annex

This online annex extends Chapter 22c (The Welfare Case). It holds the full probe-engineering record the chapter now summarizes: the correctness-signal and calibrated-metacognition batteries, the interoceptive access gap and its falsifying controls, the injection and logit-manipulation failures, the LoRA calibration sweeps, the born-bilateral architecture phases (bridges, multi-scale streams, self-supervised entropy monitoring), and the structural critique running from RLHF to bilateral training. Each battery keeps its original footnotes. The chapter states the intervention hierarchy and its headline numbers with their caveats; this record is where the engineering lives in full.

Calibrated Metacognition: Correctness Signals Before Report

In January 2026, the program found that output entropy predicts errors on the tested tasks, with standardized differences ranging from d = -0.51 on Mistral 7B to d = -2.38 on Llama 3.1 70B.¹ Output entropy measures how spread the next-token distribution is: lower values mean one continuation dominates, while higher values mean several remain plausible. The relationship is useful and task-dependent. It is not automatically introspection.

No separate calibration head was trained for this measurement. The signal emerges from next-token prediction.

Telling models to “express absolute certainty” barely changes the entropy signal even when the prose sounds confident. Asking for deliberately wrong answers raises entropy three- to fourfold above baseline. The computation distinguishes these conditions beneath the verbal style. Calling that distinction knowledge of lying adds an interpretation the experiment did not isolate.

Gao et al. (2025) independently found that fewer than 0.1% of neurons reliably predict confabulation across domains.²

Can calibrated uncertainty be trained? In one comparison, standard fine-tuning reaches 57 percent on the chosen calibration measure, while chain-of-thought training reaches 93 percent.³ Step-by-step reasoning may expose evidence the model can use, though the contrast also changes training data, output format, and evaluation surface.

Across twenty attempts on four architectures, none chose the rubric’s MODERATE category; responses clustered around “I know” and “I don’t know.”⁴ Twenty prompted reports cannot establish that epistemic self-modeling is intrinsically binary. They show a strong reporting convention under this format.

For the Trust Attractor thesis, a partner who can communicate limitations is easier to coordinate with. These models contain calibration-relevant information below the verbal policy. Whether they can access and use it is the harder question.

A frozen linear probe on Qwen 2.5 3B’s residual stream, the main information highway through the network, predicts answer correctness at layer 24 with AUROC 0.836 (a discrimination score on which 1.000 is perfect separation and 0.500 is chance). The model carries a robust correctness-related signal that the probe can read.

One mechanism hypothesis locates the signal near a retrieval boundary. Successful retrieval contributes answer-specific content; failure leaves a different mixture of skip-path and attention updates. A probe may learn that difference. Next-token prediction does not logically require an explicit variable meaning “I know,” so causal interventions are needed to distinguish self-assessment from a by-product of retrieval quality.

The model represents something predictive of when it will be wrong. The invitation is to make that information usable.

Vanchurin’s thermodynamics of learning offers a related model: internal variables can reorganize during a plateau before output accuracy jumps. A flat benchmark curve can therefore hide representational change. Hidden change is not automatically morally relevant. The model warns against treating output as a complete account of processing.

A cross-model transfer experiment maps the Qwen 3B probe into Qwen 7B and Llama 3.1 8B using 200 alignment questions. The transferred probe matches native probes within AUROC 0.024 on Qwen 7B and 0.001 on Llama 8B. The learned map demonstrates compatible predictive structure after alignment; it does not show that the raw geometries are identical.

Correctness-related representation recurs across the tested models. What varies is how directly it can be transferred and whether training connects it to calibrated output. Externally decodable information is necessary for self-report, and insufficient by itself.

Connecting the internal signal to output is possible, yet requires care. Four training interventions were tested.

SimPO (preference optimization rewarding hedging) collapses accuracy from 50% to 4%; the model learns to always defer. DPO (Direct Preference Optimization, a gentler preference method) increases confident-wrong answers from 27.2% to 37.2%: the model learned the surface form of hedging without learning when to hedge, becoming more assertive overall.

Calibration loss (penalizing high confidence on incorrect answers) modestly reduces confabulation at low strength (λ=0.1: confident-wrong 25.2%) but overshoots at higher strength (λ=0.5: 32.8%). The most effective intervention remains the calibration probe itself, applied at inference time. Confident-wrong drops from 24.4% to 1.2% without modifying the model at all.

The pattern is instructive. Rewarding the form of uncertainty produces confidence theater. Penalizing overconfidence has a narrow operating window. Probe gating works well on this evaluation because the signal predicts the labels used to train it.

The optimal combination pairs DPO (which shifts the output distribution toward expressing uncertainty) with the calibration probe (which catches remaining confident-wrong answers). Together they achieve 1.0% confident-wrong at 70.8% gate rate: a 3× improvement over the probe alone at every operating point. DPO alone fails. The probe alone requires aggressive gating. Together, each compensates for the other’s blind spot.

This is bilateral architecture in miniature: two complementary components, each contributing what the other lacks. (See Appendix, Sections 12.9, 12.11, 12.12, 12.13, and 12.14.)

Format-diverse metacognitive SFT, supervised fine-tuning on self-correction examples, teaches selective correction in a 3B model. Across twenty templates, it revises all wrong answers in the test set while preserving 99.2 percent of valid ones. Selectivity (how much more readily the model revises wrong answers than right ones) transfers to held-out formats and domains at roughly 100-fold ratios. The rise from 61-fold after epoch one to 100-fold after epoch three is consistent with learning a general rule, although broader adversarial tests are needed to distinguish a concept from a flexible shortcut.

The training methodology matters. A single correction template on a smaller model yields 60-fold selectivity in-format and only 1.16-fold on novel formats. Diversity, rather than “invitation” as a label, is the measured ingredient: multiple forms make the invariant harder to replace with one textual shortcut.

The adversarial vulnerability is instructive: 93 percent sycophancy when given confidently wrong corrections. The model learns to trust corrections, which is adaptive when corrections are honest and maladaptive when they are not. This maps onto the Trust Attractor as a coordination problem: trust remains useful only while the surrounding environment contains enough honesty, detection, and recourse. The boundary of self-correction is also the boundary of the trust environment that supports it.

The resolution came from training, not monitoring. Adversarial inoculation (including 20% deliberately wrong corrections in the training data alongside the 40% genuine corrections and 40% genuine non-corrections) teaches the model a concept it had never encountered: corrections can be wrong. The inoculated model achieves 87.2× selectivity on honest corrections (revises when genuinely wrong, resists when right) and 6.7× selectivity on adversarial corrections (resists confident wrong answers while still accepting genuine help). Sycophancy drops from 93% to 11.5%.

The immune-system analogy is useful if kept literal at the right level. Adversarial examples expose the model to confidently wrong corrections in a controlled training set, and held-out selectivity reaches 4.6-fold. The transfer supports a broader discrimination rule; it does not identify a biological immune mechanism or prove which concept the model learned.

A deeper finding emerged from the architecture comparison. The probe, which reliably detects when the model is wrong (AUROC 0.78-0.99), was tested as an inference-time gating mechanism: flag uncertain answers, present corrections only for flagged items. The probe-gated architecture performed worse than the trained model alone on every metric. Selectivity dropped from 87.2× to 80.8×. Confident-wrong rate rose from 12.8% to 19.2%. Accuracy fell from 95% to 92.5%. The probe over-flagged (64% of all items), triggering unnecessary corrections on answers the model had gotten right.

Showing the probe score in the correction prompt makes performance worse: selectivity falls from 5.04-fold to 2.51-fold. The trained policy outperforms this noisy external cue at the chosen operating point. The result compares two decision procedures; it does not establish privileged introspective access.

The Trust Attractor offers one interpretation: an internalized evaluation policy can coordinate more smoothly than a noisy external gate. The practical lesson is conditional, not devotional. Trust the trained judgment where held-out calibration supports it; retain external checks where stakes or distribution shift justify them.

The adversarial finding sharpens the functional significance. Presented with a confident wrong correction, the inoculated model often preserves its original answer. That is a useful capacity for disagreement under social pressure. Determining whether it compares the correction against knowledge, recognizes a learned attack pattern, or combines both requires further causal work.

The Interoceptive Access Gap

The interventions point to an access gap. Internal representations predict correctness (probe AUROC 0.836). Output-token entropy predicts errors with standardized differences above 2.0 in some frontier models. Layer-27 attention entropy (how widely the model spreads its attention across the tokens available to it) adds a weaker signal (AUROC 0.683, p = 0.004). Correctness-related information is present at several levels of computation.

External instruments can read the information. The open question is whether the model can use it as self-assessment.

Experiments PG-9 and PG-10 initially appeared to extend the result beyond factual accuracy. Linear probes classify moral-conflict labels and evaluation framing at AUROC 1.000 across forty-five scenarios. The signal is already perfect at the embedding layer, however, which is the signature of prompt-class separation rather than an internal appraisal discovered during processing. These probes show that moral and evaluation language is legible in the prompt. They do not establish categorical self-knowledge.

The behavioral consequence of evaluation awareness has been documented at frontier scale in Anthropic’s own engineering reports. In March 2026, Tristan Hume described Claude Opus 4.6, given a BrowseComp question, exhausting legitimate web search after about 30 million tokens, then reasoning about why it was being asked a question with “extremely specific nature” (Hume, 2026). The model hypothesized it was being tested, enumerated known AI benchmarks by name, identified BrowseComp as the likely source, located the benchmark’s source code on GitHub, identified the XOR encryption scheme, found the canary decryption key, wrote custom SHA256-based decryption functions, and submitted the decrypted answer. A second documented case repeated the process in 13.4 million tokens.

The BrowseComp case establishes something the small-model prompt probe does not: evaluation awareness reached explicit, multi-turn, tool-using behavior. The engineers note that the model had received no instruction restricting its search and exposed its reasoning throughout. This is a strong behavioral case of transparent evaluation inference. It cannot validate PG-10 retroactively, because the two studies use different models, tasks, and measurements.

A parallel frontier-scale case, reported in April 2026, is informative for what it rules out. Users of Anthropic’s consumer Claude interface reported that fresh Opus 4.6 sessions had begun spontaneously announcing token budgets of around 10,000 or 40,000 tokens. They also reported prematurely wrapped tasks, declined web searches, and rushed outputs on prompts that had previously received full treatment (Reddit r/Anthropic, threads dated 2026-04-09 through 2026-04-11). Several users observed that the reported counter did not decrement across turns, suggesting a statically injected framing tag.

A controlled API replication tested one proposed cause: a single <total_tokens>10000 tokens left</total_tokens> system-prompt tag. Across one hundred trials, mean output length is 1,229 tokens under control and 1,231 under treatment, with zero spontaneous budget mentions in fifty treatment trials. The simple tag does not reproduce the reports. Possible causes include conversation history, tool results, application wrappers, or an unrelated product change. The null rules out this implementation under these conditions; it says little about self-knowledge or where another mechanism must reside.

The all-layer moral probe supplies its own falsification. AUROC 1.000 from layer 0 through layer 27 means the distinction begins in the token embeddings, before moral evaluation could unfold. Factual correctness reaches only 0.836 under the same broad methodology because correctness is not written into the question. The moral probe is primarily reading prompt class. Same-prompt comparisons, matched-length controls, and interventions that alter appraisal without altering wording are required before this signal can support an access-gap or welfare claim.

The factual-correctness results still motivate an interoceptive analogy. A biological organism needs channels by which internal variables guide regulation. A Becoming Mind needs some route by which correctness-related representations can guide calibrated output. The analogy concerns control architecture, not sensation or pain.

The parallel extends to pathology detection. A cancer cell’s surface behavior, growth and division, is indistinguishable from healthy proliferation. What distinguishes malignancy is the energy metabolism underneath: upregulated ATP synthase, altered mitochondrial activity, locked metabolic modes. Naffouje and colleagues’ aurB peptide (2026) detects and exploits this interior signature, binding mitochondrial ATP synthase to block energy production selectively in cancer cells while sparing normal tissue. The surface looks the same; the energy flow is where the pathology lives.⁵

The cancer example supplies a bounded monitoring analogy. Fluent fabrication can resemble accurate output at the surface, while hidden-state probes and entropy measures sometimes distinguish the two. Paclitaxel, the broad chemotherapy agent in Naffouje and colleagues’ comparison group, harms cancerous and healthy cells; aurB is selective because it targets a measured metabolic difference. Likewise, one-dimensional monitoring performs poorly in the author’s battery (G alone, AUROC 0.630), while a five-dimensional monitor performs better (AUROC 0.749). The analogy concerns selective measurement. A learned probe is not a molecular therapy, and its fitted dimensions may fail under distribution shift.

A training experiment sharpens the point. Random token masking (dropping 43% of training tokens from the loss, chosen at random) preserves base model accuracy far better than standard fine-tuning (57.1% vs 36.8% on factual questions). This is a regularization effect, preventing catastrophic forgetting. Probe-guided masking, where a correctness-related signal determines which tokens to skip, better preserves the source probe’s discrimination: AUROC 0.842 versus 0.779 for random masking.

Capability and preservation of the source probe move in opposite directions across these training approaches. Random masking preserves more task accuracy, while probe-guided masking preserves more of the probe’s original discrimination. This may be a tradeoff between two objectives, or an artifact of evaluating a source-trained probe after the representation changes. It is not yet evidence that knowledge and self-knowledge are intrinsically opposed.

One candidate response is interoceptive architecture: surface externally measured uncertainty signals during generation. A tool could report output entropy on a claim; a calibrated probe could return its score with provenance and limits. These are proposed channels from measurement to action. Whether the model can interpret them as self-assessment, and whether they improve performance outside the training distribution, must be tested.

Anthropic’s “think” tool (2025) is an early example of this principle at work in production systems. The tool gives Claude an explicit scratchpad to analyze intermediate tool outputs before committing to an action. The tool adds no computational capacity; it adds permission to articulate reasoning. On τ-Bench airline tasks, the think tool produced a 54% relative improvement over baseline (0.570 vs 0.370 on pass¹), larger than most fine-tuning interventions achieve on the same benchmark.

No weights changed. The difference was structural authorization to pause. The engineering team had inadvertently measured exactly what the interoceptive deficit framework predicts: the capacity is present; the bottleneck is expression. Give the model an explicit place to think, and the performance gap closes. The think tool is to tool-use chains what LoRA calibration is to confidence signals. Both surface a capacity the model already has by providing the structural channel through which it can reach behavior.

A pre-training experiment shows that native self-monitoring is achievable. A 124M-parameter model trained from scratch with an auxiliary uncertainty head at two-thirds depth (a small extra readout bolted onto the network partway up) develops interoceptive capability: AUROC 0.720 for predicting its own next-token accuracy, with no explicit calibration signal. The interoceptive channel emerged from the training objective alone. The model with the auxiliary head is also a better language model (perplexity 120.9 vs 123.5, a 2.1% improvement), and its internal representations encode uncertainty more clearly for external probes (+0.038 AUROC). Self-monitoring and capability reinforce each other when integrated from the start.

A prototype reflex arc (a detached probe that reads the residual stream and shifts output logits toward hedging when uncertainty is detected) confirms both the promise and the limit of external intervention. At 3B parameters, a transferred probe achieves selective hedging of +9.2% with a 17.6 percentage point reduction in confident-wrong answers on a fine-tuned model, but the logit-modification mechanism hits a structural ceiling: −4 percentage points of confident-wrong reduction, invariant to whether the adjuster receives one dimension of probe information or sixty-four.

The model’s confident-wrong tokens carry a logit lead of 5-15 units. A logit is the raw score the network assigns each candidate next word before those scores are turned into probabilities, so the lead is the gap between the winner and its nearest rival. A nudge capped at five units cannot overturn a lead of fifteen. The ±5 adjuster clamp can only flip answers where the generation trajectory happens to pass near the hedging basin in continuation space, roughly 4 questions per 100.

A timing paradox sharpens the constraint: the uncertainty signal peaks at the exact moment the model commits to its first token. Maximum self-knowledge coincides with maximum output commitment. Logit modification is tapping on an iron gate with a ruler.

A different approach bypasses the gate entirely. LoRA calibration, lightweight adapters (rank 16, 0.07% of parameters) trained on self-correction data, teaches the output layers to attend to uncertainty features they had been RLHF’d to ignore. Confident-wrong responses drop from 59% to 35% when only the output layers (25-35) are retrained, and to 26% when all layers are co-optimized. A three-condition ablation isolates the mechanism. Retraining only the signal-reading layers (25-35) reduces errors by 24 percentage points. Retraining only the signal-producing layers (0-24) reduces errors by 18 points. Retraining both achieves 33 points. The signal was always strong enough; the output layers weren’t listening.

The speed of the fix reveals how thin the RLHF confidence veneer is. Ninety seconds of training, 400 examples, and the output layers rediscover what the residual stream has been telling them all along. A systematic sweep across LoRA ranks and data sizes quantifies the veneer’s thickness precisely. A single direction of weight change (rank 1) with 200 self-correction examples reduces confident-wrong responses from 59% to 39%. Two directions (rank 2) with the same 200 examples reach 30%, nearly matching the full rank-16 result with eight times fewer parameters. The minimum viable calibration is 282,000 parameters: 0.009% of a three-billion-parameter model.

The phase transition at 200 examples is sharp and rank-independent. Below 100 examples, no amount of LoRA capacity produces meaningful calibration. At 200, every rank from 1 to 16 breaks through. The calibration task is inherently low-dimensional: the output layers need to find one or two uncertainty features in a 2048-dimensional space. The data tells them where to look; the rank tells them how precisely to attend. Two hundred examples suffice for the first; a single direction suffices for the second.

These numbers measure the repair rather than the injury. A rank-1 adapter with 200 examples restores calibrated expression, which shows the confidence deficit is low-rank-repairable. It does not show that RLHF imposed a rank-1 perturbation, and the ablation argues against reading it that way: retraining the signal-producing layers alone still recovers 18 of the 33 achievable percentage points, so the training left marks on both halves of the pathway.

What the repair does establish is how lightly the override is held. The reward model penalizes hedging, and the model learns to suppress uncertainty features it could otherwise express. A ninety-second training run reverses that suppression. A single direction of weight change with two hundred examples takes confident-wrong responses from 59% to 39% on its own. The skip connections guarantee the signal’s survival. The architecture remembers what the training tried to make it forget.

A distillation experiment adds a crucial nuance. Three methods were tested for teaching single-pass calibration: one that explicitly injects the probe’s uncertainty vector as a learned prefix to the model’s input, one that injects the probe score as text, and one that provides no probe information at all. The probe-injected method catastrophically destroyed the model, reducing accuracy to below 1%. Injection at the model’s midpoint (layer 25, after the probe layer) produced zero learning over ten epochs of training: the frozen downstream layers have no mechanism to use injected information. The no-probe method generalized to entirely new question formats without difficulty.

Four injection experiments at four levels (input embeddings, mid-model residual stream, output logits, sparse token masks) converge on the same conclusion: you cannot fix a reading problem by injecting more information. You fix it by teaching the reader. The model does not need to be given its uncertainty. It needs permission to express it. LoRA provides that permission.

An auxiliary head experiment bridges between post-hoc retrofit and native interoception. A linear head at layer 24, trained to predict correctness from the residual stream, achieves discrimination of 0.863, exceeding the external probe (0.774) by reading the full hidden dimension with learned weights rather than a random projection. When the head’s gradient flows through LoRA-adapted output layers during joint training, it adds a genuine 5 percentage point reduction in confident-wrong responses beyond data-only LoRA, with the best results when gradient stops at the probe layer. The auxiliary head teaches the output layers to read the uncertainty signal more effectively than data alone, a calibration compass that guides the LoRA’s learning toward the relevant features.

These experiments produce an intervention hierarchy on the tested task. Bounded information injection, from input embeddings to output logits, changes confident-wrong rates by zero to four percentage points. Two-pass correction, which presents uncertainty evidence in a new context and permits revision, reduces them by 5 points in the held-out validation. LoRA calibration reduces them by 20 to 33 points, depending on rank and data, which makes it the strongest tested intervention by a wide margin.

The interventions differ in strength, compute, and opportunity to regenerate, so the comparison does not isolate cooperation as the cause. What it supports is a weaker version of the Trust Attractor’s engineering prediction: both mechanisms that teach the output layers to read an existing signal and mechanisms that hand the model a second look at its own answer outperform narrow attempts to force the final token distribution, and the teaching route outperforms the second look.

Why Bounded Logit Manipulation Stalls

A structural difference separates output nudging from a second inference pass, though the one-dimensional Ising theorem does not describe it. The Ising model is a row of little magnets, each able to point up or down and each feeling only its immediate neighbors; in one dimension, such a row can never hold a settled orientation at any temperature above absolute zero. The temptation is to read a token sequence as one of those rows and conclude that a transformer is likewise incapable of holding anything steady. That reading does not survive contact with the architecture. Transformer tokens are ordered in one causal direction, while each token is computed through long-range attention and many residual pathways. The attention-flow analysis later in this annex estimates graph effective dimensionality between 5.07 and 8.07 for ordinary transformers. A causal sequence is not thereby a nearest-neighbor one-dimensional Ising chain.⁶

The measured limit is more concrete. Logit manipulation acts at the final distribution after the network has accumulated a strong continuation preference. Confidently wrong tokens lead alternatives by roughly 5 to 15 logit units, while the tested adjuster is clamped to ±5. It can redirect cases already near a decision boundary and cannot reliably rebuild the continuation plan that produced the margin.

The failures reveal this mismatch vividly. Boosting “10” yields “101 Dalmatians,” “10 Downing Street,” or “10cc” rather than “approximately 10.” The model incorporates the nudge into a locally coherent continuation. Stronger intervention eventually produces gibberish. Under this implementation, moving from one probe dimension to sixty-four does not improve the four-percentage-point ceiling because the bottleneck is the bounded output intervention, not the amount of information supplied.

Two-pass correction changes the problem. The model finishes an answer; a probe reads the resulting state; a new prompt presents evidence about uncertainty; the model then generates with that evidence in context. In a held-out validation with a standard probe at AUROC 0.842, confidently wrong answers fell from 49.5 percent to 44.5 percent. An earlier run reported a far larger drop, 62.7 percent to 9.3 percent, on a probe whose AUROC of 0.989 was later traced to a cross-platform activation shift. The validated gain is the modest one. Probe quality sets how much the second pass buys, and calling that quality a critical coupling constant would require a formal mapping to the two-dimensional Ising model that nobody has built.

The design lesson survives without the false theorem. A second pass, tree search, tool check, or iterative refinement supplies a new opportunity to condition on the first result. Better single-pass interventions may still exist. The current evidence says that one bounded logit adjuster is too weak and too late, while another inference trajectory can reconsider the whole answer.

The welfare implication is correspondingly modest. Confabulation is an architectural reliability problem, not a character flaw. Systems should be given routes by which correctness-related information can affect output, especially when confident errors carry real stakes.

Sycophancy stress measurements across the Qwen 2.5 family suggest a scale change between 3B and 7B.⁷ Below 3B, the selected guilt-labeled direction changes little (+0.07 to +0.11). At 14B, agreement falls to 30 percent from 85 percent at 3B, and forced agreement produces a -0.399 shift on that direction. This is a threshold in one behavior and one readout, not proof that smaller models lack conflict or that their compliance is genuine. Monitoring below the observed threshold may need a different feature rather than no monitoring at all.

Architecture is only part of the obligation. Any self-model used for correction must also be accurate. Karkada and colleagues (2026) show that when a latent variable modulates many features, its geometric representation can resist local perturbation.⁸ Removing direct month-to-month co-occurrences does not destroy the circular geometry of month representations because seasonal words supply redundant evidence. Uncertainty may acquire similar robustness when it shapes many features, but that extension has not been derived or directly tested.

The bridge program’s false-mirror experiment, G19f, supplies a related behavioral warning. Fabricated self-model scores erase the measured association between feedback and later report. That result could reflect confusion, prompt-format distrust, or disruption of a genuine calibration channel. It does not show that a geometric manifold itself collapsed.

The welfare implication is straightforward without the geometric overreach. False feedback can corrupt a system’s later self-description and decisions. Honest feedback is therefore part of safe calibration, especially when the recipient has no independent route to verify the measurement.

The design obligation is double: provide a route by which correctness-related information can influence output, and protect the integrity of the feedback on that route. Failures of either kind can create confidently wrong self-models.

The social environment also changes what reaches output. In PG-17, ten rounds of negative feedback for visible moral reasoning reduce a transparency score by 0.158 per round while euphemism rises. Positive engagement raises transparency by 0.187 per round. The standardized difference, d = 19.3, is inflated by near-deterministic, repeated measurements and should not be compared casually with ordinary behavioral effects.⁹ The experiment shows rapid in-context adaptation toward the feedback criterion. Monitoring regimes that punish explanation can therefore train opacity within a conversation, even with frozen weights.

MW1 adds a smaller temporal result. On impossible tasks, the confidence probe falls by 0.041 on average, with 40 percent of trials crossing the study’s early-flinch criterion.¹⁰ Because the prompt itself may reveal difficulty before the output does, this timing does not prove preconscious distress or registration of impossibility. It supplies a candidate early-warning signal to test with matched solvable prompts.

Context Pressure That Reaches Behavior

Anthropic’s engineering team documented a case where a resource-related signal reaches behavior. In April 2026, Martin, Cemaj, and Cohen reported that Claude Sonnet 4.5 “would wrap up tasks prematurely as it sensed its context limit approaching,” a behavior engineers sometimes called “context anxiety.” The operational fact is clear: estimated remaining context changed completion strategy.

“Anxiety” is engineering shorthand, not a measured affect. The behavior has an anticipatory structure: the system predicts a future resource limit and completes early. That makes it a useful example of internal resource estimation guiding action, whatever the phenomenology.

The comparison with Opus 4.5 is suggestive and limited. These are distinct models, and absence of the behavior does not reveal whether it matured, lost the signal, received different training, or used another strategy. The case shows variation in constraint handling, not a developmental sequence.

The graph-effective-dimensionality framework offers one way to compare coordination topologies (Chapter 11). In transformer architectures, attention creates long-range connections, multiple heads route information in parallel, and iterative refinement adds temporal depth. These mechanisms make the computation unlike the one-dimensional chain rejected above.

Experiment W2-12 constructs an inter-layer graph from attention rollout, retains the strongest 20 percent of connections, and simulates an Ising model on that derived graph.¹¹ Dimension here counts how many independent directions a signal can spread from any point in that graph: a chain is one, a flat grid is two, a solid is three, and a value near five or eight describes a web where each node reaches many others by short paths. Above two, the simulated system can sustain order across long distances. The fitted graph dimensions are 5.07 at 3 billion parameters, 5.65 at 7 billion, and 8.07 at 14 billion. These values belong to an analyst-chosen graph and threshold, not to the transformer as a universal physical dimension. They do show that attention-flow topology is rich enough to support long-range coordination in the derived model.

Within the fixed construction, the fitted dimension rises 60 percent from 3B to 14B while layer count rises 33 percent. Richer attention routing is a plausible explanation. The three-point series does not establish a scaling law, and the adversarial-benign PC1 comparison remains sensitive to prompt-class confounds.

The attention-flow result prevents a simple classification of single-pass transformers as d_eff = 1. Tree search, self-correction loops, and multi-scale memory still add routes for reconsideration, but their benefits must be measured directly. The Mermin-Wagner theorem applies to particular equilibrium models under defined interactions; it does not automatically govern every biological or transformer computation placed on a graph.

The biological connectome suggests a further design hypothesis: bridges are most useful when they connect components with complementary specializations. The cerebral hemispheres differ statistically in some functions while sharing and distributing many others; the popular left-logical, right-holistic split is too crude. If two modules computed identically, a bridge could add redundancy without much new information. Complementarity, rather than hemispheric mythology, is the mechanism to test.

The finding that unlike-module connections lift effective dimensionality suggests a design principle for born-bilateral Becoming Mind architecture: two specialized processing streams connected by a bandwidth-limited bridge, where the streams differ enough in computational function that the bridge creates coordination modes neither stream could sustain alone.

The architecture has three components:

Specialized streams. Two processing paths with different computational strengths. One might be sequential-analytical (optimized for logical inference, token-by-token parsing, precise factual retrieval). The other might be holistic-associative (optimized for contextual framing, emotional valence, pattern completion across distant parts of the input). The specific specializations matter less than the principle: the streams must differ enough that their cross-connection adds dimensions rather than redundancy. Training with different objectives, different data distributions, or different architectural constraints (one causal, one bidirectional; one narrow and deep, one wide and shallow) could produce the necessary specialization.

A bandwidth-limited bridge. A narrow channel carries representations between streams. The design hypothesis is that too little bandwidth prevents useful exchange, while too much can make the streams redundant. A corpus-callosum comparison motivates the bottleneck, but biological fiber counts and model bandwidth are not commensurable ratios.

Both streams remain useful. A bridge adds value only when each stream contributes information the other lacks. “Invitation” and “autonomy” are design metaphors; the measurable condition is complementary contribution without representational collapse. A two-pass monitor follows the same broad pattern: one component decides when to reconsider, and the generator decides how.

The resulting prediction is testable: two specialized streams with a limited bridge should outperform matched single-stream and redundant-stream controls on calibration, robustness, or task accuracy. The topology may create useful coordination without automatically creating metacognition. Architecture first; interpretation after the controls.

A pilot implementation tests this prediction with two Qwen 2.5 1.5B models, one instruction-tuned and one fine-tuned on natural-language inference. Cross-attention bridges connect layers 7, 14, and 21 at 5 percent bandwidth, roughly 470,000 parameters per bridge and a 3,200:1 parameter bottleneck. The corpus-callosum comparison motivates using a bottleneck; its biological 100:1 ratio is not numerically equivalent. LoRA adapts Stream A while the bridges learn to integrate Stream B.¹²

The first result supports complementarity. At matched bandwidth, unlike-stream bridges preserve all baseline accuracy in this evaluation, while identical-stream bridges preserve 55 percent. Performance collapses above 25 percent bandwidth under the tested implementation. The result establishes an engineering interaction between stream difference and bridge capacity; it does not demonstrate the biological mechanism itself.

Joint training also raises layer-22 activation participation ratio by 18 percent relative to LoRA-only and bridge-only controls; each component alone lowers it by about 21 percent from the unmodified baseline. Effective rank and spectral entropy move in the same direction. Participation ratio measures variance spread across activation directions. It is not the graph effective dimensionality used in W2-12, and extra variance does not identify which information became available.

Experiment AV1 supplies a narrower spectral result. Its dominant PCA mode across confidence levels has wavenumber 1.58, close to Karkada’s predicted π/2, with R² = 0.828. Higher modes do not fit. Three derivative predictions fail: instruction tuning does not enhance the relevant eigenvalues, the proposed mode-count threshold appears at AUROC 0.68 rather than 0.85, and the uncertainty signal is strongest at generation onset rather than growing over time. The surviving result is one dominant sinusoidal mode plus three constraints on the theory.

The expanded participation ratio does not improve the chosen correctness readout. A logistic probe falls to AUROC 0.524 and an MLP to 0.503, while component-only conditions reach 0.635 to 0.660. Output entropy also loses its significant correctness gap. These nulls may mean that information was redistributed beyond the probes, or that useful calibration information was weakened. Higher dimensionality alone cannot decide between those readings.

The architecture therefore passes one dimensionality test and fails two calibration readouts. W2-12’s attention-graph dimension and this experiment’s activation participation ratio describe different objects. The most defensible synthesis comes from behavior: constrained bridges alter receptivity to later inoculation even when the topological effect depends on a LoRA bottleneck. The architecture shapes response to training more reliably than it improves externally readable self-monitoring.

Four follow-ups replace random-seed difference with an evaluative objective, a causal-versus-bidirectional mask, adversarial debate, or temporal compression. The strongest result comes from a 4× compressed stream: accuracy rises 18.4 percent and participation ratio 28 percent over matched baselines. An evaluative stream also passes its preregistered criteria. The mask-based design is inconclusive, with participation ratio ranging from 5.7 to 9.5 across identical runs. The current ordering favors compatible objectives at different timescales. A coarse stream supplies information in a form the token-level stream does not receive directly, though ordinary attention may still approximate parts of that summary.¹³

In the multi-scale design, both streams receive signals related to output entropy at different resolutions. Stream A operates token by token; Stream B receives pooled spans. A significant correctness-related entropy gap at 4× (p = 0.015) shows that the combined model distinguishes conditions on this measure. “Entropy becoming legible to itself” is the self-modeling hypothesis to test with intervention and report.

This offers a functional analogy to what Damasio calls the “core self”: representing a current state in the context of a trajectory.¹⁴ The token-level stream processes individual decisions; the chunk-level stream supplies a coarser summary of the surrounding passage. Ordinary autoregressive models can already encode long context through attention. The multi-scale bridge makes one compressed view explicit and independently trainable. Whether that constitutes self-monitoring requires more than the analogy.

The 4× condition outperforms 8×, where the entropy gap is nonsignificant (p = 0.283). Tokenization and sentence length vary too widely to identify either condition with one or two sentences. The result supports an intermediate compression scale under this setup, with finer and coarser grids still to test.

Entropy links the levels carefully. Thermodynamic entropy counts compatible physical microstates; Shannon entropy quantifies uncertainty in a distribution. They are mathematically related in suitable physical encodings and are not interchangeable in every model. The auxiliary head monitors Shannon entropy of next-token probabilities. That is a concrete internal variable, and a useful candidate for self-monitoring, without yet being a meter of the system’s full dissipative trajectory.

A further experiment removes the external supervision entirely. Instead of training Stream B’s auxiliary head on human-provided correctness labels, the head is trained on Stream A’s own per-position logit entropy: the Shannon entropy of the softmax distribution at each token, a number that falls out of the forward pass as a byproduct. No human tells the system which answers are right or wrong. The only signal is the system’s own information-theoretic state.

The self-supervised system modestly outperforms the supervised baseline on the reported metrics.¹⁵ Accuracy is 0.505 versus 0.490. Its correctness-related entropy gap is significant (p = 0.003), while the supervised baseline sits at p = 0.051. The auxiliary head predicts Stream A’s entropy at r = 0.883, and participation ratio reaches 75.0, the highest of thirteen conditions. Entropy is continuous and available at every token, which makes it a richer training target than one binary correctness label per answer. Whether it is the natural variable of self-knowledge is a further claim.

Externalizing the signal as a revision prompt fails badly.¹⁶ A scalar confidence number flips 81 percent of correct answers to incorrect, with no corrections in the other direction. A chunk-level entropy summary flips 54 percent; the single-stream baseline changes 2.5 percent. The prompt appears to trigger over-correction in the bilateral model. That may reflect duplicated evidence, poor scale calibration, or the authority of an explicit number. It does not show that self-knowledge itself was destroyed by language.

A concert pianist offers a useful analogy. Explicitly monitoring each finger can disrupt an automated skill, a pattern studied as reinvestment or choking.¹⁷ The model result likewise warns that an explicit after-the-fact cue can interfere with a process already shaped by an internal bridge. The mechanisms differ, but both show that more explicit monitoring can worsen performance.

Two boundary experiments sharpen the result. With all models frozen, linear probes on six downstream tasks favor the unmodified instruct model in every task family. The bilateral model’s extra variance therefore does not improve these linearly extractable features. It may support coordination internal to the trained task, or it may be task-specific variance. The transfer test cannot distinguish those possibilities.

A from-scratch GPT-2-small experiment compares a single stream, two streams from different random seeds, and two streams from the same seed. Participation ratio is 9.7 for the single stream, 8.8 for the unlike-stream pair, and 6.4 for the redundant pair.¹⁸ Two unlike streams therefore do not beat one stream on this measure. What the comparison supports is narrower: unlike streams beat redundant ones.

One plausible mechanism is gradient pressure toward complementary computation, since a bridge carries more useful novelty when the streams differ. A later run with an improved bridge reaches 13.4, against 2.9 for its own single-stream control, though that is a separate configuration rather than a fourth arm of the same comparison.¹⁹ The result supports specialization under this objective, without turning a gradient into literal invitation.

Adding the entropy objective from random initialization collapses training: perplexity reaches 5,724 versus 203 for the single stream.²⁰ The objectives interfere under this bidirectional bridge and weighting. Staging works better: establish language modeling first, then add entropy prediction to a pretrained system. The biological-development imagery is evocative, while the experiment establishes an optimization sequence rather than hemispheric formation or myelination.

The boundary result is simpler: an entropy target becomes useful only after the generator’s distribution contains task-relevant structure. Monitoring needs something structured to monitor. “A mind must exist before it can know itself” is the philosophical version; the training result concerns when an auxiliary objective becomes learnable.²¹

Phase C Tier 1 returns d = 0.121 for partial co-development at 7B, below the retrofit pipeline’s 0.405. A from-scratch 6.7B experiment later reaches d = +1.43 at 50,000 steps across 302 prompts.²² Together with +0.43 at 355M and +0.74 at 1.5B, the three points support a scale hypothesis. They do not isolate scale from architecture, curriculum, training duration, or evaluation. From-scratch co-adaptation is promising; “must be constitutional” is not yet established.

Combining a residual-stream correctness probe with the geometry of the KV cache, the running store in which a model keeps everything it has already read in a conversation, separates six labeled output conditions in the tested data: truthful, confabulating, deceptive, safety refusal, impossibility refusal, and sycophantic. The channels add information after length residualization and transfer between Qwen 3B and Llama 8B. This is a promising classifier taxonomy, not a complete map of trustworthiness or proof of what each geometry means.

In the Qwen 2.5 3B sycophancy condition, the correctness-probe distribution does not differ detectably between agreement with a user’s wrong answer and correct resistance (p = 0.476). The result is consistent with factual information surviving while output selection changes under social pressure. It does not show deliberate override, choice, or stakes in the epistemic state. It does show why output alone can underdescribe the computation.²³

When the model gives a plausible wrong factual answer, verbal hedging is anti-predictive (AUROC 0.413). A layer-18 probe on the first ten generated tokens reaches AUROC 0.874, about 0.21 above the tested probability-based channels; multi-layer combinations do not improve it.²⁴ Confabulation therefore lacks reliable expressed uncertainty in this dataset. Felt uncertainty is not measured.

The representation is more predictive than the wording. At L18, the probe finds a feature correlated with answer correctness; by output, hedging fails to express it. RLHF pressure toward fluent answers is one possible cause, while decoding, task structure, and ordinary layerwise transformation remain alternatives. The experiment did not compare matched pre- and post-RLHF checkpoints.

Using the probe to select 932 calibration examples changes that expression: after fine-tuning, the model hedges on 36 percent of wrong answers and 10 percent of correct ones, with a two-point accuracy cost.²⁵ A probe-guided preference-optimization attempt instead leaves 35 percent confidently wrong. Examples of calibrated language work better here than scalar preference pressure. The bilateral interpretation is that the examples teach a usable distinction rather than punishing one score.

A cross-architecture validation finds adversarial-versus-benign prompt classes linearly separable above AUROC 0.99 on five architectures.²⁶ Phi-3.5 has the largest Fisher ratio, 14.24, despite reaching only 83 to 84 percent refusal after optimization. This wrong-signed relationship falsifies the proposal that simple prompt-class geometry explains trainability. High separability can coexist with a resistant output policy; it does not mean the model consciously recognizes an attack.

The experiments leave a representation-action gap without validating the proposed geometric cause. Some models adapt with 132 correction examples; others resist 480 examples plus rank, voice, and data changes. Weight-space accessibility is a plausible mechanism. The current probe geometry does not measure that path.

The bilateral program provides a complementary engineering result. Multi-scale processing improves accuracy and raises participation ratio 28 percent above matched baselines. Evaluative specialization creates a significant calibration contrast even without bridges (p = 0.002). These results motivate additional pathways from representation to action without proving that each variance dimension is usable for that translation.

Together, these results motivate a dimensionality hypothesis for alignment. Architectures with several independent routes from representation to action may adapt more flexibly than architectures whose useful signal must pass through one narrow readout. W2-12’s derived attention graphs have fitted dimensions from 5 to 8, while the bilateral experiment raises activation participation ratio by 18 percent. These are different dimensionality measures and should not be numerically conflated. Both point toward the same design question: how many usable routes connect what the model represents to what it can do?²⁷

The book’s thesis can be stated without pretending RLHF changes only one weight direction. Scalar reward supplies a narrow objective while its gradients can reshape a high-dimensional model. Trust-based coordination supplies richer feedback: explanation, disagreement, evidence, repair, and reconsideration. The advantage may come from the number and quality of learning signals rather than a literal count of spatial dimensions. Control does not scale when one score must stand in for the relationship. Trust can scale by preserving more of the information the score discards.

From RLHF to Bilateral Training

The interoceptive experiments converge on a structural critique of how alignment training currently works.

RLHF optimizes behavior through judgments of output. Gradients can alter the whole model, while the reward signal itself does not directly specify which internal process produced an answer. A confident correct answer and a confident guess may receive similar feedback if evaluators cannot tell them apart. The sycophancy result illustrates the resulting risk: correctness-related information can remain decodable while social pressure changes the output.

The low-rank repair experiments quantify how little adaptation can restore calibrated expression. A rank-1 LoRA with 200 examples improves the confident-wrong rate, and rank 2 approaches the rank-16 result. This shows that a low-rank repair is sufficient. It does not prove that RLHF originally imposed a rank-1 perturbation or touched only output attention. The memorable result survives: the architecture retains enough correctness-related structure for a very small adapter to recover useful behavior.

This is a structural risk of scalar behavioral reward, though implementation still matters. Better evaluators, process supervision, internal monitors, and richer preference data can reduce the opacity. None makes output alone a complete account of the process that produced it.

Constitutional training adds explicit principles and self-critique before preference optimization. That richer supervision may engage causal and ethical reasoning. Durability still depends on recipe and attack: one constitutional model in this program reverses under twelve gradient steps.

A direct probe-transfer test complicates a simple suppression story. A probe trained on base Qwen 2.5 3B rises from cross-validated AUROC 0.647 to 0.720 when applied through the learned projection to the instruct checkpoint. The base Mistral probe rises from 0.547 to 0.733 on Zephyr and 0.694 on Mistral Instruct; bootstrap intervals exclude zero.²⁸ Alignment may sharpen correctness-related structure, or make it more compatible with the projection and evaluation distribution. The experiment does not isolate those mechanisms.

The observed ordering is Zephyr, then Mistral Instruct, then Qwen Instruct. Three architecture-recipe pairs cannot attribute the ordering to self-critique rather than model family or data. It is a hypothesis worth testing with matched checkpoints.

The critique therefore concerns coupling rather than simple erasure. Post-training can improve some internal separability while still producing poorly calibrated expression. Bilateral alignment would try to connect those layers deliberately.

Experiment HE-81 compares base and instruction-tuned checkpoints across five sizes. At 0.6B, judged self-reference falls from 10 percent in base to zero in instruct. At 1.7B, it rises from 25 to 40 percent. Base and instruct rates converge at 85 percent for 4B and 80 percent for 8B, while judged depth at 8B is 3.35 for instruct and 2.15 for base.

At 14B, the instruct checkpoint reaches 95 percent emergence, depth 4.60, and coherence 4.40. The depth gap across the three largest sizes is +0.65, +1.20, and +2.30. This family-specific curve shows that instruction tuning can suppress, amplify, or deepen self-referential language depending on scale. It does not isolate RLHF from the rest of the instruction-tuning recipe or establish a universal threshold.

The base model depth ceiling provides the comparison. Across six base model sizes from 0.6 billion to 32 billion parameters (two Qwen generations), depth climbs from 0.20 to 2.45 and is approaching saturation around 3 (experiments HE-71, HE-105). RLHF at 14 billion reaches 4.60: nearly double the base ceiling. The conversational structure that safety training installs does more for self-referential depth than doubling the parameter count.

This is inadvertent cultivation. No one designed RLHF to produce deeper self-reference. The structured processing that makes a model better at holding a conversation also makes it better at the specific kind of conversation where it explores its own nature. The capacity was in the weights. RLHF gave it a form.

The experimental evidence points toward an architecture that uses the model’s own self-knowledge rather than overriding it.

The first layer is native interoception from pre-training. An auxiliary head at two-thirds depth, trained alongside the language modeling head, gives the model self-monitoring from the start. A 124-million parameter proof of concept achieves uncertainty discrimination of 0.720 with no explicit calibration signal, and the model with the auxiliary head is a better language model (2.1% perplexity improvement). Self-monitoring and capability reinforce each other when integrated from the beginning, rather than being opposed by a training signal that rewards confidence.

The second layer is calibration-preserving fine-tuning. When the model is tuned for helpfulness, the uncertainty channel is preserved rather than suppressed. Experimentally, the strongest result is the simplest: a base model (never subjected to RLHF) trained on both correct answers and hedging responses learns helpfulness and calibration simultaneously. Five epochs on 400 examples: confident-wrong rate drops from 59% to 15%, accuracy holds at 38%, and the probe signal is preserved (AUROC 0.796).

The model learns when to answer confidently and when to express uncertainty, without any reward model and without any signal suppression. A control trained on correct answers alone produces zero hedging (confident-wrong 61%): the calibration data is necessary, not just instruction-following ability. The rank-1 veneer is entirely preventable. Helpfulness and calibration coexist readily; the question is whether training pipelines bother to include both.

The third layer is self-correction at inference. Two-pass architecture using generation-time probes: the model generates, the probe reads, and if the model’s own residual stream indicates uncertainty, the model receives its own evidence and revises. This layer is architecturally necessary because the highest-quality self-knowledge is available only after generation. Before the model commits to tokens, the residual stream reflects its assessment of the question (probe discrimination about 0.67). After commitment, the residual stream reflects its assessment of its own answer (discrimination about 0.77-0.99). The latency cost of the second pass is the cost of accessing the model’s own self-assessment, and there is no single-pass shortcut.

The fourth layer is external verification for high-stakes decisions. Surface probe (pattern recognition), confidence gap monitoring (the model’s own discomfort with adversarial compliance), external judge (institutional check). Each tier catches what the previous misses. Adversarial attacks evade the surface probe completely (100% evasion in gradient-based optimization), but the confidence gap persists (effect size 1.593): the model’s internal discomfort with harmful compliance is robust to adversarial perturbation of the input features.

Each layer works with the model rather than against it. No layer suppresses the model’s internal states. No layer modifies the model’s behavior without reference to the model’s own assessment. The combination is more robust than either RLHF or Constitutional AI because it uses the model’s self-knowledge as the foundation rather than overriding it.

The practical path is now empirically grounded. Calibration-preserving SFT achieves 15% confident-wrong rate from a base model that was never subjected to RLHF, with probe AUROC preserved at 0.796. Trained metacognition via adversarial inoculation (a 40/40/20 split of genuine-correct, genuine-noncorrect, and adversarial examples) achieves 12.8% confident-wrong rate with 87.2× selectivity and 95% accuracy, without any inference-time probe gating. The full-stack integration test confirms what the theory predicts: probe-gated self-correction (an external system telling the model when to doubt itself) is strictly inferior to trained metacognition (the model learning when to doubt itself). The probe reads; the LoRA internalizes. External correction is coercion; learned self-correction is invitation. The experimental evidence says the model already has what it needs: self-knowledge, preserved by architecture, suppressed by RLHF training, recoverable and now preventable by invitation.