The Trust Attractor at the Architecture Level

The experiments so far in this chapter tested the Trust Attractor mostly in behavior, self-report and simulation. A different question concerns a network’s internal wiring: can its components learn to coordinate, and how much does the network come to depend on that coordination once it has? The answers below come from performance, representations and ablations of one learned modulation pathway. They measure reliance. Entry costs, free-energy differences and recovery toward an attractor are different quantities, and measuring them would take experiments not yet run.

The evidence comes from a programme of over three hundred experiments testing a single mechanism: stream-directed attention group modulation. The programme tests one mechanism within one architecture family (decoder-only transformers) at modest scale (12 to 50 million parameters) on a single task (language modeling). Within that frame, it sweeps seven group counts, two depths, two model sizes, and two training lengths; the claim that these results generalize to other architectures rests on the cross-substrate experiments earlier in this chapter.¹¹⁷⁶

The Mechanism

In a standard transformer, each layer computes attention over the input (deciding which other words to draw on when predicting the next one) and adds the result to the residual stream. The stream carries information forward; the attention heads decide what to attend to; neither consults the other about combining their contributions. The wiring is fixed.

Stream-directed attention group modulation uses the residual stream to generate per-group scale and shift coefficients. These coefficients amplify, attenuate, and shift the computed attention output before it is added to the stream. The coefficients depend on the current residual state; they do not directly inspect the attention output.

The implementation is a lightweight scale-and-shift operation. Each layer receives two small learned projections, gamma and beta, that map the residual stream to per-group scale and shift parameters. The overhead is 0.26% of model parameters for 8 groups, rising to 6.2% for 192 groups (where each group is just two dimensions). The mechanism is initialized to identity: at the start of training, it does nothing. Whatever coordination emerges is learned, not imposed.¹¹⁷⁷

Validation: The Coordination Is Real

Every number in this section is measured in perplexity (PPL): roughly how many words the model is effectively torn between at each step. Lower is better. A model that has absorbed more of the language hesitates over fewer candidates. The differences that matter here are small, some of them fractions of a point, so the null-pairing baseline below is the number to read first.

Six independent tests check whether the effect is genuine (TA-ARCH-8, 70 runs). A null-pairing baseline (training the same architecture twice with identical seeds) established that CUDA nondeterminism, the GPU hardware’s own run-to-run randomness, contributes a standard deviation of 0.017 PPL between runs, 22.8 times smaller than the coordination signal. The main-effect comparison yielded paired p = 0.0015, Cohen’s d = −1.43 (the sign marks direction; a size above 0.8 is conventionally called a large effect), with nine of ten seeds showing improvement. A parameter-matched control (widening the feed-forward network to add the same number of extra parameters without any coordination pathway) produced no improvement (p = 0.27). An architecture control (the same gamma-beta pathway with frozen random weights, providing modulation without learning) produced no improvement (p = 0.19).

A fresh replication with ten seeds never used in any prior experiment confirmed the effect (p = 0.0012, Cohen’s d = −1.47). The main benefit replicates on novel initializations. The nonsignificant wider and frozen controls do not establish equivalence to baseline or a significant difference from the main intervention, so added capacity and modulation remain incompletely separated from learned coordination.

The sixth test is the most revealing. Disabling the coordination at evaluation time (setting gamma to one and beta to zero, returning the mechanism to its initialized identity state) causes the model’s perplexity to increase by 29 points, from 61.2 to 90.6. The mechanism contributes 0.5 points of perplexity improvement during training. Removing it destroys 29 points. The ratio is roughly 60 to 1.

This asymmetry measures functional reliance on the coordination pathway: the reported training comparison changes perplexity by about 0.5, while disabling the trained pathway raises it by about 29. Perplexity is a measure of predictive error, a pure number. It is not work, an entry cost or a free-energy difference. The ablation shows how much damage removing the pathway does; calling the coordinated state an attractor would need evidence of how the model returns to it or moves between states.

How Deep the Reorganization Goes

A direct comparison of the hidden representations at each layer of the standard and coordinated models (trained on the same data with the same seed) reveals cosine similarity near zero: 0.03 to 0.07 across all layers, on a measure where 1.0 means parallel directions and 0 means orthogonal directions. At every layer, the average hidden state of one model points in an almost entirely different direction from the other’s, though both learned the same language. Two networks can encode the same thing along different axes, and this measure would then read near zero, so a cross-model alignment and functional test are needed to infer different processing strategies from these directions.

Scaling Across Every Tested Axis

The ablation cost rises with every axis tested. The benefit rises with finer groups and more depth, and at four times the model size it appears only when the groups are made fine enough.

Coordination granularity. The modulation divides the attention output into groups. Finer groups produce larger benefits and deeper dependency:

Table: Group count sweep at 6 layers, 12M parameters. All comparisons paired p < 0.01 against standard baseline. Ablation cost measured as PPL increase when coordination is disabled at evaluation time.

At 192 groups, each pair of dimensions in the attention output receives its own modulation signal. The model’s perplexity improves by 1.1 points. Removing the modulation destroys 187 points: the model becomes nearly non-functional. The dependency ratio is 170 to 1.

Network depth. Moving from 6 to 12 layers amplifies both the benefit (from −0.54 to −0.82, a 52% increase) and the ablation catastrophe (from +29 to +40, a 38% increase). Combining 192 groups with 12 layers produces the programme’s strongest result: paired p = 6 × 10⁻⁸, Cohen’s d = −5.08 (six times the threshold conventionally called a large effect), all ten seeds positive. The PPL improvement is 1.50, and the ablation catastrophe ranges from +854 to +1,517 across seeds. The model’s perplexity rises from 59 to between 900 and 1,600, fifteen to twenty-seven times worse than before. The model has come to lean on the coordination almost entirely.

Model scale. At 50 million parameters (roughly four times the 12-million-parameter base model), the coordination effect is present when the group count matches the model’s capacity. Eight groups produce a non-significant effect at this scale (p = 0.53). Sixty-four groups (matching the 8-dimensions-per-group granularity that worked at 12M) produce a significant effect (p = 0.006). One hundred twenty-eight groups produce a larger effect (p = 0.002, improvement of −0.80 PPL) with an ablation catastrophe of +609. The coordination principle carries over to a model four times larger, provided the group count grows with the model’s width.

Training duration. The benefit stabilizes early (roughly −0.5 PPL from 5,000 to 10,000 training steps). The ablation catastrophe does not stabilize. It grows from +29 at 5,000 steps to +73 at 7,500 steps to +202 at 10,000 steps. The model continues reorganizing around the coordination pathway long after the performance benefit has plateaued. When training stopped, the dependency was still deepening, and faster than before.

Bookends and Interaction: Where Coordination Lives

A layer-by-layer ablation (disabling coordination at one layer at a time while leaving the other eleven intact) reveals the internal structure of the coordination.¹¹⁷⁸

Layer 0 (the first layer, where the residual stream first encounters attention output) contributes +22 to +32 PPL when ablated. Layer 11 (the final layer, where the model makes its predictions) contributes +40 to +73. The middle layers, 3 through 9, each contribute less than 1 PPL when ablated individually.

The sum of twelve single-layer ablations is +72 to +118 PPL; full ablation is +488 to +854. Perplexity is an exponential of the model’s average error, so layer effects that simply add on the error scale can multiply on the perplexity scale. The large full-ablation figure is not, by itself, evidence that layers compensate for one another; that would need interaction tests, with controls, on the error scale.

Single-layer and full-pathway ablations have different performance effects. Reading the difference as biological redundancy remains a hypothesis until interactions are tested on an additive loss scale.

Rescuing the Hard Predictions

The mechanism analysis (TA-ARCH-10, two seeds) compared the standard and coordinated models’ predictions token by token across the validation data. The question: does coordination improve everything uniformly, or does it help with specific kinds of predictions?

The answer is specific. For the hardest tokens (the top 10% by standard-model loss, where prediction is genuinely difficult), the coordinated model improves by 0.32 nats (a nat is the natural-logarithm cousin of a bit) and is better 59% of the time. For the easiest tokens (the bottom 50%, where prediction is trivial), the coordinated model is slightly worse: −0.11 nats, better only 44% of the time.¹¹⁷⁹

Coordination does not make the model uniformly better. It trades easy predictions for hard ones. The gamma modulation reveals how. In the early layers, modulation is gentle (mean gamma shift +0.05, low variance). In the middle layers, modulation is suppressive (mean shift −0.15 to −0.19), attenuating attention outputs. In the deep layers, modulation is highly selective (gamma variance exceeds 0.5), with context sensitivity increasing through the network (position-dependent standard deviation rising from 0.05 in layer 0 to 0.43 in layer 11).

The coordination learns a depth-dependent processing strategy: gentle at the surface, suppressive in the middle, aggressively selective at the depth where predictions are made. This strategy is not available to the standard architecture. The standard model adds each layer’s attention output to the stream as it comes. The coordinated model rescales and shifts that output group by group, differently at each layer and position, according to what the stream already holds. The additional expressiveness is concentrated where it matters most: on the predictions the standard model struggles with.

The Tested Retrofit Protocols

A final experiment tested three protocols for adding coordination to an already-trained model. None succeeded within the tested 2,500-step continuation budget.¹¹⁸⁰ The result bounds these recipes, rather than proving retrofit impossible.

Training only the coordination parameters while freezing the base model produces no change (delta = −0.05, ablation +0.03): the coordination layers have nothing to coordinate because the base representations cannot adapt. Fine-tuning everything at a reduced learning rate produces no change (delta = −0.12): the base representations barely budge. Fine-tuning at full learning rate actively degrades performance (delta = +1.06, p < 0.0001): the likeliest culprit is the warmup phase, which ramps the learning rate back up to full strength and can shake loose weights that had already converged.

The tested schedules yielded no successful retrofit within their continuation budgets. Attribution of the full-rate damage to warmup requires its own control. These finite nulls do not establish that crystallized representations cannot reorganize under other schedules, longer continuation, distillation, or retraining.

The retrofit results motivate comparisons with alignment applied after pretraining. Gamma modulation on these small models does not prove that reward-based alignment at frontier scale must begin during initial training. That proposal requires direct tests of the relevant training protocols.

What the Experiments Show

The complete experimental programme yields five findings at the architectural level:

1. Stream-directed modulation benefits several tested configurations. The main arm outperforms standard processing by 0.5 to 1.5 perplexity points in the reported comparisons, with seed replication in the configurations trained from scratch except eight groups at 50 million parameters. Superiority over frozen-random modulation remains unestablished without a direct contrast. A significant main effect and a nonsignificant control effect do not themselves establish a difference between those arms.

2. Coordination creates structural dependency. The ablation-to-benefit ratio runs from 49:1 to 170:1 across the six-layer group sweep (62:1 at the eight-group configuration used in the validation tests, 29 points of ablation cost against 0.47 of benefit) and reaches between 570:1 and 1,010:1 at the strongest (192 groups at twelve layers, where the ablation catastrophe runs from 854 to 1,517 points across seeds against a benefit of 1.50). The spread at the strong end is seed variation; the ratio is quoted across that spread rather than from the best seed alone.

   Removing the learned pathway produces large predictive damage. This is functional dependency; no thermodynamic preference or entry/exit work has been measured.

3. Dependency keeps deepening. More coordination granularity, more network depth, and more training time all increase the ablation catastrophe monotonically. The dependency deepens on every axis tested, even after the performance benefit has plateaued. The system was still organizing around coordination when training stopped.

4. The modulation implements depth-dependent suppression and selection. It helps hard predictions while worsening some easy predictions. Low cosine similarity says the two models’ hidden states point in different directions in the coordinates chosen; whether they process differently needs the models aligned first.

5. The tested retrofit recipes failed. Three continuation protocols did not recover the from-scratch benefit within 2,500 steps. This is a protocol-specific null.

The experiments show modest predictive benefits and large ablation effects for a learned modulation pathway across the tested configurations. They support architectural investigation of coordination. They do not measure a thermodynamic attractor, and they do not show that retrofitting is impossible in general.