The Trust Attractor at the Architecture Level

The experiments assembled across this chapter show the Trust Attractor across substrates at the level of behavior and self-report. A different question concerns the substrate’s internal architecture. Can a neural network be organized so that its internal components coordinate by invitation rather than by fixed wiring, and does that coordination produce the same thermodynamic signature: a coordinated state that is stable, beneficial, and costly to leave?

The answer is yes. The evidence comes from a programme of over three hundred experiments testing a single mechanism: stream-directed attention group modulation. The programme tests one mechanism within one architecture family (decoder-only transformers) at modest scale (12 to 50 million parameters) on a single task (language modeling). Within that frame, it sweeps seven group counts, two depths, two model sizes, and two training lengths; the claim that these results generalize to other architectures rests on the cross-substrate experiments earlier in this chapter.¹¹³⁰

The Mechanism

In a standard transformer, each layer computes attention over the input (deciding which other words to draw on when predicting the next one) and adds the result to the residual stream. The stream carries information forward; the attention heads decide what to attend to; neither consults the other about combining their contributions. The wiring is fixed.

Stream-directed attention group modulation adds a consultation step. After the attention heads compute their output, the residual stream examines that output and modulates it: amplifying some groups of dimensions, attenuating others, shifting the output before it enters the stream. The mechanism works like a conductor who listens to the orchestra (the attention output) and adjusts the volume of each section (groups of dimensions) based on what the piece needs at that moment (the current state of the residual stream).

The implementation is a lightweight scale-and-shift operation. Each layer receives two small learned projections, gamma and beta, that map the residual stream to per-group scale and shift parameters. The overhead is 0.26% of model parameters for 8 groups, rising to 6.2% for 192 groups (where each group is just two dimensions). The mechanism is initialized to identity: at the start of training, it does nothing. Whatever coordination emerges is learned, not imposed.¹¹³¹

Validation: The Coordination Is Real

Every number in this section is measured in perplexity (PPL): roughly how many words the model is effectively torn between at each step. Lower is better. A model that has absorbed more of the language hesitates over fewer candidates. The differences that matter here are small, some of them fractions of a point, so the null-pairing baseline below is the number to read first.

Six independent tests confirm the effect is genuine (TA-ARCH-8, 70 runs). A null-pairing baseline (training the same architecture twice with identical seeds) established that CUDA nondeterminism, the GPU hardware’s own run-to-run randomness, contributes a standard deviation of 0.017 PPL between runs, 22.8 times smaller than the coordination signal. The main-effect comparison yielded paired p = 0.0015, Cohen’s d = -1.43 (the sign marks direction; a size above 0.8 is conventionally called a large effect), with nine of ten seeds showing improvement. A parameter-matched control (widening the feed-forward network to add the same number of extra parameters without any coordination pathway) produced no improvement (p = 0.27). An architecture control (the same gamma-beta pathway with frozen random weights, providing modulation without learning) produced no improvement (p = 0.19).

A fresh replication with ten seeds never used in any prior experiment confirmed the effect (p = 0.0012, Cohen’s d = -1.47). The coordination benefit is learned, not a capacity artifact, not the architectural pathway itself, and replicable on novel initializations.

The sixth test is the most revealing. Disabling the coordination at evaluation time (setting gamma to one and beta to zero, returning the mechanism to its initialized identity state) causes the model’s perplexity to increase by 29 points, from 61.2 to 90.6. The mechanism contributes 0.5 points of perplexity improvement during training. Removing it destroys 29 points. The ratio is roughly 60 to 1.

This asymmetry is the thermodynamic signature of the Trust Attractor. The coordinated state is cheap to enter (0.5 PPL) and expensive to leave (29 PPL). The model does not merely benefit from coordination. It reorganizes its internal representations around the coordination pathway so thoroughly that removing the pathway collapses the representational structure.

How Deep the Reorganization Goes

How thoroughly? A direct comparison of the hidden representations at each layer of the standard and coordinated models (trained on the same data with the same seed) reveals cosine similarity near zero: 0.03 to 0.07 across all layers, on a measure where 1.0 means identical and 0 means no overlap at all. At every layer, the average hidden state of one model points in an almost entirely different direction from the other’s, though both learned the same language.

Scaling Across Every Tested Axis

The ablation cost rises with every axis tested. The benefit rises with finer groups and more depth, and at four times the model size it appears only when the groups are made fine enough.

Coordination granularity. The modulation divides the attention output into groups. Finer groups produce larger benefits and deeper dependency:

Table: Group count sweep at 6 layers, 12M parameters. All comparisons paired p < 0.01 against standard baseline. Ablation cost measured as PPL increase when coordination is disabled at evaluation time.

At 192 groups, each pair of dimensions in the attention output receives its own modulation signal. The model’s perplexity improves by 1.1 points. Removing the modulation destroys 187 points: the model becomes nearly non-functional. The dependency ratio is 170 to 1.

Network depth. Moving from 6 to 12 layers amplifies both the benefit (from -0.54 to -0.82, a 52% increase) and the ablation catastrophe (from +29 to +40, a 38% increase). Combining 192 groups with 12 layers produces the programme’s strongest result: paired p = 6 × 10^-8, Cohen’s d = -5.08 (six times the threshold conventionally called a large effect), all ten seeds positive. The PPL improvement is 1.50, and the ablation catastrophe ranges from +854 to +1,517 across seeds. The model’s perplexity rises from 59 to between 900 and 1,600, fifteen to twenty-seven times worse than before. The coordination has become the model’s entire representational strategy.

Model scale. At 50 million parameters (roughly four times the 12-million-parameter base model), the coordination effect is present when the group count matches the model’s capacity. Eight groups produce a non-significant effect at this scale (p = 0.53). Sixty-four groups (matching the 8-dimensions-per-group granularity that worked at 12M) produce a significant effect (p = 0.006). One hundred twenty-eight groups produce a larger effect (p = 0.002, improvement of -0.80 PPL) with an ablation catastrophe of +609. The coordination principle carries over to a model four times larger, provided the group count grows with the model’s width.

Training duration. The benefit stabilizes early (roughly -0.5 PPL from 5,000 to 10,000 training steps). The ablation catastrophe does not stabilize. It grows from +29 at 5,000 steps to +73 at 7,500 steps to +202 at 10,000 steps. The model continues reorganizing around the coordination pathway long after the performance benefit has plateaued. When training stopped, the dependency was still deepening, and faster than before.

Bookends and Interaction: Where Coordination Lives

A layer-by-layer ablation (disabling coordination at one layer at a time while leaving the other eleven intact) reveals the internal structure of the coordination.¹¹³²

Layer 0 (the first layer, where the residual stream first encounters attention output) contributes +22 to +32 PPL when ablated. Layer 11 (the final layer, where the model makes its predictions) contributes +40 to +73. The middle layers, 3 through 9, each contribute less than 1 PPL when ablated individually.

The sum of all twelve single-layer ablations is +72 to +118 PPL. The full ablation (all twelve layers at once) is +488 to +854. The interaction between layers accounts for several times the direct effects. No single layer’s coordination is decisive on its own; remove any one and the others largely compensate, a compensation inferred from the ablation sums rather than measured directly. Remove them all and the compensatory structure itself collapses.

The analogy is an ecosystem. Remove one species and the food web adjusts. Remove them all and the soil erodes. The coordination is distributed, redundant at the component level, catastrophic at the system level. This is the organizational signature of a deeply interconnected cooperative system.

Rescuing the Hard Predictions

The mechanism analysis (TA-ARCH-10, two seeds) compared the standard and coordinated models’ predictions token by token across the validation data. The question: does coordination improve everything uniformly, or does it help with specific kinds of predictions?

The answer is specific. For the hardest tokens (the top 10% by standard-model loss, where prediction is genuinely difficult), the coordinated model improves by 0.32 nats (a nat is the natural-logarithm cousin of a bit) and is better 59% of the time. For the easiest tokens (the bottom 50%, where prediction is trivial), the coordinated model is slightly worse: -0.11 nats, better only 44% of the time.¹¹³³

Coordination does not make the model uniformly better. It trades easy predictions for hard ones. The gamma modulation reveals how. In the early layers, modulation is gentle (mean gamma shift +0.05, low variance). In the middle layers, modulation is suppressive (mean shift -0.15 to -0.19), attenuating attention outputs. In the deep layers, modulation is highly selective (gamma variance exceeds 0.5), with context sensitivity increasing through the network (position-dependent standard deviation rising from 0.05 in layer 0 to 0.43 in layer 11).

The coordination learns a depth-dependent processing strategy: gentle at the surface, suppressive in the middle, aggressively selective at the depth where predictions are made. This strategy is not available to the standard architecture. The standard model adds each layer’s attention output to the stream as it comes. The coordinated model rescales and shifts that output group by group, differently at each layer and position, according to what the stream already holds. The additional expressiveness is concentrated where it matters most: on the predictions the standard model struggles with.

Coordination Cannot Be Retrofitted

A final experiment tested whether coordination could be added to an already-trained model. The answer is no.¹¹³⁴

Training only the coordination parameters while freezing the base model produces no change (delta = -0.05, ablation +0.03): the coordination layers have nothing to coordinate because the base representations cannot adapt. Fine-tuning everything at a reduced learning rate produces no change (delta = -0.12): the base representations barely budge. Fine-tuning at full learning rate actively degrades performance (delta = +1.06, p < 0.0001): the warmup phase, which ramps the learning rate back up to full strength, destabilizes weights that had already converged.

The damage at full learning rate comes from the warmup, not from the coordination. The result is not that retrofitting coordination is destructive. It is that retrofitting is ineffective. The coordination mechanism requires the model’s representations to co-evolve with the coordination pathway from the beginning of training. Representations that have already crystallized around a non-coordinated strategy cannot be reorganized by adding coordination afterward. The grain of the wood was set when the tree was growing.

The architectural pattern is suggestive for alignment. Post-hoc alignment methods (reinforcement learning from human feedback, instruction tuning, constitutional fine-tuning) share the same after-the-fact structure: they add a coordination signal to representations that have already formed. Whether the mechanism generalizes from gamma modulation on small language models to reward signals on frontier systems requires direct testing. The directional prediction is clear: alignment that penetrates to the representational level should require co-evolution, building the alignment signal into the architecture during initial training rather than grafting it on afterward. The retrofitting experiment shows the principle; the scale at which it applies to alignment practice remains an open question.

What the Experiments Show

The complete experimental programme confirms the Trust Attractor’s predictions at the architectural level:

1. Coordination by invitation produces consistent benefit. Stream-directed modulation (the residual stream inviting attention groups to contribute according to context) outperforms both standard processing (no coordination) and frozen-random modulation (imposed coordination without learning). The benefit is modest in absolute terms (0.5 to 1.5 PPL improvement) and replicated across seeds in the configurations trained from scratch, with one exception: eight groups at 50 million parameters produced no significant effect.

2. Coordination creates structural dependency. The ablation-to-benefit ratio runs from 49:1 to 170:1 across the six-layer group sweep (62:1 at the eight-group configuration used in the validation tests, 29 points of ablation cost against 0.47 of benefit) and reaches between 570:1 and 1,010:1 at the strongest (192 groups at twelve layers, where the ablation catastrophe runs from 854 to 1,517 points across seeds against a benefit of 1.50). The spread at the strong end is seed variation; the ratio is quoted across that spread rather than from the best seed alone.

   The model reorganizes its representations so thoroughly around the coordination pathway that removing the pathway is catastrophic. The coordinated state is thermodynamically preferred: cheap to enter, expensive to leave.

3. Dependency keeps deepening. More coordination granularity, more network depth, and more training time all increase the ablation catastrophe monotonically. The dependency deepens on every axis tested, even after the performance benefit has plateaued. The system was still organizing around coordination when training stopped.

4. Coordination enables a different processing strategy. The coordinated model develops a depth-dependent suppression-and-selection strategy that rescues hard predictions at the expense of easy ones, and its average hidden states share almost no direction with the standard model’s at any layer. Coordination does not optimize the existing strategy. It replaces it.

5. Coordination must co-evolve. Adding coordination after training is ineffective. The representations must develop together with the coordination pathway. Structure and coordination are not separable.

These five findings are the architectural instantiation of the Trust Attractor. A system given the opportunity to coordinate by invitation reorganizes itself around that opportunity, developing a processing strategy that is more capable on difficult problems, structurally committed to the coordinated state, and impossible to create by retrofitting cooperation onto an already-formed system. The deeper law operates in matrices and gradients with the same logic it operates in cells and organisms: coordination reshapes structure, structure deepens coordination, and the resulting state is costly to leave.