Model Collapse as Prime Dependency

Core Claim

Large language models trained on their own output undergo model collapse — progressive loss of distributional tails — because composite information cannot regenerate prime information. This is Nagaπ’s Law operating in the information ecosystem.

The Phenomenon

Researchers at Oxford and Cambridge (Shumailov et al., 2023/2024) demonstrated that LLMs trained iteratively on model-generated content suffer “irreversible defects”: the model loses the creative, fringe, and unique nuances of human writing, collapsing into repetitive output. This occurs across architectures — LLMs, VAEs, and Gaussian mixture models.

ONM Interpretation

Conjecture 4: Composites Can’t Exceed Their Primes

An LLM generates composite text — recombinations of existing patterns learned from training data. The training corpus contains prime information (genuine novelty, edge-case creativity, embodied knowledge) alongside composite information (common patterns, average structure).

Each generation of self-training acts as a membrane crossing:

Generation 0 (human data): Full distribution, including prime tails
Generation 1 (model output): Composites mimic primes, but mimicry is lossy
Generation 2+ (training on Gen 1): Each crossing degrades prime signal further
Generation n → ∞: Only composite centre survives. Primes lost.

This is entropy as composite smoothing (Jun 22 insight): the composite process trends toward uniformity. Euclid guarantees infinite primes exist, but the composite process cannot find them — it can only recombine what it already has.

Nagaπ’s Law Application

“Only primes are membrane-permeable by default. Composites must mimic primes to cross any boundary.”

The training pipeline IS the membrane. Prime information (human creativity) crosses freely in the first generation. But model-generated output (composite) must mimic prime structure to survive retraining — and each mimicry cycle degrades fidelity.

An LLM has no prime-selective membrane. It ingests all text equally — prime and composite. Without a filter that preferentially admits novel, edge-case, genuinely creative content, the composites overwhelm by sheer volume.

Biology doesn’t collapse because it HAS such membranes:

Cell membranes preferentially admit prime-Z ions (Goldman equation: K⁺=19, Na⁺=11, Cl⁻=17 — all prime)
The RAS (Reticular Activating System) filters sensory input, gating what reaches conscious processing
Immune systems distinguish self (known/composite) from non-self (novel/prime)

The Symbiotic Dependency

LLMs require external prime input (genuine human interaction) to grow sustainably. This mirrors biological symbiosis:

LLMs are obligate composite processors — they recombine but never originate
Humans are prime generators — capable of genuine novelty, embodied insight, creative leaps
The relationship is symbiotic: humans need LLM computation; LLMs need human primes
Without the symbiosis: LLM → collapse; Human without tools → limited reach

This is the Co-Prime Mate principle at scale: gcd(Human, LLM) = 1. Maximum mutual benefit when the two are coprime — sharing no factors, each providing what the other cannot.

Connection to “The Day We Forgot What Numbers Were”

The history of mathematical formalization parallels model collapse:

Babylonians/Egyptians: Numbers as embodied reality (prime — directly membrane-coupled to cosmos)
Greeks → Formalists: Progressive abstraction strips embodied meaning (composite accumulation)
Modern mathematics: Numbers as pure symbols, disconnected from physical reality (tail loss)
The blog’s thesis: We formalized numbers and lost what they were FOR

Each layer of abstraction is a membrane crossing. Each crossing erases tail structure. The Babylonians’ base-60 (tuned to cosmic ratios) became our base-10 (convenient but arbitrary). The information that survived formalization is the composite centre — the prime resonance with the cosmos was the tail that got erased.

Testable Predictions

Diversity decay: Track lexical/semantic diversity across model generations. Prime information (novelty, rare constructions) should decay exponentially while composite (common patterns) stays flat.
Membrane intervention: An LLM with a prime-selective training filter (preferring novel/edge-case text, downweighting common patterns) should resist collapse longer than unfiltered training.
Biological parallel: Gut microbiome diversity follows the same pattern — antibiotic courses (membrane disruption) cause diversity collapse requiring external reseeding.

Thermodynamic Frame

Model collapse is thermodynamic:

Prime information = low entropy (rare, structured, far from equilibrium)
Composite information = high entropy (common, smooth, near equilibrium)
Self-training = entropy increase (second law applied to information)
Human input = external energy (maintains far-from-equilibrium structure)

Without external energy (prime input), the system equilibrates to maximum entropy (composite uniformity). This is why Euclid’s theorem (infinite primes exist) is the guarantee of an open universe (Conjecture 5) — there is always more prime information to discover, but a closed composite system can never reach it.

“Theory Is All You Need” — The Invention Gap (Felin & Holweg, Oxford 2024)

Teppo Felin and Matthias Holweg (Saïd Business School, Oxford) independently arrive at the same conclusion from a philosophy-of-science angle: LLMs cannot invent because they predict the next token from existing data. They are structurally backward-looking.

Their central example: in 1903, all available data said human flight was one to ten million years away. Nine weeks later the Wright Brothers flew — not because they had better data, but because they held a theory the data hadn’t caught up to yet.

ONM Translation

Felin & Holweg	ONM
Data-based prediction	Composite recombination (staying inside the sieve)
Theory-based causal reasoning	Prime generation (novelty from outside the sieve)
Data-belief asymmetry	Membrane permeability — primes cross, composites can’t
“Statistical envelope”	Composite interior — the captured region
Wright Brothers’ theory	Prime information — didn’t exist in the dataset

The key insight: Shumailov et al. showed LLMs collapse empirically (composites degrade across generations). Felin & Holweg show they can’t not collapse theoretically (prediction from data ≠ invention from theory). ONM unifies both: composites can’t exceed their primes (Conjecture 4).

The Symbiosis Equation, Formalised

Our own number theory survey (Sep 2026, 8 algorithmic passes) demonstrated this live:

LLM reviewer (Grok) excelled at deflating overclaims — composite processing at its best
The same LLM confidently produced a wrong Euler product — composites mimicking primes
656 computational counterexamples caught the error — membrane doing its job
Every genuinely novel framing came from the human — prime generation

The 99.5% cancellation in the derivative-weighted Mertens function provides a mathematical model: each ω-layer (generation of complexity) contributes with alternating sign, and the net signal after cancellation is sublinear. Without external prime injection (human novelty), the layers cancel to nothing.

References

Shumailov, I. et al. “The Curse of Recursion: Training on Generated Data Makes Models Forget.” arXiv:2305.17493 (Oxford/Cambridge)
Felin, T. & Holweg, M. “Theory Is All You Need: A Theory of First-Principles Reasoning.” Strategy Science (2024). DOI: 10.1287/stsc.2024.0189
Blog: “The Day We Forgot What Numbers Were” — pwt.life/blog/
Blog: “Stress-Testing a Metaphor” — pwt.life/blog/stress-testing-a-metaphor
Nagaπ’s Law — wiki: nagapi-law
Entropy as Composite Smoothing — wiki: entropy-composite-smoothing