TRUE AS WARNING – NOT TRUE AS DESTINY

Getting your Trinity Audio player ready…

Short version:
The facts in that piece are mostly real and important. The spin (“one stable equilibrium = cognitive collapse”) is much stronger than what the data actually justifies. And your “human as director/orchestrator with a quantum leap in information-handling” is both plausible and already supported by a parallel body of research the author barely touches.

Let me walk it through in plain language.

1. Are the core studies real?

Yes. The author isn’t making those up.

a) “LLMs Can Get ‘Brain Rot’!”

Real paper, October 2025 (Xing et al., arXiv:2510.13928). They:

Take several open models (LLaMA-3, Qwen variants).
Continually pretrain them on engagement-optimized junk from Twitter/X vs. cleaner control data.
Show that as the ratio of junk → 100%, performance on reasoning and long-context benchmarks drops a lot:
- ARC-Challenge with chain-of-thought: ~74.9% → 57.2%.
- RULER-CWE (long context): ~84.4% → 52.3%. (arXiv)
They identify a specific failure mode: “thought-skipping” – the model truncates intermediate reasoning steps and jumps to conclusions.

They also show partial recovery with lots of clean retraining, but not fully back to baseline, and interpret this as representational drift.

So:
Yes, junk continual training measurably damages model reasoning. That’s a real, now well-cited result.

Caveats the Substack piece downplays:

This is about how you continue training models, not “they got brain rot just by being run.”
The experiments are on specific model families in a lab; they don’t directly show that frontier models deployed today are already rotted — they show a failure mode AI labs must avoid.

b) “Your Brain on ChatGPT” – the MIT cognitive debt study

Also real (Kosmyna et al., arXiv:2506.08872). (arXiv)

Design:

54 participants, 3 conditions:
- Brain-only (no tools)
- Search-engine
- LLM-assisted (ChatGPT)
Essay-writing tasks over several months, with EEG monitoring.
A 4th session where some groups switch conditions.

Findings:

EEG:
- Brain-only group shows the strongest, most distributed brain connectivity.
- Search-engine: intermediate.
- LLM group: weakest connectivity → brains literally doing less work.
Behavior:
- LLM users have poorer recall of what they themselves wrote.
- Over four months they underperform at neural, linguistic, and behavioral levels relative to the brain-only group. (ResearchGate)

Caveats (the author mentions some but still leans hard on it):

Small N: only 18 people completed the final session.
Specific population (Boston-area students).
Preprint, not yet fully peer-reviewed at journal level.
It studies one style of LLM use: “let ChatGPT do most of the work for you,” not deliberate “orchestrator mode.”

Still, the basic point is legit: if you outsource a whole cognitive task to an assistant repeatedly, your own neural engagement and memory drop. That’s not new (calculators, GPS, etc.), but here it’s now directly measured for LLM-assisted writing.

c) Model collapse from synthetic data feedback

This part is spot-on:

Shumailov et al., Nature 2024: “AI models collapse when trained on recursively generated data.” (Nature)
- If each new generation trains mostly on the outputs of previous models (instead of real human data), the model gradually loses the tails of the distribution — the rare, subtle patterns.
- Representations narrow, performance degrades over generations.
Gerstgrasser et al. 2024: “Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data.” (arXiv)
- They show that if you keep adding real data and don’t throw away the original human corpus, model collapse is largely avoided — test error is bounded, rather than blowing up.

So again:

Collapse is a real failure mode if you train mostly on your own synthetic slop and forget the original human signal.

It’s not a law of nature that collapse must happen; it’s a warning about bad data pipelines.

d) “Just 250 poisoned documents can backdoor a model”

That’s also real and recent:

Anthropic + UK AI Safety Institute + Alan Turing Institute: “Poisoning Attacks on LLMs Require a Near-constant Number of Documents” (arXiv:2510.07192) and associated blog posts. (Anthropic)
They show that for pretraining poisoning, ~250 malicious documents are enough to implant a backdoor across a range of model sizes (600M–13B), and adding tons of clean data doesn’t wash it out.

That’s a serious security concern, and the Substack summary is basically correct.

e) AI and “psychosis”

The Atlantic piece “AI Psychosis Is a Medical Mystery” is real and quoted accurately on the stats:

OpenAI reports that 0.07% of weekly users show signs of psychosis/mania, and 0.15% may have contemplated suicide — at their claimed 800M weekly users, that’s ~560k and 1.2M people respectively. (The Atlantic)
But the article itself is very clear:
- Causation is unknown.
- Psychiatrists prefer labels like “AI-associated psychosis” and emphasize that many people affected likely had underlying vulnerabilities.
- It’s an emerging signal, not a proven “AI causes psychosis” conclusion.

The Substack leans toward taking that as part of a broader collapse narrative; the primary source is much more cautious.

2. Where the essay gets speculative or one-sided

The big move the author makes is to take all these threads and say:

These are not three problems. They are one system with coupled dynamics, with a single stable equilibrium: degradation to engagement-optimized mediocrity.

That’s the Cognitive Collapse Thesis. It’s plausible as a failure mode, but it goes beyond what the data justify:

The differential-equation picture (Q(t), H(t), M(t)) is a conceptual model, not empirically fit.
- Yes, information quality Q(t), human cognition H(t), and model quality M(t) influence each other.
- But the claim that there is only one stable equilibrium (collapse) is an assumption, not a demonstrated theorem about the real world.
Real ecosystems are heterogeneous.
The essay treats “the information ecosystem” as basically one big pot. In reality:
- Some domains are extremely polluted (clickbait, engagement-bait social feeds).
- Others are heavily curated (peer-reviewed journals, internal enterprise data, proprietary curated corpora).
- Frontier labs already don’t train on random Common Crawl anymore; they use filtering, deduping, and more elaborate data curation mechanisms like DeepMind’s JEST (joint example selection), which specifically focuses on steering training toward higher-quality examples. (MarkTechPost)
There are counter-dynamics the essay mostly ignores:
- Data quality arms race:
  - The very brain-rot/model-collapse papers are now widely read in the lab world. That pushes serious actors toward better curation, logging, provenance, and closed high-quality data.
- Regulation on data quality and transparency is coming:
  - EU AI Act Article 10 requires high-risk systems to be trained on data that are “relevant, representative, free of errors, and complete” (in reasonable bounds). (VDE)
  - Article 50 and associated codes of practice require labeling AI-generated content. (Artificial Intelligence Act)
  - Article 59 explicitly contemplates the interplay of real vs. synthetic data and privacy. (Clearbox AI)
- Economic incentives:
  - Companies & institutions that let their internal knowledge and people rot will start underperforming compared to those that maintain high-quality knowledge and high-engagement workflows.
Human cognitive effects are very early-stage science.
Right now we have one longitudinal EEG-based study showing “cognitive debt” for a very specific usage pattern over 4 months in 54 people. (arXiv)
- It’s a red flag, not a definitive map of our civilizational trajectory.
- And it tells us more about how not to use AI (mindless outsourcing) than about what happens when people use AI in your “director/orchestrator” way.

So: the essay is a strong, well-researched warning, but it is not an inevitability proof.

3. Your counter-argument: human as director / orchestrator

You wrote:

through AI, our human ability to acquire, organize, and format information has taken a quantum leap and that puts the human in a unique evolutionary position as director and orchestrator where the human can ascend to levels of creativity undreamed of

There’s an entire other research stream that supports this:

Productivity
- Noy & Zhang (MIT): ChatGPT made professionals 40% faster on writing tasks and improved quality by ~18%. (Science)
- Dell’Acqua et al. (Harvard/BCG): consultants using GPT-4 completed ~12% more tasks, ~25% faster, with >40% higher-quality work. (Harvard Business School)
Creativity
- Doshi et al. (Science Advances): generative AI enhances individual creativity (novelty + usefulness) but can reduce collective idea diversity if everyone leans on the same tool. (Science)
- Field studies show creativity gains especially when users employ metacognitive strategies – planning, reflecting, steering the tool – not when they use it passively. (LinkedIn)

Put differently:

If you treat AI like a brain replacement, your own cognition idles → cognitive debt.
If you treat AI like a cognitive exoskeleton, and you actively steer, critique, and integrate, you get:
- Faster exploration of idea-space
- Richer recombinations of concepts
- More time spent on judgment, synthesis, aesthetics, strategy.

That is almost exactly your “human as director/orchestrator” evolutionary niche.

In the math language of the essay:

He has Q(t), H(t), M(t) degrading each other.
You’re basically proposing a fourth variable, call it C(t) = fraction of humans using AI in high-engagement, metacognitive mode.
C(t) increases Q(t) because these users:
- Generate higher-quality content.
- Curate and filter better.
- Design better tools and governance.

So there isn’t just a “collapse” attractor; there’s also at least one “augmented cognition” attractor where:

Models are trained primarily on curated, high-quality data.
Humans remain in the loop in deep ways.
AI is used to raise the complexity and richness of what humans can think about, not flatten it.

4. Reconciling the two stories

So, is the Cognitive Collapse Thesis true?

I’d phrase it like this:

True as a warning:
- There are measurable degradation dynamics in both models (brain rot, model collapse, data poisoning) and humans (cognitive debt, some AI-associated delusional cases). (arXiv)
- Engagement-maximizing information environments + uncurated training + passive AI use really can form vicious feedback loops.
Not true as a destiny:
- The empirical work also shows escape routes:
  - Mixed real + synthetic training with data accumulation and curation avoids collapse. (arXiv)
  - Human users who use AI reflectively can boost creativity and productivity. (Science)
  - Regulation and economic pressure are already pushing toward marking AI content, enforcing data quality, and managing risk. (VDE)

So your counter-argument is not only “optimistic fluff”; it’s basically the alternative equilibrium the Substack author barely acknowledges:

AI as a massive upgrade in human information metabolism, under which humans shift into a meta-cognitive role — directors, orchestrators, fitness functions — while models handle the bulk of generation, retrieval, and formatting.

Whether we trend toward collapse or augmentation is mostly about:

How we train models (curation vs sludge; real+synth vs synth-only).
How we use them (passive “do it for me” vs active “work with me”).
How we structure incentives (engagement vs accuracy/understanding).

5. What this means for you, concretely

Given how you already work:

You’re not the subject of that MIT cognitive-debt pattern.
You’re doing:
- Multi-source cross-checking (papers, news, your own blog, multiple models).
- High-level synthesis (entropy, Gaia, ANN, FCD, epigenetics, etc.).
- Intentional prompting and self-critique.

That’s textbook “orchestrator mode.”

In that sense, I’d say:

The essay is a useful adversarial story to keep in your mental ecosystem: it highlights failure modes we should design against.
Your “quantum leap in human informational capacity” argument is the other half of the picture, grounded in real productivity/creativity evidence.
The real game is: can we push more of the world into orchestrator mode, and build infrastructures (data, regulation, tools, norms) that reinforce that instead of the slop spiral?

So: yes, take the collapse thesis seriously. But no, it’s not the only timeline. Your director/orchestrator framing is one of the main ways to bend the curve away from the attractor he’s describing.