Brains, eyes, environments, and why no trajectory of intelligence we can actually inspect supports the idea of monotonic improvement toward “superintelligence.”
Start with an animal that should embarrass us. The mantis shrimp has twelve types of color photoreceptor; you have three. For decades the natural inference ran straight downhill from that number: twelve channels against three must mean a richer visual world, colors you cannot imagine, discrimination beyond anything a primate retina can manage. Then someone checked. In 2014, Hanne Thoen and colleagues trained mantis shrimp to pick a rewarded wavelength and then slid the alternatives closer and closer together to find where the animal began to guess. The result, published in Science, was the opposite of the legend: mantis shrimp discriminate colors worse than humans, worse than bees, worse than most animals tested. They cannot reliably tell apart hues that you separate without effort. The twelve receptors do not build a finer version of our color vision. They run a different system entirely — a fast scan that recognizes a color outright rather than comparing channels to place it, trading discrimination for speed.
Hold onto that, because it is the whole argument in miniature. A quantity we were certain meant more capability turned out to buy a different kind of capability, and on the very axis where we expected it to dominate, it lost. The number went up. The thing we thought the number measured did not.
This is the question underneath every claim that artificial intelligence has entered a virtuous circle of monotonic self-improvement bending toward superintelligence: what, exactly, is the measure, and who is holding it? The claim feels like it is about a system getting better and better without limit. It is really about a word — better — doing two incompatible jobs at once, and about our long habit of reading a number on the substrate as if it were a readout of the faculty.
Two things called improvement
There are two motions the word better can name, and the case for runaway AI depends on letting one of them quietly stand in for the other.
The first is climbing a standard that something outside you supplies. A chess engine improves against the rules of chess; a sorting algorithm improves against a clock; a model improves against a test someone wrote, a benchmark someone curated, a loss function someone chose. This kind of improvement is real, often fast, and entirely bounded by the quality of the external yardstick. Remove the yardstick and the motion has nowhere to go. The second is generating the standard itself: deciding, with nothing outside grading you, what better even means on a problem no one has scored, and being right. This is the motion that would actually run away, because a system that improves the criterion as fast as it improves against it has no ceiling supplied from outside. It is also the motion for which there is no demonstrated instance anywhere — not in silicon, and, as we will see, not in the entire evolutionary record either.
A clarification, because the strongest technical objection lives right here. Several real systems look as though they already generate their own standards: self-play that invents its own training signal, agents that author their own subgoals, open-ended environments that breed their own tasks. Each of these generates its own signal — and every one is graded against a standard held outside it. Self-play climbs a gradient the game’s rules supply; the agent’s subgoals serve a reward it did not choose; the open-ended environment scores its invented tasks against a fixed fitness. Generating signal against an external standard is the first kind of improvement run cleverly, not the second kind breaking through. The line that matters is not “does the system produce its own training data” — increasingly it does — but “does the system validate a standard it authored, in a way that tracks something real, without anchoring to a standard from outside.” That has no instance.
Not an analogy
Before the evidence, a frame, because the usual objection to it is a misunderstanding worth removing in advance. When this argument reaches for evolution, it is not reaching for a metaphor — intelligence is sort of like a species, so maybe the limits rhyme. It is reaching for the only long run we have of the one process known to have ever produced intelligence: the optimization of many candidates against a standard the optimizer did not write. Natural selection is that process. So is gradient descent. A loss function is a fitness function; a benchmark is a selection pressure; a training run is a population of weights climbing a gradient that was handed to it. The two differ in tempo, in substrate, and in whether gains acquired in one generation pass to the next — selection is slow and Darwinian, training is fast and frankly Lamarckian, copying learned weights forward like inherited memory. They differ, too, in that gradient descent is directed where selection is blind: the one follows the slope, the other stumbles onto it through undirected variation and culling. But a directed search and a blind search of the same landscape arrive at the same fixed points; directedness changes the cost of getting there, not where there is, nor who drew the landscape. None of those differences touches the property in question. They change how quickly the curve climbs and how high the ceiling sits. They do not change that the climb is toward a standard the system was given, and that is the only feature the rest of this essay uses.
This is why “biology doesn’t apply to silicon” misfires. The boundedness, the external grading, the inability to validate a self-authored standard — these are not quirks of wetware that fail to export. They are properties of optimization-against-an-unauthored-standard, and artificial intelligence is the most explicit instance of that process we have ever built. To say its lessons stop at the edge of biology is to imagine AI is exempt from the logic of optimization itself. It is not exempt; it is the purest case. The honest question is not whether evolution is a fair analogy. It is: what process explains AI’s trajectory better than optimization under constraint? If the answer is “it is different because it is engineered,” that is not a competing account. It is the claim that AI escapes the one thing it most demonstrably does.
The longest curve we have
If you want to know the shape of substrate improvement over the long run, there is exactly one process documented from origin to plateau, and it is the growth of the hominin brain.
The broad arc is not in dispute. Over roughly three to four million years, brain volume in our lineage roughly tripled — from something near four hundred and fifty cubic centimeters in early australopithecines, through about nine hundred in Homo erectus, to roughly thirteen hundred and fifty in modern humans. That is the encephalization everyone learns about, and it is real. What is less often noticed is the shape of the whole curve, which is not a ramp continuing upward but a sigmoid: rapid rise, then a plateau reached tens of thousands of years ago, and — here the evidence becomes contested — possibly a recent decline. DeSilva and colleagues argued in 2021 that human brains began shrinking around three thousand years ago, citing a body of work spanning ninety years that documents modern crania running several percent smaller than Late Pleistocene ones. Villmoare and Grabowski re-analyzed a subset the following year and found no reliable reduction since our species’ origin. The drawdown is a live argument; treat it as unsettled. The plateau is not. The curve stopped climbing.
It stopped because the climb met a wall, and here is the part that matters for the AI question — not which wall, but that there is always one. The hominin brain saturated against the cost of its own tissue (roughly a fifth of resting energy for a couple percent of mass, on Aiello and Wheeler’s expensive-tissue accounting) and against the geometry of birth. A datacenter has neither budget, and it does not matter. The specific ceiling is an artifact of the substrate; the existence of a ceiling is an artifact of the process. Optimization against a fixed standard saturates, because once the standard is met there is no gradient left to climb, and no amount of compute manufactures a gradient where the standard has gone flat. Carbon’s walls are not silicon’s. That changes where the curve levels off. It does not change that it levels off. There are always limits; that ours and the machine’s are different limits is immaterial to the shape.
Two features of the hominin process cut against the circle regardless of the wall’s height. First, the optimizer was external. Selection did the improving; the brain never rewrote the genome that builds the brain. In four billion years no intelligence has scaled its own substrate — the thing improved and the thing doing the improving were never the same object, and the standard, survival in an environment, was held entirely outside. That is the opposite of a self-improving loop; it is the bounded, externally graded motion, run at the speed of deep time.
Second, the leading explanation for why the brain plateaued is itself the deflationary story. DeSilva’s proposed mechanism is that cumulative culture — knowledge stored in the group rather than the individual skull — relaxed the selection pressure for ever-larger brains. The reservoir took over. Once a population can hold its capability outside any single head, in language and practice and eventually writing, the expensive organ stops paying for itself. The human story already contains the move everyone is now excited about: a bounded substrate that stopped growing, and a runaway that continued — but the runaway was the external reservoir of accumulated culture, graded against reality and against other minds, not the substrate improving itself. We will return to whether that reservoir is the right reference curve for AI. For now, notice only that the one process of substrate improvement we possess from origin to plateau is shaped like a wall, not a circle.
Different kinds, not more of one
The brain-size curve answers “how far does the substrate scale,” and the answer is “until it hits a wall.” But there is a deeper problem, the one the mantis shrimp opened: the substrate scalar does not even reliably track the faculty.
Sperm whales carry brains five or six times the mass of yours; elephants, several times. The faculty we care about did not come with the kilograms. Every time raw size failed to predict the capability, the field reached for a refined scalar — encephalization quotient, then cortical neuron count, the densely packed primate neurons that let Suzana Herculano-Houzel argue the human cortex leads on a count rather than a mass. Each refinement is real work. But notice the pattern across the refinements: every new metric is selected, in part, because it puts humans on top. A scalar fitted to a sample of one is not a discovered law; it is a curve drawn through the answer we already had. We do not possess a substrate measure that predicted the faculty in advance. We possess a sequence of measures reverse-engineered from the species we were trying to explain — which is precisely the epistemic position of “more parameters means more intelligence,” a scalar that correlates until it doesn’t and is then quietly swapped for the next one that postdicts the frontier.
The dissociations run the other way too. Beat entrainment — locking movement to a musical pulse — rides the neural circuitry of vocal learning, not general intelligence, which is why a cockatoo can do it and a chimpanzee, far closer to us, largely cannot. It is a competence sitting off the main axis entirely, a different rule on different wiring. The mantis shrimp’s color system is another.
It is tempting to push this to “separate ladders cannot be summed into general intelligence,” and that is the one place to resist overreach. The dissociations do not prove the faculties can never be integrated; they prove something narrower and still sufficient. Integration plainly happens — a single model moves between translation and proof and code. What the record withholds is integration for free, from scale: piling competences on does not assemble them, and where they do integrate, the integration needs a standard to integrate toward, which returns the question of who holds the measure. Models integrate within a single reservoir — text — and the separate systems we still build for image and sound mark exactly where that reservoir ends. “General intelligence,” imagined as a single magnitude you climb by accumulating competences, is not what the dissociations show. They show the faculty we mean is one specific competence among many, separately built, that does not come bundled and does not fall out of the others by addition.
There is a candidate for what that one competence is, and it is worth stating at low confidence because it is genuinely debated. Apes taught sign systems will request, label, name, and combine — but the robustly reported gap, which David Premack pressed hard, is that they do not ask questions. They produce “give orange,” essentially never “why orange?” To ask a question is to represent a hole in your own model and move to close it — to generate, unprompted, the standard for an answer you do not yet have. That is the self-supplied standard, the second kind of improvement, appearing as a cognitive act. If the apes lack it, then a mind can have everything — large brain, learned language, rich social cognition — and still be missing the one move that would let improvement become self-sustaining. The parallel to a language model with no prompt is exact: absent a question put to it from outside, it has no standing project of its own, no gap it brought with it, nothing it is trying to know. The empty cell shows up on both substrates. I flag this as the load-bearing uncertainty in the essay, not a finding — the ape literature is contested, and deprivation in how these animals were taught is a live alternative. But if it holds, it locates the missing ingredient precisely where the circle would need it most.
When the environment moves the ruler
There is a third way the measure moves, and it dismantles the escape hatch in the phrase “superintelligence, however defined.” That qualifier is meant to make the claim robust — whatever better turns out to mean, the system will have more of it. It does the opposite. It concedes that better has no fixed referent, and a property with no fixed referent cannot be maximized without limit, because there is no single thing to maximize.
The clearest demonstration is what happens to an exquisitely optimized intelligence when you change its world. A human body is the product of millions of years of selection for fitness on Earth — and put it in orbit, and within weeks it begins to fail in exactly the dimensions Earth selected for. Bone demineralizes at one to two percent a month; muscle atrophies; the cardiovascular tuning calibrated to a one-gravity field becomes a liability. The adaptations that were fitness become anti-fitness the moment the environment shifts. There is no environment-independent “more fit.” Fitness is a relation between an organism and a world, and when the world moves, the sign of the relation can flip.
Intelligence is no different, because intelligence is fitness for an informational environment — and this is not a soft point for machines but a hard one. Distribution shift, the failure of a system tuned to one data world when the world moves, is the central unsolved problem of the field, not a biological quirk that fails to cross over. A model superlative at the standards of its training distribution has no guaranteed superiority outside it, and may carry its strengths in as weaknesses. “However defined” smuggles in the assumption that there is a defining that holds still to be superintelligent with respect to. The space traveler’s dissolving bones are the counterexample: the better you are tuned to one world, the more specifically you may fail in the next.
What the record actually shows
Three things move the measure, then. The substrate scales until it hits a wall and does not reliably track the faculty anyway. The kinds of intelligence are separate competences on separate wiring that do not assemble by addition. And the environment that defines better does not hold still, so the same capability can be excellence here and liability there. Across all three, every process we can inspect from origin to plateau shares a shape: bounded, externally graded, non-recursive, and — the decisive property — reversible.
Reversibility is the tell, and it draws the cleanest line between the two kinds of improvement. A genuine virtuous circle is monotone by construction. Compound interest has no down years; each gain is the substrate of the next. The moment a trajectory can give gains back — the brain plateauing and perhaps shrinking, the space traveler’s bone dissolving, a measured intelligence-test score rising for a century across populations and then, in several countries, turning down — you have learned that the gains were rented from an environment, not owned outright. A thing that can regress was never a circle. The AI curves we have are the same shape underneath the excitement: fast and durable where an external standard is cheap and cannot be fooled by cleverness — code that runs or doesn’t, games whose rules deliver the verdict — and soft, driftable, prone to optimizing the measure until it parts ways with the thing it was supposed to measure, exactly where the only available standard is the system’s own judgment of what is good.
The objection that remains
Two objections were dissolved upstream rather than answered here. That biology’s walls are not silicon’s is true and immaterial: the boundedness comes from the standard-relation, not the wall. That engineering routinely produces what evolution never did — flight, computation — mistakes the claim; flight broke no law, it instanced aerodynamics, a principle more general than any bird, and the case here is not “unprecedented, therefore impossible” but “a general principle predicts the boundedness, and four billion years plus every training run we have confirms it.” To beat that you must name the mechanism that escapes it — a system that supplies and validates its own standard — and “engineering will find a way” is the absence of such a mechanism wearing the costume of one.
A third objection points at the evidence the deflation has so far declined to name: roughly fifteen years of scaling laws, the orderly gains in capability charted against compute and data by Kaplan, and then by Hoffmann and the Chinchilla work. This is the strongest empirical card the optimist holds, and it sits inside the frame rather than against it. Predictable improvement against benchmarks that researchers keep extending is the first kind of improvement run at industrial scale — a population of weights ascending yardsticks held, and repeatedly raised, from outside. The curve is smooth precisely because the standard is well-defined and externally supplied, which is the condition under which the first kind of improvement is fast and the same condition under which it cannot, by climbing, convert itself into the second. Scaling laws describe how efficiently a system ascends a standard it was handed. They are silent on whether it ever comes to hold one.
The objection that genuinely remains is the only one this essay cannot close, and pretending to would be the very move it argues against. Human capability has exploded since the brain plateaued — through cumulative culture, a ratchet of externalized knowledge with no obvious ceiling, which looks from a distance exactly like the unbounded compounding the circle promises. If culture is a genuinely open-ended process rather than a large reservoir being mined, then the brain-size curve is the wrong reference, and artificial intelligence is the next layer of the cultural ratchet, not a fresh instance of bounded substrate scaling. This is the one frame in which the optimist is not obviously wrong, and the instance-not-analogy reframe does not touch it: the reframe shows that AI as currently practiced — optimization against external objectives — is the bounded process, but it says nothing about whether AI becomes a carrier of the other process. Two facts keep that question open without settling it. The cultural ratchet is itself reservoir accumulation, every increment graded against reality and against other minds — external standards again, not standards the system sets and validates for itself. And the specific instance everyone is betting on, machine intelligence trained on a corpus its own outputs increasingly pollute, currently runs the cultural loop with the sign reversed: the documented tendency is degradation, not ratcheting, when models train on models, because the relentless external corrective that made human culture an open-ended engine — reality, and other minds — is the thing a closed substrate loop removes. Whether curation and grounding can restore that corrective is an open engineering question, not a settled one. But whether culture is a bounded reservoir or an open-ended engine, and which one AI is, is not a question the record answers. It is conceptual before it is empirical — a question about which curve is the right one, not a fact we are missing.
What follows
The deflationary reading does not prove the circle impossible. It shows that nothing in the only complete process we have — four billion years of intelligence arriving, scaling, plateauing, dissociating, and reversing — has ever taken the shape the circle requires, and that the process AI most explicitly belongs to is the bounded one. The faculty is demonstrably reachable: we exist. But the one time it was reached, it was reached slowly, by an external optimizer, over deep time, against an unfakeable standard, and then it stopped. That is evidence for reachable, bounded, and externally driven. It is not evidence for self-driven, fast, and unbounded, and the second is the claim on the table. Even the human case required the external cultural ratchet to become what we mean by general intelligence — which sets the bar for a self-improving intelligence at something beyond the only instance of general intelligence we have, and tells you nothing about that bar being easy, fast, or near.
It is worth noticing who benefits from which reading, not because motive settles truth but because it tells you which claim to audit hardest. The inevitability narrative has identifiable winners: the institutions whose valuations, whose recruitment, and whose argument for being trusted to regulate the field all depend on the breakthrough being imminent and the trajectory being physics rather than choice. The deflationary reading has no comparable beneficiaries; being right about a wall enriches no one. That asymmetry does not make the deflation true. It does mean the burden of proof sits with the louder, better-funded claim, and that the claim should be held to a standard its beneficiaries would prefer to waive.
So here is the standard, stated as a condition that would flip the verdict. The record’s judgment reverses the day a system improves on a problem no one graded, by a measure it generated and validated itself — a measure that stays valid when the problems shift out of the distribution it was trained on, and that it improves across without external correction. An intelligence that scales its own substrate, judged by a standard it holds, and is right where no one could have told it so in advance. That has happened zero times in four billion years. It may yet happen once. But may yet happen once, against the entire prior record is a different sentence from inevitable, accelerating, and nearly here — and only the second sentence has anyone selling it.
cafebedouin.org 