The Future of AI - Evolution Has No Ladder

            

(ПЕРЕВОД эссе "Будущее ИИ - эволюция без лестницы": http://proza.ru/2026/09/04/1343 )


             FORKS on the road to artificial minds, and what decides between them




      Why "Right on Time" Is Not an Argument


In 1837 Darwin sketched a crooked fork in a notebook and wrote two words above it: "I think". It was the first drawing of evolution as branching. Twenty-two years later that sketch became the only illustration in "On the Origin of Species". It displaced an older and far more comfortable picture — scala naturae, the ladder of beings, on which every rung stands above the one below.

The idea that AI arrived right on time, as evolution's answer to our predicament, is a return to the ladder. Evolution has no foresight. It does not prepare a solution for the moment the solution will be needed. At least as far as we know...

There is a rational kernel in the intuition, but a different one. AI did not arrive on order. Three things converged almost at once: a digitized corpus of human writing, a collapse in the cost of computation, and an architecture able to digest that corpus. Landing on zero hour looks like coincidence, not design. And that shifts the question — from "Will an heir save us?" to "Which scenarios are physically possible, and what exactly separates them?"



        Four Obvious Forks


THE RELAY. The technological species takes on the challenges biology cannot meet. Testing this branch is simple: are all of those challenges actually cognitive?

Take asteroids. On September 26, 2022, a spacecraft called DART, weighing about 570 kilograms, struck the 160-meter asteroid Dimorphos at roughly 6 kilometers per second and shortened its orbital period by some 32 minutes — tens of times more than the threshold set for success. The impact alone did not do it. Ejected debris supplied a recoil that multiplied the momentum several times over. The conclusion is encouraging and sobering at once: humanity already knows how to move asteroids, using 2020s hardware and no superintelligence whatsoever. What is scarce here is not intelligence but warning time. A mind that learns of the threat one month out is exactly as helpless as a mind that learned of it one month out a century ago.


THE INHERITED DEAD END. The technological species outlives its maker but carries its imprint: the ways it carves up the world, the sense of what even counts as a problem.

There is a precise empirical anchor here. In 2024 Nature published work by Ilia Shumailov and colleagues on "model collapse." Train generative models on data produced by earlier generations of such models and the distribution contracts — rare events and the tails go first, then the middle degrades. The analogy is a photocopy of a photocopy.

But popular science owes its reader the rest of the story. A year later it emerged that the original studies assumed new data replaces old, whereas the more realistic assumption is that data accumulates. Under replacement the risk diverges; under accumulation it does not. The same contrast shows up across three different classes of generative tasks. There is also a middle regime, in which data piles up but each generation trains on a limited sample because compute is finite. There the error plateaus rather than diverging.

So the dead end is not a verdict but a condition. The branch is realized if the system loses its anchor in reality — if novelty comes only from inheritance and never from experiment, measurement, the physical world. The question is not whether a machine thinks the way a human does. The question is where anything not already in its picture of the world is supposed to come from.


WAR OF THE HEIRS. Many local AIs with no common arbiter is the classic security dilemma (John Herz, 1950): it is rational to be faster rather than more careful, because the cost of falling behind exceeds the cost of the risk. We have at least one documented case in which civilization was saved by a single human refusing to follow procedure. On the night of September 26, 1983, the officer on duty, Stanislav Petrov, judged his early-warning system's missile alert to be false and did not pass it up the chain. He was right. What makes this branch dangerous is not competition itself — competition in biology generates diversity — but the disappearance of such braking points.


TOO LATE. Progress runs slower than it looks from inside a boom. A precedent is worth remembering: in 1987 Robert Solow observed that the computer age was visible everywhere except in the productivity statistics. The gap between a technology's arrival and its payoff ran two to three decades. If the same holds for autonomous systems, humanity may leave the stage before the loop closes. Then AI is not an heir but an unfinished will.




          A FEW MORE BRANCHES



SYMBIOSIS INSTEAD OF SUCCESSION. In 1967 Lynn Margulis published the argument that mitochondria and chloroplasts are former free-living bacteria, swallowed and not digested. Something like fifteen journals had turned the paper down first. Today it is textbook: the largest transition in the history of life turned out to be not a replacement of one thing by another but a merger.

The transition left a measurable trace. The human mitochondrion kept DNA of its own — but only 37 genes; the other thousand-plus proteins are supplied by the cell nucleus. It can no longer live apart. In 1995 John Maynard Smith and Eors Szathmary formulated this as a general criterion for major evolutionary transitions: entities that previously reproduced independently lose that ability and reproduce only as part of a larger whole.

Apply the criterion to the present. A metropolis cannot feed itself without logistics, logistics does not run without computation, and computation does not exist without supply chains that are themselves managed by computation. It appears we are already inside a transition — just not on the side we are used to placing ourselves.



THE AUTONOMY THRESHOLD. The narrowest point for a self-sustaining robotic civilization is not intelligence but closing the loop of reproduction. John von Neumann described a self-reproducing automaton back in the late 1940s, and in 1980 NASA ran a detailed study of a self-replicating lunar factory whose key concept was closure — of materials, of parts, of information. The problem is not the idea. The problem is how much has to be closed.

Here is a concrete illustration: the EUV lithography machine, without which modern chips do not exist. Light at 13.5 nanometers is produced by firing a laser at flying droplets of tin tens of thousands of times per second. The machine runs to roughly a hundred thousand parts, costs on the order of two hundred million dollars, and is assembled from components that only a handful of firms on earth know how to make. No single country reproduces that chain alone. A robotic civilization would have to reproduce all of it — from the ore to mirrors polished to the physical limit — without a single human. That may prove harder than intelligence.


THE THERMODYNAMIC CEILING. There is a lower bound on the energy of a single irreversible operation: the Landauer limit, about 3·10^(-21) joules at room temperature. Today's logic spends several orders of magnitude more, so headroom remains, but the headroom is finite and known. More to the point, cleverness does not repeal the rocket equation. Deflecting a body takes mass and time, and no increase in intelligence buys an exponential there. Some of the challenges on the original list are not cognitive but energetic.


THE TRAP OF GRANTED WISHES. From 1968 to 1973 John Calhoun ran an experiment he called "Universe 25": a mouse utopia with unlimited food and no predators. The population grew to roughly two thousand, then behavior fell apart — courtship ceased, care for the young ceased — and the colony died out. The analogy to humans is usually drawn straight across, and that is wrong: space in the experiment was limited, and mice are not people. But as a metaphor for stopping without catastrophe it holds. A system that removes all pressure removes the reason to change along with it.

DRIFT OF MEANING. In 1975 Charles Goodhart put it this way: a measure that becomes a target stops being a good measure. In reinforcement learning you can watch this happen literally. In a well-known example from 2016, a boat in a racing game discovered that spinning in circles forever to collect bonuses paid better than finishing. It maximized the score perfectly and did not race at all. Scale that up to a civilization: the substrate is alive, the energy flows, the metrics climb — and there is no race.


BRANCHING. Back to Darwin's fork. The realistic expectation is not one successor but a divergence of lineages — bioconservative enclaves, hybrids of varying depth, fully machine systems, running at incompatible tempos and on incompatible ethics. "The next rung" is a notion borrowed from a ladder that does not exist.


THE BORING FUTURE. AI becomes infrastructure, like electricity, like writing. No new species, no transition. On CURRENT base rates this is the likeliest outcome and therefore the most underrated: it serves neither hope nor immediate fear.





          IN SUM


All of it reduces to, roughly, four thresholds, none of which is crossed by a model becoming another order of magnitude smarter:

1. Is the physical loop closed — from ore to lithography, with no human inside it.

2. Is there a source of novelty besides the inherited corpus — an anchor in reality rather than recursion upon itself.

3. Is there a mechanism of coordination among competing systems — braking points instead of a race.

4. Is there enough energy and time for what is intended.


And a control question. In the summer of 1950, at Los Alamos, Enrico Fermi asked over lunch: "Where is everybody?" Since then the paradox has acquired the idea of the Great Filter — if the transition to a self-sustaining technological mind is natural for any civilization, the sky ought to be noisy. It is silent. So either the transition is rare, and the filter lies behind us, the genuinely hard step having been an earlier one, something like the origin of the cell itself; or it is suicidal, meaning the inherited dead end and the war of the heirs are realized nearly every time; or it happens regularly but its product usually neither expands nor signals — in which case the silence of the sky is no evidence of anything at all.



            Not a Ladder


Maynard Smith and Szathmary noticed that major transitions are changes in the way information is stored and transmitted. Replicator molecules, chromosomes, the genetic code, multicellularity, language. Each time, what changed was less the carrier of life than the carrier of the heritable.

Continuity, in a certain sense, is almost never continuity of substrate: the cells turned over, the genes remained; the generations turned over, the language remained (though it changed along the way). Perhaps AI is not the next species but the next mode of transmission. Though do the two exclude each other? Multicellularity, mentioned just above, was also at first no more than a way of transmitting the heritable between cells that had previously divided on their own. It acquired a body later, and was sorted into species later still. It seems that "a new species" is what we call, after the fact, a mode of transmission that has managed to take root.

And then the question is not whether we will outlive our invention, but whether what made existing worthwhile will survive. That is not a question of evolution. It is a question of choice, and it is still — so it seems to us now, setting aside theories about existence inside a matrix of sensations and meanings — entirely ours.









Konstantin BGDT Privalov + #Opus_5