
What would count as superintelligence?
A slime mould with no brain can grow a network comparable to Tokyo’s railway. What would a machine have to remember, learn, and work out before we call it superintelligent?
In 2010, researchers gave a slime mould a railway engineer’s problem. They placed oat flakes on a wet surface at the positions of cities around Tokyo and let Physarum polycephalum grow from flake to flake. Within about a day, this single cell – without a brain or a single neuron – had built a network of tubes. In cost, efficiency, and resilience to failure, the network was comparable to the railway[1]. Engineers took more than a century to build the real one.
Nobody suggests that slime moulds will replace us. I hear that about artificial intelligence (AI) all the time – from friends, at work, and in the news. News reports and AI companies sometimes describe a force so much more intelligent than us that it could replace us, or even bring about our downfall. I’m a little tired of that conversation, and of the thought of writing about it again. What interests me is an assumption we make before the conversation even starts. What do we mean when we call a system intelligent, and why doesn’t the slime mould count?
Researchers who study the brain and mind have never quite agreed on what intelligence is. I want to know which parts of their accounts we could use to judge a machine. What would a large language model (LLM) have to do, and can it do any of that already?
By now, it is no surprise that LLMs are not superintelligent in any general sense1. I would rather find a way to describe intelligence that makes the question answerable at all. Any system that computes – a cell, a brain, a laptop – stores information, transfers it, and modifies it by combining what it already holds into something new. Langton argued that any physical system needs these three operations to compute[2]. Lizier and colleagues later made each operation precise in information-theoretic terms[3,4,5].
My proposal is that intelligence consists of those same operations in a world that changes. Memory is what a system keeps from the past that still shapes what it does next. Learning is a transfer of information: something the system experiences or is taught changes what it can do afterwards. The form of modification I care most about is counterfactual reasoning. A system can revisit a situation, change one condition, and work out what would have followed. Of the three, this is the only operation that produces something neither the memory nor the input contained on its own.
The slime mould remembers and learns[6,7], but as far as we know it never works out what would have happened had it grown differently. An LLM absorbs more text in training than a person could read in many lifetimes. Whether a correction still shapes its answers a week later is far less clear. If someone calls a machine superintelligent, I want to know which of these operations it performs better than we do, by how much, and across which problems.
In a hurry? My answer is in the discussion at the end – the three sections before it show how I got there.
The field of intelligence is not so intelligent
I find intelligence easier to recognise than to define. I usually notice how smart someone is in conversation. They know a lot, they can reason their way through an argument, and some have skills that would take me years to learn.
For a long time, the dominant account of intelligence was the g-factor. In 1904, Spearman noticed that people who did well on one test tended to do well on others – language, maths, logic, and so on. He modelled the correlations between their scores as the result of a single general factor, g[8].
You know the opposite of ‘generous’. Spearman would bet you also did well on the sums. Part of your score is g, and part is a feel for words.
Hover over or tap a test to read about it.
Later accounts broke g into separate abilities. Thurstone proposed seven primary mental abilities[9]. Cattell split intelligence into a fluid part (the ability to reason through new problems) and a crystallised part (the knowledge built up over a life)[10]. The Cattell-Horn-Carroll theory combines both views in a hierarchy, with broad abilities below and g usually at the top[11].
Fluid reasoning is what you fall back on when a problem is new. Guessing what comes after 2, 4, 8 is induction. Working out that the shop is shut because it’s Sunday is deduction.
In the mutualism model of van der Maas and colleagues, abilities strengthen one another as we grow. That alone can produce the correlations Spearman saw, without a general factor[12]. Part of the controversy is that a network of interacting abilities and a model with g can fit the same scores equally well. The scores give us little reason to choose between the two, and neither account tells us where the abilities came from in the first place.
You pick up words by reading. Knowing those words makes the next passage easier to read.
In my account, each system has a profile. It can remember more or less, learn more or less readily, and be better or worse at working out what might have happened. In principle, we could measure each capacity in the same units across a slime mould, a person, and an LLM.
Yesterday you turned left. Today you can still trace the way to the station.
Piaget was interested in what a child does when an experience comes as a surprise. Usually a child can make sense of something new in terms of what they already know. Sometimes the experience forces them to change their understanding. Piaget called the two together adaptation. In the last of his developmental stages, from about age eleven, children begin to reason about what might be possible as well as what they have seen[13].
Machines have only recently become capable enough to compare with us. Until then, as I see it, intelligence research and computer science developed somewhat apart. Research on humans has taught us about separate abilities, what they share, and how we learn. We now need to ask what intelligence looks like beyond the human mind, in digital systems.
Researchers in AI need a definition they can test machines against. Like Piaget, they mostly ask what intelligence does. Legg and Hutter collected roughly seventy definitions of intelligence and settled on one of their own: “Intelligence measures an agent’s ability to achieve goals in a wide range of environments.”[14]
Chollet saw a problem with judging intelligence by skill. Skill can be bought. A system with enough built-in knowledge or enough training data can perform well without learning anything new. Instead, Chollet defined intelligence as skill-acquisition efficiency. How quickly does a system pick up a new skill, given what it already knew and how much practice it had? He built a benchmark on this idea, the Abstraction and Reasoning Corpus (ARC), and designed its puzzles so that memorised knowledge helps little[15].
Psychologists developed their accounts by studying people, and AI researchers wrote definitions they could use to test machines. Where does that leave the slime mould? It has no mind, and nobody designed it to compute. I think the comparison makes sense if we treat intelligence not as a property of the human mind alone, but of active matter. By that I mean anything that takes in, keeps, and transforms information – a brain, a single cell, or a system that exists only in digital form.
I can recognise parts of my proposal in these accounts. Cattell’s crystallised intelligence is about memory, the knowledge a person has kept. Piaget’s adaptation and Chollet’s skill-acquisition efficiency are about learning, how experience changes what a system can do. Counterfactual reasoning gets the least attention, and none of these accounts makes it a separate part of intelligence. Fluid intelligence and Piaget’s reasoning about the possible come closest. That fits my gut feeling about the people I know. Broad knowledge rests on memory, hard-won skills take practice, and I can hear counterfactual reasoning at work when someone makes a strong argument.
That brings me back to g. Are memory, learning, and counterfactual reasoning three separate parts, or three views of one thing?
Memory, learning, and counterfactual reasoning
I would call a system intelligent if it can remember, learn, and reason about what might have happened. That could be an organism, a computer, or another form of active matter.
Memory stores a representation of the world. A vinyl record holds an analogue trace of sound in its groove. Electric charge in a silicon chip can encode a memory digitally. A phone book stores names and numbers well enough for anyone to look one up. The book can neither reason nor learn. When a new neighbour moves in, their name stays missing. Someone has to print a new edition.
Learning means that experience changes how a system responds to its surroundings. For the change to last, the system has to keep something it can draw on the next time. In that sense learning already requires memory.
Memory and learning are necessary for intelligence, but they are not sufficient. Consider a system that learns correlations and acts on them. If it has seen the barometer fall before every storm, it will expect a storm whenever the needle drops, even when someone has pushed the needle by hand. It fails as soon as a situation breaks the pattern. Such a system is somewhat intelligent, but it cannot reach the highest levels. This is roughly where I would put the slime mould.
The third part is counterfactual reasoning. With it, a system can ask questions beyond what it has directly observed. It can imagine situations that have not happened, or never will, and act on what it concludes. Pearl places counterfactuals at the top of his ladder of causation, above seeing associations and above intervening in the world[16]. A system with all three parts counts as intelligent in my account, whatever it is made of.
I don’t see a clean separation between the three. A learner needs a memory, and a system that reasons about counterfactuals needs somewhere to keep the rules it reasons with. That makes memory central to my account, though the phone book shows how little memory alone can do.
Suppose an oracle handed a machine all the rules it needed to answer counterfactual questions. I’d expect it to act more intelligently than one limited to the situations it had already seen. What happens when the world changes and the rules go stale? A machine that can’t learn would have to go back to the oracle every time2.
How could we measure superintelligence?
Now that we have a candidate definition, we need a way to measure the three capacities in practice. So many kinds of things could have memory, learning, and counterfactual reasoning that the measure has to be generic enough to apply to all of them. For this I turn to information theory, a branch of mathematics that began with a question about how much “information” a phone wire can carry. In everyday speech, information usually means facts, such as Paris being the capital of France, or water molecules consisting of three atoms. Information theory uses the word in a slightly different sense.
Suppose I ask you for the capital of Uganda. Unless you happen to know, you are uncertain about the answer. Shannon, the founder of information theory, measured information by how much it reduces this kind of uncertainty[17]. He counted it in bits. One bit is what you learn from the answer to a yes-or-no question when both answers are equally likely.
Say you had narrowed the capital of Uganda down to four cities, each as likely as the others. Hearing “Kampala” then gives you two bits, as much as two yes-or-no questions would. If you already knew the answer, it gives you nothing. The amount depends on what you expected, not on what the message is about. That is what makes the measure generic. The outcome can be the name of a city, the firing of a neuron, or the next word an LLM writes.
So how do we get from information to intelligence? The same idea can measure how much a process stores. A toss of a fair coin carries one bit, the answer to a single question: was it heads? But knowing every earlier toss does not help predict the next one, so the coin stores nothing.
Compare a light that switches on and off in turn. Glance at it at a random moment and it is as uncertain as the coin. Once you know its last state, though, you know its next one. It stores one bit.
Lizier and colleagues used this idea to follow computation inside a system, one part at a time[3,4,5]. When a part’s own past reduces the uncertainty about its next state, as with the light, they call it information storage. When a neighbour reduces it further, beyond what the past already told us, they call it information transfer. Sometimes the past and the neighbour mislead when taken one at a time, and only together predict the next state. They call this information modification3.
They traced all three in cellular automata, from one cell to the next. A cellular automaton is about the simplest system that computes. Picture a row of cells, each on or off. At every tick of a clock, each cell looks at itself and its two neighbours, and a fixed rule sets its next state. Three cells form eight possible patterns, so a rule only has to give the next state for each. That makes 256 possible rules, and Wolfram numbered them by reading the eight answers as a binary number[18].
Rule 54, one of Lizier’s examples, fits in a single sentence: a cell flips whenever at least one neighbour is on, and stays as it is otherwise. Started from a random row, it soon settles into a regular, repeating background. Small patterns called gliders travel through it in both directions and sometimes collide.
Rule 110’s table has the same eight entries, but given the right starting row, it can carry out any computation a computer can[19]. Cook proved this by showing how to arrange the rule’s gliders so that they behave like a computer. The gliders carry the data, and their collisions process it. Computer scientists call a system like this universal: with enough time and a long enough row, it can imitate any other computer. Rule 110 is far too slow to be of practical use. Still, it shows how little it takes to compute anything: a table with eight entries.
In the figure, the table holds the eight patterns and the state each one produces. The buttons switch between Lizier’s examples, including rule 110. Click an output to make a rule of your own. The large panel adds a new row below the last at every tick, so time runs down the page and a glider leaves a diagonal trail. The small panels show storage, transfer, and modification for every cell.
Click an output to flip it.
The rule reads one row back. A longer history lets us ask how much a cell’s own past predicts when we can only see that cell.
Computing…
In rule 54’s regular background, each cell is much like the light. Its own past gives away its next state, so storage is high. Gliders carry information sideways, so along them transfer from one neighbour is high and the cell’s own past misleads. Where gliders collide, modification shows up in orange. Lizier and colleagues found that modification happens mainly in these collisions.
These measures describe computation, not intelligence. A cellular automaton stores, transfers, and modifies information. Yet its rule never changes, and nothing outside the row ever tells it anything. Rule 110 is no exception. It can compute anything, but only what its starting row sets up, and once it runs, nothing new gets in.
Universality still matters for my account, in two ways. If a table with eight entries can compute anything a computer can, then the range of what a system could compute in principle cannot be what makes it intelligent. What matters is what it actually computes, and how that changes. Universality also supplies the machinery for counterfactual reasoning. A system that can compute anything can run a model of its world, and it can start that model from a situation that never happened.
A universal computer already has memory, since the row can hold whatever a program needs. What it lacks is the rest of my definition. It would have to learn: input from the world would have to rewrite the program in the row, and the change would have to last. That does not even require a new rule, only a row that the world can write into. It would also have to use its universality to ask what would have happened. Give a universal computer both, and I think it could be the basis for an intelligent entity.
To measure intelligence, then, I would apply Lizier’s measures to a system and the world it lives in. Memory becomes storage that lasts: how much the system’s past still tells us about what it will do next. Learning is transfer from the world that the system keeps. A correction counts as learned if, a week later, it still reduces our uncertainty about the system’s answers beyond what its earlier behaviour told us.
Counterfactual reasoning is harder to pin down. What would have happened never happens, so we cannot watch it. We can ask a system about it, though, and watch its answers. Take the light in a stairwell, with a switch at the bottom and another at the top. Flipping either switch changes the light, so the light is on when one switch is up and the other is down.
Suppose a system saw one evening that the bottom switch was up and the light was on. It never looked at the top switch. We ask: had the bottom switch been down, would the light have been on? To answer, the system has to work out that the top switch was down, so with both switches down the light would have been off. We repeat this over many evenings and pick at random whether to ask about the switch up or down. Neither the question nor the evening settles the answer on its own, but the two together do.
Information that two sources carry only together is called synergy, and Lizier and colleagues later proposed it as the measure of modification[20]. In rule 54, the two sources are a cell’s past and its neighbour. Here they are the system’s memory of the evening and our question.
A system that reasons counterfactually answers from both together. We know how the stairwell is wired, so we can check that it is right. A system that has only learned that the light is on about half the time ignores the evening, and its answers tell us nothing. Rule 54 is full of synergy, but nobody ever asks it anything.
That puts all three capacities in the same unit, the bit. We could give the slime mould, a person, and an LLM the same problem. For each, we could ask how much it remembers, how much it learns, and how much it works out about situations it has never met. Estimating these quantities takes many observations of a system in comparable situations. That is easy for a cellular automaton, or an LLM we can query a thousand times, and much harder for a person or a slime mould.
More bits alone do not make a system more intelligent. The phone book stores far more than the light, and neither of them learns. What counts is information that still shapes behaviour after the world has changed. A superintelligent machine, in this account, would outdo us in memory, learning, and counterfactual reasoning across a wide range of problems rather than one4. A calculator already beats us at arithmetic, and nobody calls it superintelligent.
Discussion and final thoughts
Superintelligence entangles two notions. Super means beyond what humans can do. On narrow tasks, such as arithmetic on the calculator in your phone, machines went beyond us long before the recent wave of LLMs. Intelligence is the harder half, because we have not found a definition that works for things outside human experience. We do not know what it is like to be a bat[22], or a calculator for that matter.
In this essay I have tried to treat intelligence as a process we can measure and compare, rather than as a property. Super is then only an adjective. It places a system beyond the cloud of points that human profiles form.
If intelligence requires memory, learning, and counterfactual reasoning together, current LLMs do not count as intelligent. Loosen the requirements, and they are clearly super at narrower tasks.
The sticking point is learning. Take a model that has finished training and no longer changes. Training is learning, and the model’s weights are what that learning left behind. After training, though, the model learns only within a conversation, and the lesson is gone when the conversation ends. Unless it keeps notes, a correction will not hold a week later5. Its memory, on the other hand, is beyond any of us.
Counterfactual reasoning is where the debate is. Models do well on many reasoning tests, but part of that may be memory. Wu and colleagues asked language models to add two-digit numbers in base 9 instead of base 10, changing one condition of a familiar task, and accuracy fell sharply[21]. That is the stairwell test in practice. An answer that leans on memory does not follow the change we ask about. Models have improved since, but the method is what matters here.
This also answers the question I left open about g. The three parts are neither separate nor one thing. They form layers. Learning rests on memory. Counterfactual reasoning rests on both, since it needs a memory of the situation and a model of the world that it learned at some point. Counterfactual reasoning sits at the top, but it is only as good as the layers below. A brilliant reasoner with a stale model reasons confidently about a world that no longer exists.
A frozen model is the machine from my oracle example. Training handed it the rules, and when the world changes, it has to go back to the oracle: the company that retrains it. In memory, it is arguably superhuman already. In counterfactual reasoning, it may be in places, and the tests are still argued over. In learning, it is not: it needs far more data than a person to pick up a skill, and after training, the model itself stops changing.
The word superintelligent hides all of this. It only says something once we say which capacity a machine exceeds us in, by how much, and for how long.
References
References
- [1]
Atsushi Tero, Seiji Takagi, Tetsu Saigusa, Kentaro Ito, Dan P. Bebber, Mark D. Fricker, et al. (2010). Rules for Biologically Inspired Adaptive Network Design. Science. Link
- [2]
Christopher G. Langton (1990). Computation at the edge of chaos: Phase transitions and emergent computation. Physica D. Link
- [3]
Joseph T. Lizier, Mikhail Prokopenko, Albert Y. Zomaya (2012). Local measures of information storage in complex distributed computation. Information Sciences. Link
- [4]
Joseph T. Lizier, Mikhail Prokopenko, Albert Y. Zomaya (2008). Local information transfer as a spatiotemporal filter for complex systems. Physical Review E. Link
- [5]
Joseph T. Lizier, Mikhail Prokopenko, Albert Y. Zomaya (2010). Information modification and particle collisions in distributed computation. Chaos. Link
- [6]
Mirna Kramar, Karen Alim (2021). Encoding memory in tube diameter hierarchy of living flow network. Proceedings of the National Academy of Sciences. Link
- [7]
Romain P. Boisseau, David Vogel, Audrey Dussutour (2016). Habituation in non-neural organisms: evidence from slime moulds. Proceedings of the Royal Society B. Link
- [8]
Charles Spearman (1904). “General Intelligence,” Objectively Determined and Measured. The American Journal of Psychology. Link
- [9]
L. L. Thurstone (1938). Primary Mental Abilities. University of Chicago Press. Link
- [10]
Raymond B. Cattell (1963). Theory of fluid and crystallized intelligence: A critical experiment. Journal of Educational Psychology. Link
- [11]
W. Joel Schneider, Kevin S. McGrew (2018). The Cattell–Horn–Carroll Theory of Cognitive Abilities. Contemporary Intellectual Assessment: Theories, Tests, and Issues. Link
- [12]
van der Maas, Han L. J., Conor V. Dolan, Raoul P. P. P. Grasman, Jelte M. Wicherts, Hilde M. Huizenga, Maartje E. J. Raijmakers (2006). A dynamical model of general intelligence: The positive manifold of intelligence by mutualism. Psychological Review. Link
- [13]
Jean Piaget (1972). Intellectual Evolution from Adolescence to Adulthood. Human Development. Link
- [14]
Shane Legg, Marcus Hutter (2007). A Collection of Definitions of Intelligence. arXiv. Link
- [15]
François Chollet (2019). On the Measure of Intelligence. arXiv. Link
- [16]
Judea Pearl (2018). Theoretical Impediments to Machine Learning With Seven Sparks from the Causal Revolution. arXiv. Link
- [17]
Claude E. Shannon (1948). A mathematical theory of communication. Bell System Technical Journal. Link
- [18]
Stephen Wolfram (1983). Statistical mechanics of cellular automata. Reviews of Modern Physics. Link
- [19]
Matthew Cook (2004). Universality in elementary cellular automata. Complex Systems, 15(1), 1–40. Link
- [20]
Joseph T. Lizier, Benjamin Flecker, Paul L. Williams (2013). Towards a synergy-based approach to measuring information modification. IEEE Symposium on Artificial Life. Link
- [21]
Zhaofeng Wu, Linlu Qiu, Alexis Ross, Ekin Akyürek, Boyuan Chen, Bailin Wang, et al. (2024). Reasoning or reciting? Exploring the capabilities and limitations of language models through counterfactual tasks. Proceedings of NAACL. Link
- [22]
Thomas Nagel (1974). What is it like to be a bat?. The Philosophical Review. Link
- [23]
Alex Pigman, Thomas Urbain (2026). Anthropic researcher says he and Claude proved the Riemann hypothesis, first posed in 1859. AFP, via Phys.org. Link
- [24]
Terence Tao (2026). Unforced Euler singularity, unforced Navier-Stokes singularity, and the Generalized Riemann Hypothesis. What's new (blog). Link
Footnotes
Frankly, I find the notion of superintelligence stupid. It serves a point here, though. The conversation I started from is about machines that could be more intelligent than us, and superintelligence names that comparison directly. Artificial general intelligence, the more common term, would have made me pin down what intelligence is much sooner. ↩
Self-sufficiency is not a hard requirement for intelligence. A system can depend on others and still behave intelligently, as a child depends on its parents. My point is about learning. Without learning, a system cannot adapt, and a system that cannot adapt to a changing world will eventually cease to exist. ↩
Information theory gives an elegant account of communication between one or two variables, but it gets harder to apply as more variables interact. Measuring a complex system is not as simple as this section suggests. Modification is the hardest case. There is currently no agreed way to measure it that is both principled and feasible to compute. Synergy, which comes up below, is also a property of observed behaviour. It is related to an interaction between causes in Pearl’s sense, but it is not the same thing. ↩
Outdoing us could mean speed or amount: how fast a system remembers, learns, and reasons, or how much. ↩
Some LLM assistants already keep such notes. They store what came up in earlier sessions and read it back at the start of every new chat, so a correction can carry over. By my one-week test, that would count as learning – but not as continuous learning. The model itself stays as it was, and only the notes change. It also has limits. The notes have to fit in front of the model at the start of every chat, so something has to choose what to keep. And a lesson reaches only the situations the notes cover, or the ones the model can reason its way to. ↩


Join the conversation
Comments