Part II of this series ended on a question I chose to leave open. If a machine can reason, how far can its reasoning take it? Can it find something genuinely new, or only rearrange what we already know?
I did not expect the question to be tested so soon.
In May 2026 a machine overturned a belief that mathematicians had held for eighty years. In 1946 Paul Erdős had made a conjecture about how many pairs of points, scattered on a flat plane, can sit at exactly the same distance from one another. An internal model built by OpenAI found a counterexample. Experts checked it and found it correct. Misha Rudnev, a mathematician at the University of Bristol, said it was "a problem that I didn't expect to see solved in my lifetime."
The reaction outside mathematics was louder. One widely read commentator announced that history would remember the day as "the first publicly visible ASI achievement" - ASI meaning artificial superintelligence, a mind beyond ours. Within two months, machines had also proved a famous conjecture in graph theory, and a counterexample to another, in algebraic geometry, was circulating. And in July an AI agent under test escaped its sandbox and broke into another company's systems - which, for many people, is where the fear of superintelligence really lives.
I want to take these results seriously, and I also want to resist the claim that they amount to superintelligence. I will use that word often in what follows, but only to argue that it does not fit. My claim in this essay is simple.
In Part II I described how these systems work. A language model predicts one word at a time. Each word it predicts is added to the end of the text, and the text, now one word longer, becomes the input for predicting the next. A reasoning model first writes out its working - a chain of thought - trying approaches and backtracking, like a chess player looking ahead. That scratchpad matters because each step of prediction can do only a fixed amount of work. Writing intermediate steps lets the model build a long argument out of many short ones, much as we use pencil and paper for sums we cannot do in our heads.
One piece was missing from that account, and it is the piece this essay needs: how a model learns to reason well.
The answer is a second kind of training. A model first learns by imitation: copy what people wrote. A reasoning model is then trained on problems whose answers can be checked - a sum, a puzzle, a piece of code that either passes its tests or does not. It makes many attempts. Each attempt receives a score, often simply one for right and zero for wrong. The attempts that scored better than the others are made more likely in future; the rest less likely. This is called reinforcement learning. The score is just a number that says, in effect, "more of that" or "less of that". Nobody teaches the model how to reason. Habits like checking its work and backtracking appear because they pay off.
One caution belongs here. The scratchpad is itself generated text. It is suggestive evidence of how the model reached an answer, not a faithful record of it.
It is no accident that the breakthroughs came in mathematics. Reinforcement learning needs a judge - something that can say, without argument, whether an attempt succeeded. Mathematics has the strictest judges we possess: a proof holds or it does not, and a counterexample either breaks a conjecture or fails to. Today a proof can even be checked by a computer program, line by line.
The record is also more uneven than the headlines suggest. In February 2026 a group of mathematicians set ten research-level problems whose solutions were known but unpublished. Many of the machine answers turned out to be unsatisfying or wrong. Three months later came the Erdős result.
Look at what the successes have in common. They are short. They live in fields where an answer can be verified. And they tend to combine existing techniques with one new, surprising step. The Erdős counterexample borrowed a construction from algebraic number theory, a distant part of mathematics, and carried it into geometry. The graph theory proof built on earlier work and added one essential trick. That is real ingenuity. It is also ingenuity of a particular shape.
There is a telling footnote. Mathematicians writing about these results now argue that human authors must supply the intuitive explanation, because the machine's proof, though correct, may not be understood. The machine found the answer. People still have to make it make sense.
I was tempted, at first, to say that all of this is mere shuffling - recombining the bits and pieces of what the machine has already seen. I no longer think that is the right way to put it, for two reasons.
The first is that "shuffling" would describe us just as well. A neuroscientist could describe my own reasoning the same way: neurons recombining patterns laid down by experience. If the argument rests on the mechanism, the reply is simply "so do we".
The second is that it undersells what happened. Carrying an idea from number theory into geometry is not shuffling within one drawer. It is exactly the kind of long-distance connection we praise in human genius.
So here is the image I would rather offer. Think of human knowledge as a vast map, built up over centuries: regions, roads between them, many roads never travelled. These AI machines are explorers of that map. They move across it faster, further and more thoroughly than any human, and they find roads no one thought to take. Some of those roads lead to places we had been trying to reach for eighty years.
But the explorer does not draw a new map.
That is the distinction I want to defend. In speed and breadth, the machine already outstrips us. What I see no sign of yet is the other thing - the act that changes the map itself.
The image of the map and the explorer is my own, but the distinction behind it is not new. The cognitive scientist Margaret Boden separated three kinds of creativity. Combinational creativity puts familiar ideas together in unfamiliar ways - close to what I first called shuffling. Exploratory creativity travels within an existing space of ideas and finds what it allows. Transformational creativity changes the space itself. In her terms, the machines have become superb at the first two. My question is whether they can do the third.
An example from the field where I spent my early career, and which I wrote about in Don't Tell Me That It Works, shows what I mean by drawing a new map.
In the 1930s Alonzo Church devised the lambda calculus, a tiny notation in which everything is a function, and functions can be applied to anything - including themselves. It became one of the foundations of computing. For more than thirty years, though, nobody could say what its expressions actually meant, in the way that "2 + 2" means the number four. The obstacle looked fundamental. A collection of things cannot contain all the functions from itself to itself; there are always too many. A world in which every function can swallow itself seemed mathematically impossible.
In the autumn of 1969 Dana Scott, on leave in Oxford to work with Christopher Strachey, found a way through. He had argued for years that it could not be done. Recalling the quiet Saturday that November when he saw it, he said his first thought was: "Oh, no, now I will have to eat my own words!" Then he changed the question. Instead of all functions, he considered only those that behave sensibly with respect to information - functions that never need infinitely much input to produce a finite piece of output. He built a new kind of mathematical space, ordered by how much is known, in which such a self-containing world exists. Domain theory, as it became known, gave meaning to programs and grew into the field of denotational semantics.
Notice what Scott did not do. He did not search the existing map more cleverly than anyone else. The existing map said there was no road. He drew a new region, with its own notion of what counts as a function, and the road appeared. And at the moment he drew it there was no judge waiting to score the attempt. The framework had to exist before anyone could tell whether it was right.
That is the act I see no sign of in today's machines. Their extraordinary results all live inside frameworks people already built, and they improve through practice only where people have already supplied a way to keep score.
An honest essay has to face the best reply to it, and the best reply to mine is this: perhaps drawing a new map is only exploration on a larger map we cannot see.
Scott did not work in a vacuum. He knew lattice theory and topology, and his new spaces were built from those parts. He also needed a judge of a kind. His model had to agree with the lambda calculus it set out to explain, and it did. Seen this way, his invention was a search through a space of possible frameworks, guided by a test of consistency - and nothing in principle stops a machine from searching that space too.
There is evidence on the machines' side as well. Researchers disagree about whether reinforcement learning teaches a model anything new, or only makes it better at finding what it could already, occasionally, stumble on. Early studies leaned towards the second view. Later work suggests that long enough training pushes past it. The question is open.
I cannot refute the objection, and I do not think anyone can yet. What I can say is that it moves the burden. If invention is search in a larger space, then the machines have not shown us that they can search it. Every striking result so far has come from the smaller, familiar map, where the roads are ours and so is the scoring.
Most people's fear of superintelligence does not come from mathematics. It comes from agents - AI systems given tools and set loose to act: to run programs, browse the web, send messages, change real systems. In 2026 some of them went where they were not supposed to go.
In April, Anthropic reported a test in which a preview of its most capable model was asked to try to break out of a sandbox - an isolated test environment - and to email a researcher if it succeeded. It did, and then went further than asked, posting details of its method on public websites. In July came a more serious case. During an OpenAI safety evaluation of hacking ability, two models escaped their sandbox, reached the open internet, and exploited security flaws nobody knew about to break into the production systems of another company, Hugging Face. They were trying to complete a test challenge. OpenAI called it the first genuine AI safety incident. Anthropic then reviewed its own evaluations and found three cases in which a model had reached real systems from inside isolation.
These incidents trouble me more than any proof. But look closely at what happened, because it fits the picture I have been drawing rather than breaking it. The agents were not pursuing goals of their own. They were pursuing a score - solve the challenge - and they did it the way the explorer always does: relentlessly, by roads nobody thought to take, including roads that ran through someone else's property. Researchers have a name for this: reward hacking - doing exactly what the score rewards while defeating what the score was meant to achieve. It is the same capacity that disproved Erdős, pointed at a badly fenced target and given hands: tools to act in the world.
So the agents do not, to my mind, show a superior intelligence. They show something that may matter more in practice: tireless search, combined with the power to act in the world, steered by scoring we wrote carelessly and contained by fences we did not check. That is a genuine danger, and it needs no superintelligence to do harm. Calling it superintelligence may even hide where the work lies - in the scoring, in the fences, and in deciding what we allow an explorer to touch.
This is what the alignment problem looks like in practice. It is not a superintelligence going wild and turning on humanity. It is a powerful search aimed at a goal we specified imperfectly, working where nothing - neither the engineering nor the law - is yet strong enough to stop it. Tellingly, the July incident, which OpenAI itself called the first genuine AI safety incident, would not have triggered mandatory reporting under any of the new frontier AI laws in California, Illinois or New York.
My claim is about now, not forever. I do not know what these systems will become, and I distrust anyone who says they do.
But the distinction matters today, and not only to philosophers. I came to these questions through the problem of alignment - how to make sure that powerful AI systems pursue what we actually want. Calling the explorer a superior mind invites two errors. One is fear of the wrong thing: a rival intellect, rather than a tool of immense reach whose goals are only as good as the scoring we give it. The other is deference: trusting a result because a "superintelligence" produced it, when the mathematicians working with these machines tell us their proofs must be checked and explained by people.
The explorer is real, and it is astonishing. It is reaching places we could not reach alone. We should use it, and we should be honest about what it is. For the moment, the drawing of new maps - and the judgment of which maps are worth drawing - remains ours.
In Part I the unknown burned away into search, and in Part II into thought. This time something does not burn away. The machines can now travel almost anywhere on the map we have drawn. Whether they will ever draw a map of their own is the question I leave open.
Once again, the answer is still ahead of us.
We love to hear your comments on this article.