Turing’s Two Penn’orth on Today’s AI Talking Points
Benjamin Skuse
What would Alan Turing have made of the 12th Heidelberg Laureate Forum? To start, it is easy to picture him marvelling at all the ‘computer scientists’ – a term coined just a few years after his death in 1954 – swarming around him. He would then be awestruck by the tiny computers each and every one of them (and every tourist on the street nearby for that matter) had in their hand or pocket. The most powerful computer Turing would have heard of was the Naval Ordnance Research Computer (NORC); under construction at the time of his death. Developed by IBM for the U.S. Navy Bureau of Ordnance, the large room-sized NORC’s 15,000 operations per second pales in significance to the trillions of operations per second a typical 2025 smartphone can perform.
Bizarrely, however, despite the staggering leaps in computing technology witnessed since Turing’s time and its growth in society – from specialist scientific and mathematical calculator to near-ubiquitous universal tool extending human capabilities – the topics on everyone’s lips were ones Turing would not only be familiar with, but be able to contribute to insightfully. Are machines (LLMs) becoming so human-like and useful in their responses that they will replace many roles? Can machines (AI) solve all of mathematics? And will machines (AI) develop superhuman capabilities?
Thoughts on Imitating and Replacing Humans
A talking point that is far from restricted to the Forum is the growing capabilities of AI to carry out human tasks, with chatbots based on LLMs particularly imitating humans in ever more realistic and useful ways. Specific impacts for science were discussed in back-to-back Hot Topic sessions ‘The Machine-Learning Revolution in Mathematics and Science’, while broader societal consequences were the focus of the Spark Session talk ‘Shaping AI’s Impact to Help Billions’ from Jeffrey Dean (ACM Prize in Computing – 2012) and David Patterson (ACM A.M. Turing Award – 2017).
On this topic, Turing was decades ahead of his time in respect to thinking about how computers might one day be capable of exhibiting human-like intelligence. In his 1950 paper Computing machinery and intelligence, he began: “I propose to consider the question, ‘Can machines think?’,” before introducing the famous Imitation Game, now more commonly referred to as the Turing Test.
This test is a game between three participants. One is a human interrogator, and the other two are a human and a machine playing the part of separate interviewees. Through a simple question and answer format, the object of the game for the interrogator is to determine which of the other two is human.
Turing’s original specimen questions and answers are informative in not only revealing how the Turing Test works in practical terms, but also how strikingly similar the back-and-forth conversation is to the way in which many people now interact with chatbots on a daily basis:
Q: Please write me a sonnet on the subject of the Forth Bridge.
A: Count me out on this one. I never could write poetry.
Q: Add 34957 to 70764.
A: (Pause about 30 seconds and then give as answer) 105621.
Q: Do you play chess?
A: Yes.
Q: I have K at my K1, and no other pieces. You have only K at K6 and R at R1. It is your move. What do you play?
A: (After a pause of 15 seconds) R-R8 mate.
It is often claimed, though also widely contested, that the Turing Test was first passed in 2014 by a chatbot named Eugene Goostman, which simulated the personality of a 13-year-old Ukrainian boy. Eugene fooled 33% of the judges at the Royal Society in London that it was human. More recently, in work yet to be peer-reviewed, researchers have made the unverified claim that GPT-4 has passed as human over 50% of the time, and GPT-4.5 73% of the time (which is better than the 67% actual humans achieve!).
Despite these results, for most, the Turing Test has not been successful in answering whether machines can think, as was originally proposed. Neither does it test true intelligence in many people’s opinion. For instance, philosopher John Searle used a Turing Test-like argument against machines ever possessing actual understanding.
In his Chinese Room argument, he devised a scenario in which an English speaker locked in a room, who cannot speak a word of Chinese, could follow detailed instructions in English for manipulating Chinese symbols in response to Chinese symbols (text) provided to them. The responses would be “absolutely indistinguishable from those of Chinese speakers” even though the English speaker would remain completely ignorant of Chinese. Searle argued computers similarly just follow instructions without true understanding.
However, the Turing Test paradigm – that two things are the same if they cannot be told apart by any reasonable test – has been staggeringly successful as a gauge of the progress in AI, and useful to scientific and technological innovation.
Moreover, the original Turing Test has inspired more stringent versions that are even more of a challenge for machine intelligence. To pass the Total Turing Test, for example, requires machines to be capable of perceiving and acting in the world in a way indistinguishable from a human – a feat that has not been achieved. And the Lovelace 2.0 Test has not been definitively passed yet either. This assesses computational creativity and intelligence by requiring a machine to create a story, poem, painting, etc, that meets a complex set of constraints specified by a human judge.
Thoughts on Mathematics
Kicking off the main programme, Sanjeev Arora’s (ACM Prize in Computing – 2011) brief talk during the first Spark Session ‘Will there be a superhuman AI mathematician?’ mulled over the idea of machines solving mathematical problems that have remained unsolved for decades and even centuries. Delving deeper, the Hot Topic session ‘The Machine-Learning Revolution in Mathematics and Science, Part 1: Mathematics’ outlined the potential of such a tool, as well as the ethical, social and professional risks for mathematicians.
Turing’s legendary 1936 paper On computable numbers, with an application to the Entscheidungsproblem has a lot to say on this topic. In it, Turing laid down the foundations for computer science, giving us the concepts of algorithms and computation with his ‘automatic machines’, later called Turing machines. “It’s hard to believe that this one maths paper probably impacted the world more than any other paper written,” said Avi Wigderson (Nevanlinna Prize – 1994, Abel Prize – 2021, ACM A.M. Turing Award – 2023) in his Lecture ‘Reading Alan Turing’. “It basically brought on the computer revolution.”
A Turing machine is a simple abstract device Turing conjured up to explore computation. It consists of a tape on which is printed a line of cells. Each cell on the tape can be blank or contain symbols 0, 1 or something else. Therefore, the tape can be thought of as the computer memory, extending infinitely in each direction.
The ‘active cell’ is the one over which the machine’s head is positioned. The head can perform three operations: read the symbol on the active cell; edit the symbol; or move the tape left or right one space. Which operation it performs depends on the input and the rules of the machine, which are much like machine-code instructions.
Surprisingly, this relatively simple abstract machine can be considered a model of a general-purpose computer. And by defining it precisely, Turing was able to prove many of the potential capabilities and limitations of computation in general.
The key limitation relevant to superhuman AI mathematicians is that Turing proved the uncomputability of the Entscheidungsproblem; in other words, he showed that no universal algorithm can exist that can take any logical statement and determine whether it is true or false. Therefore, no computer, no matter how advanced, can prove all mathematics.
Even if AI technologies conspire to take humans completely out of the loop, as Arora outlined as a possibility, there will still be a subset of mathematical theorems that computers cannot solve. This remains as true today as when Turing predicted it in 1936.
Also in this celebrated paper was a key capability of any superhuman AI mathematician: Turing envisaged the possibility of machines being capable of verifying proofs, just as we are seeing today with AI-based formal proof assistants. Turing called these devices choice machines, capable of going beyond normal deterministic computation: “The machine can have a non-deterministic decision whether to go to one state or another,” explained Wigderson in his Lecture. “This defines what happens in proof verification – somebody supplying the proof and the machine just verifies it.”
Thoughts on Superhuman Capabilities and Learning
Discussions of superhuman AI capabilities were not restricted to mathematics at the Forum. In two Spark Session talks, David Silver (ACM Prize in Computing – 2019) and his former PhD advisor Richard S. Sutton (ACM A.M. Turing Award – 2024) outlined the wide-ranging transformative potential of AI that learns from its own experiences of the world. They both argued that through continually generating data by interacting with its environment, combined with powerful self-improvement methods, AI will transcend human knowledge and capabilities.
In a different Spark Session, Leslie Valiant (Nevanlinna Prize – 1986, ACM A.M. Turing Award – 2010) spoke of ‘educability’ – the capability to learn and acquire belief systems from one’s own experience and from others, and to apply these to new situations – being the innate core capability that currently distinguishes humans from machines. However, he believes that ‘supereducated’ machines could attain this skill in future, thereby leading to superhuman capabilities.
These arguments bear a striking resemblance to that given in one of Turing’s talks to the London Mathematical Society in 1947, where he called for “a machine that can learn from experience” by devising a way to let the machine alter its own instructions. In his own words:
Let us suppose we have set up a machine with certain initial instruction tables, so constructed that these tables might on occasion, if good reason arose, modify those tables. One can imagine that after the machine has been operating for some time, the instructions would have altered out of all recognition, but nevertheless still be such that one would have to admit that the machine was still doing very worthwhile calculations. Possibly it might still be getting results of the type desired when the machine was first set up, but in a much more efficient manner.
In such a case one would have to admit that the progress of the machine had not been foreseen when its original instructions were put in. It would be like a pupil who had learnt much from his master, but had added much more by his own work. When this happens, I feel that one is obliged to regard the machine as showing intelligence.
As this eerily prescient quote shows, Turing may not have lived to see the smartphone or the rise of generative AI, but his thoughts and opinions – his timeless ‘two penn’orth’ – remain remarkably relevant at this critical moment in the advancement of computation and AI.
The post Turing’s Two Penn’orth on Today’s AI Talking Points originally appeared on the HLFF SciLogs blog.