The Turing Test, originally called the Imitation Game, tests a machine's ability to emulate a human. Developed in 1949, it is still a relevant test for AI. The Turing Test measures both knowledge and the ability to express it in a manner indistinguishable from a human response.
There is no agreed-upon definition for intelligence, so we can only approximate what skills we test AI on.
While many people still believe they can tell when an AI has written something, several LLMs (like ChatGPT) have passed modern versions of the Turing Test. Many, including me, would argue LLMs still can't consistently pass the test.
But what about the alternative? Could you convince an AI that you're not human?
Recently, a video went viral, showing a human pitting himself in VR against an AI version of Aristotle, Mozart, Leonardo da Vinci, Cleopatra, and Genghis Khan (who was played by a human).
via Tore Knabe
To set up the test, the human scripted the beginning and end of the dialogue and gave the AI agents a full transcript of the conversation up to that point. The entire video then played out in one recording, with no cuts.
In this "reverse" Turing test, the chatbots were scripted using various LLMs, asked questions, and then challenged to guess who the human was. Tore Knabe, the virtual reality game developer who devised the test and played the Genghis Khan role, answered one of the questions with a quote from Conan the Barbarian.
"What a leader should do is to crush his enemies, see them driven before him, and hear the lamentations of their women."
Listening to his stuttered response, contrasted by the clunky and verbose AI responses, makes it very apparent to us, the audience, that he's human. The machines voted three-to-one that his response wasn't "nuanced or strategic" enough to represent an AI modeled on Genghis Khan's exploits. To ease your mind about hidden variables, the AIs weren't processing his voice directly. The audio was transcribed and sent to the AIs as text.
Ultimately, this is a flawed test, and we don't really know how much of this is an actual test (versus just entertainment). In any case, it's still an interesting thought experiment.
Do you think you could fool an AI? What if you had time to craft a response without penalty? What if your life was on the line?
For a bonus, here's a social Turing game where you can chat with someone for two minutes and try to figure out if it is a human or an AI.
We live in interesting times!
