Bots fail to live up to Turing Test - again
Published: 22 Oct 2003 14:35 BST
Artificial Intelligence (AI) 'bots' still have a lot of evolving to do before they can successfully impersonate human behaviour in a chatroom, according to the results of this year's Loebner Prize Contest in AI.
The Loebner Prize is based on a prediction made in 1950 by Alan Turing, the famous British mathematician and code breaker. According to Turing, in the year 2000, computer programming would have advanced to a stage where, after five minutes of conversation, the "average interrogator" would only be able accurately to tell if they were talking to a human or a machine 70 percent of the time.
Although this means that the computer would only fool humans one in every three attempts, it still poses a huge challenge for AI programmers. This year, the Loebner Prize contest was held in Guildford. Two humans and ten machines were hidden away in a "server room" and nine judges had a five-minute conversation with each of the contestants in order to try and guess if they were communicating with a human or a machine. Marks were awarded out of five, with five meaning "definitely human" and zero meaning "severe system malfunction" -- and obviously a computer.
Predictably, the AI bots were unable to convince the testers that they were human, but top of the pile was Jabberwock, a creation of Juergen Pirner from Hamburg in Germany. Jabberwock managed to score 1.928, which -- according to the ratings -- puts it on the boundary between being "definitely a machine" and "probably a machine". So although the judges were able to tell it was not a human, the bot did sow a seed of doubt.
However, the testing procedure may be flawed, because even the human decoys did not convince the judges they were definitely made of skin and bones. The highest score was 3.867, which according to the ratings means "could be a machine or human; undecided".
The only British entrant was Jabberwacky, which came in joint fifth place (joint third place if the human entrants are discounted) and was created by Rollo Carpenter.
Did you find this article useful?
94 out of 196 people found this useful
Digg
Slashdot
Del.ici.ous
Stumble
Reddit
