More Derision for Turing Test “Pass”

“A great science fiction detective story” – Ian Watson, author of The Universal Machine
“Cutting edge speculative fiction” – Ernest Hogan, author of Cortez on Jupiter
“Sharply erudite, with the vicious tang of cordite”– Paul Morris, author of Time Traveller Danny and the Codebreaker

Nas LD Pirate Banner

The novel, the album, and the videos.  One bundle, free forever

Days since the Centenary: 723
Days to the Bicentennial: 35,801

Further to the post of a few days ago (“Did a Supercomputer Finally Pass the Turing Test? Don’t Bet On It“), skepticism about the alleged passing of the Turing Test by a chatbot named Eugene Goostman is spreading, and in some instances turning to outright derision.

The story–with a focus on the failings of the supposedly groundbreaking event–was the lead story in the June 15 edition of BBC’s The Science Hour (available online here for six days as of today).

BBC's The Science Hour

BBC’s The Science Hour

On the menu were some complaints heard previously, including the gaming of the system by having the chatbot claim to be a young boy and a non-native speaker of English (the language in which the test was conducted), leading judges to treat its errors with indulgence, as well as the five-minute time limit on the test.

Meanwhile, over on the web site of the Institute for Ethics and Emerging Technologies, Ben Goertzel, author of Creating Internet Intelligence and The Path to Posthumanity, called it:

… a bogus event trumped up by someone’s PR agent in an attempt to get some excitement about their work.

You can watch the entire Goetzel interview below.

By now the University of Reading, where the test was held, must rue its decision to allow itself to be associated with such transparently sensational baloney.  Presumably it didn’t appear so calculatedly disingenuous in advance, but it’s looking pretty shoddy in retrospect.

 

Posted in Uncategorized | Leave a comment

Did a Supercomputer Finally Pass the Turing Test? Don’t Bet On It.

“A great science fiction detective story” – Ian Watson, author of The Universal Machine
“Cutting edge speculative fiction” – Ernest Hogan, author of Cortez on Jupiter
“Sharply erudite, with the vicious tang of cordite”– Paul Morris, author of Time Traveller Danny and the Codebreaker

L+D Soundtrack

Now Available: the Free Soundtrack for the Novel! Click the Banner to Stream or Download.

Days since the Centenary: 719
Days to the Bicentennial: 35,805

The headlines have been nearly ecstatic–and almost uniformly uncritical.  Here are just a few:

Well, no.

As several commentators have pointed out, the “victory” is pretty dubious. Mike Masnick at TechDirt was quick to question the alleged result, listing several important points that call it into question:

  1. It’s not a “supercomputer,” it’s a chatbot. It’s a script made to mimic human conversation. There is no intelligence, artificial or not involved. It’s just a chatbot.
  2. Plenty of other chatbots have similarly claimed to have “passed” the Turing test in the past (often with higher ratings). Here’s a story from three years ago about another bot, Cleverbot, “passing” the Turing Test by convincing 59% of judges it was human (much higher than the 33% Eugene Goostman) claims.
  3. It “beat” the Turing test here by “gaming” the rules — by telling people the computer was a13-year-old boy from Ukraine in order to mentally explain away odd responses.
  4. The “rules” of the Turing test always seem to change. Hell, Turing’s original test was quite different anyway.
  5. As Chris Dixon points out, you don’t get to run a single test with judges that you picked and declare you accomplished something. That’s just not how it’s done. If someone claimed to have created nuclear fusion or cured cancer, you’d wait for some peer review and repeat tests under other circumstances before buying it, right?
  6. The whole concept of the Turing Test itself is kind of a joke. While it’s fun to think about, creating a chatbot that can fool humans is not really the same thing as creating artificial intelligence. Many in the AI world look on the Turing Test as a needless distraction.

I personally think that the test still has an important place in our thinking about artificial intelligence, but there’s certainly nothing wrong with questioning its value–in science there is no such thing as unquestionable canon, after all–and Masnick’s other points are pretty much on the money.

The Guardian also ran with the original dramatic story (“Computer simulating 13-year-old boy becomes first to pass Turing test“), prompting a number of comments taking it to task, which it had the good sense to publish as well (“Claims that the Turing test has been passed are nonsense“).

For instance, Professor Robert Epstein of the American Institute for Behavioral Research and Technology (bio here) wrote:

Professor Warwick’s claim that a computer has now passed the Turing Test [...] is nonsense. Turing never set a 30% mark as a criterion for “passing” his test. In his famous essay on this topic, which is reprinted with commentaries in my book, Parsing the Turing Test: Methodological and Philosophical Issues in the Quest for the Thinking Computer, Turing merely conjectured that by 2000 a computer program would be able to fool an “average interrogator” into thinking it was a person 30% of the time in a five-minute conversation. He didn’t propose that as a test of anything; he was merely speculating.

Turing never actually said how his test could actually be passed, but a blue ribbon panel of computer scientists and philosophers from Harvard, MIT, and elsewhere which I directed for several years in planning the first Loebner Prize contest in 1990, came up with with a brilliant method that I am sure would have pleased Turing greatly: after lengthy conversations with both hidden humans and hidden computers, a panel ranks the humanness of each, and when the median rank of a computer exceeds the median rank of a human, it wins. No computer has ever crossed that line in the more than 20 years the contest has so far been held, but it will happen eventually.

Buzzfeed also managed a skeptical response (“No, A Computer Did Not Just Pass The Turing Test“).  Professor Murray Shanahan of the Department of Computing at Imperial College London told Buzzfeed:

Of course the Turing Test hasn’t been passed. I think its a great shame it has been reported that way, because it reduces the worth of serious AI research. We are still a very long way from achieving human-level AI, and it trivialises Turing’s thought experiment (which is fraught with problems anyway) to suggest otherwise.

And The Verge, which reported the alleged “pass,” also reported on some skeptical reactions (“Google futurist Ray Kurzweil and other experts say chatbot didn’t pass Turing Test“), including one from Ray Kurzweil:

I chatted with the chatbot Eugene Goostman, and was not impressed.  Eugene does not keep track of the conversation, repeats himself word for word, and often responds with typical chatbot non sequiturs.

Others echoed his distrust of the hyped announcement:

New York University cognitive science professor Gary Marcus agrees, writing in The New Yorker that the test wasn’t taken by “innovative hardware but simply a cleverly coded piece of software.” Marcus writes that the chatbot often resorts to misdirecting the person it’s speaking with using humor so that it can avoid questions that it doesn’t understand. “It’s easy to see how an untrained judge might mistake wit for reality, but once you have an understanding of how this sort of system works, the constant misdirection and deflection becomes obvious, even irritating,” Marcus writes. “The illusion, in other words, is fleeting.”

Marc Andreessen, co-founder of Netscape and one of today’s biggest names in tech investing, isn’t taking much stock in the claims over this chatbot either. “My view is that [the] Turing Test has always been malformed,” he writes on Twitter. “Humans are too easy to trick, passing [the] test says almost nothing about software.”

Marcus gave a sample of the chatbot’s chatting in his New Yorker piece (“What Comes After the Turing Test“):

Marcus: Do you read The New Yorker?

Goostman: I read a lot of books … So many—I don’t even remember which ones.

Marcus: You remind me of Sarah Palin.

Goostman: If I’m not mistaken, Sarah is a robot, just as many other “people.” We must destroy the plans of these talking trash cans!

At least in this small sample, it doesn’t seem distinguishable from other chatbots I’ve seen.

As much as I would enjoy the drama of seeing the Turing test actually passed, a little more critical thought would have made for less hyped, more accurate reporting.

If the horizon is populated by terminators, it’s a ways off yet.

Posted in Turing Test | 2 Comments

Hum Along to Turing with the Pet Shop Boys

“A great science fiction detective story” – Ian Watson, author of The Universal Machine
“Cutting edge speculative fiction” – Ernest Hogan, author of Cortez on Jupiter
“Sharply erudite, with the vicious tang of cordite”– Paul Morris, author of Time Traveller Danny and the Codebreaker

L+D Soundtrack

Now Available: the Free Soundtrack for the Novel! Click the Banner to Stream or Download.

Days since the Centenary: 666
Days to the Bicentennial: 35,858

The Pet Shop Boys are icons of 1980s electronic music.  They’re also the most successful music duo that the United Kingdom has ever produced.  And they’re Turing Elves.

The Pet Shop Boys, then and now.

The Pet Shop Boys, then and now.

This July, at the BBC Proms music festival, the Boys will present their newest work, A Man from the Future, based on the life of Alan Turing and including text from Alan Turing: The Enigma, by Andrew Hodges.  

The Boys are reported to have worked closely with Hodges, as well as collaborating with Sven Helbig (who they previously worked with on their ballet, The Most Incredible Thing).

As we’ve seen in previous posts on this site, Turing has inspired rather a lot of music recently:

Thus far the only part that’s been made public is a song called “He Dreamed of Machines” (embedded videos below).

The Boys have previously interceded on Alan’s behalf, sending a message to David Cameron’s assistant (in the period before Turing finally received a royal pardon in 2013) that read in part:

…sorry to bug you, but could you pass on to the Prime Minister that in Alan Turing’s centenary year it would be an amazing, inspirational thing to do to pardon him?”

On their web site the Boys have said:

“It is an honour for us to be invited to present some new music at The Proms and to celebrate Alan Turing 60 years after his death.”

Andrew Hodges, meanwhile, has been quoted as saying

“It’s just been wonderful working with Neil and Chris on this project”.

Here’s “He Dreamed of Machines,” the JCRZ remix:

I remember the 80s, and I have decidedly mixed feelings about the era, but my feelings about A Man From the Future are anything but:  I’m waiting with Frankenfurterian antici—–pation.   When there’s more to report, it will be reported right here.

And for those of you who don’t remember the 80s, let’s close for now with the Boys back in the day.

Posted in Uncategorized | Leave a comment

Turing Elves Strike Again: New Turing Infographic

“A great science fiction detective story” – Ian Watson, author of The Universal Machine
“Cutting edge speculative fiction” – Ernest Hogan, author of Cortez on Jupiter
“Sharply erudite, with the vicious tang of cordite”– Paul Morris, author of Time Traveller Danny and the Codebreaker

L+D Soundtrack

Now Available: the Free Soundtrack for the Novel! Click the Banner to Stream or Download.

Days since the Centenary: 666
Days to the Bicentennial: 35,858

We’re almost two years past the Turing centenary now–in fact we’ve gone some ways toward the bicentennial–but the momentum that began in the lead-up to the centenary is still in evidence.

Regular readers will know that a Turing Elf is someone who creates a work in relation to Turing out of private passion rather than as part of a set of official duties, and the Turing Elves are still out there cobbling together works of art, educational materials, memorials and tributes, and iPhone apps.

Today’s column honors Steve Ollington, Lis, and the Manchester Jury’s Inn, who together have created an online infographic that’s a mini-tutorial on Turing’s life and work.

As Steve wrote, the Jury’s Inn:

…recognise his importance as key figure in British History and a national treasure, and they liked the thought of providing a memorial that would engage, teach and honour a great figure, especially one who is such a local hero.

Their infographic succinctly (and attractively) sums up many of the key points of Turing’s story and is a great point of access for those who are just learning about the man and his place in history.

It’s reproduced below,and  you can visit it on the Jury’s Inn site here.

Alan Turing of Manchester, by Jurys Inn Manchester Hotel
Alan Turing Infographic by Jurys Inn Hotels

Posted in Uncategorized | 6 Comments

Three Words to Sum Up Alan Turing

“A great science fiction detective story” – Ian Watson, author of The Universal Machine
“Cutting edge speculative fiction” – Ernest Hogan, author of Cortez on Jupiter
“Sharply erudite, with the vicious tang of cordite”– Paul Morris, author of Time Traveller Danny and the Codebreaker

L+D Soundtrack

Now Available: the Free Soundtrack for the Novel! Click the Banner to Stream or Download.

Days since the Centenary: 326
Days to the Bicentennial: 36,198

I’m not sure that Jack Copeland actually manages to sum up Alan Turing in three words.

Nonetheless, this brief video provides a nice portrait that helps rescue the real Alan Turing, the man you would meet face to face, from the that other Turing:
Turing the Icon
Turing the Guy on the Postage Stamp
Turing the Entry in Wikipedia.

It’s easily worth two minutes and forty-seven seconds of your time.

Copeland’s first three words: Humour, courage, isolation.
Copeland’s second three words: Patriotic, unconventional, genius.

It’s a compact Turing poem:

Humour, courage, isolation.
Patriotic, unconventional, genius.

Posted in Jack Copeland | Leave a comment

Hum Along to Turing, Part III (+ Free Bonus Album)

“A great science fiction detective story” – Ian Watson, author of The Universal Machine
“Cutting edge speculative fiction” – Ernest Hogan, author of Cortez on Jupiter
“Sharply erudite, with the vicious tang of cordite”– Paul Morris, author of Time Traveller Danny and the Codebreaker

L+D Soundtrack

Now Available: the Free Soundtrack for the Novel! Click the Banner to Stream or Download.

Days since the Centenary: 313
Days to the Bicentennial: 36,211

Holy cats! The melodic interpretations of the Turing legacy just keep on coming.

(To catch up on previous installments, see: Hum Along to Turing and Hum Along to Turing, Part Deux)

I’ve actually got another music-related post backlogged at the moment, but since this one is time-sensitive I thought I’d better post it now. (Plus, this is a good time to mention a relevant musical freebie, which I’ll get to in a moment.)

1. Album for Bletchley Park

First, we have a brand new album of electronic music called Music by Programmers, released to raise funds for the Bletchley Park Trust and the National Museum of Computing.

Bletchley Park, of course, was where Turing and his fellow codebreakers worked during World War II (see the About Alan Turing tab if you’re new to the subject).

Bletchley Park

Bletchley Park

According to TechWeek Europe:

The National Museum of Computing and Bletchley Park Trust have announced a novel fund-raising effort to attract young people to begin studying computer programming… The aim is to raise £5,000 to be spent on parent-child maths workshops at Bletchley Park and to allow [the museum] to start a regular computer club for young people. The download will be available from 29 April from CD Baby, iTunes, Amazon MP3 And Google Play with all of the profits going towards the projects.

The album was curated by Jason Gorman, who created Apes With Hobbies, Love Songs from a Gilded Age.  If you want to get an idea of Gorman’s musical sensibilities, you can download Apes for free here.

Gorman is clearly  predisposed to making music that honours great minds, given that Apes open with a track called Feynman Dreams of California Nights.

Music by Programmers and Jason Gorman

Music by Programmers and Jason Gorman

The eight tracks on the album are all original, and pay tribute to an earlier generation of electronic musicians, like Kraftwerk (still going strong), Jean-Michel Jarre (who recently released a new album), and Tangerine Dream (who have a new album out this month with awesome Queen-guitarist-cum-astrophysicst Brian May).  Gorman also recorded two of the eight tracks on the album himself.

I think this is an awesome way to raise money for a great cause. You can hear samples on the iTunes web page for the album. Buy it, rate it, and pass the word.

2. The Free Bonus Album

At the outset I mentioned a free bonus album related to Turing, so here’s the scoop.

Regular readers of this page will know that, while I post here for the love of it, professionally I’m a writer and editor.  I’m the non-fiction editor at International Speculative Fiction, which has published numerous award-winning authors, and I have several published works of my own.

My novel Luck and Death at the Edge of the World (Amazon.com, Amazon.co.uk, Kobo) features a sentient artificial intelligence instantiated in a synthetic human body. By its own choice it is called Alan and the body it uses is the image of Alan Turing. At the outset these choices seem trivial, but they turn out to go deeper than it first appears.

Luck + Death

Luck + Death

As with all my books, I try to give readers of Luck + Death a lot of free stuff to go along with the story. There are bonus sections right inside the book that provide background material (including a section on Turing), and the home page (www.LuckAndDeath.com) has more bonus material and a library of free PDFs.

Well, now the novel also has a free soundtrack that you can stream or download, Luck + Death: the Soundtrack for the Movie in Your Head.

Here’s the official announcement for anyone who wants to try it out:

After much work and planning, the free soundtrack album for Luck + Death is now available. Fourteen tracks by artists in eight countries (Belgium, Brazil, Canada, China, Hungary, Sweden, Turkey, and the U.S.A.).

This is a varied array of musicians and composers, from independent musicians like guitarist Jason Brock and saxophonist Stefan Thaens, to film composer Nathan Fleet, to veteran performer John Pazdan, to classical composer (and music professor) Russell Wilson.

This Version

Styles range from dreamy electronic to jumped up funk to electronic classical, all of it spiced up with spoken word performance and bookended by field recordings from the streets of Shanghai.

Here’s a sample, track number eight:

____________________
08 Distrito Federal (Stefan Thaens, Belgium) 4:18

____________________

All tracks are available for streaming or for download as MP3s and the album comes with the choice of two different versions of the cover art, This Version (above) and That Version (below).

That Version

And this is a project that appreciates music as more than just a spectator sport. One track is a mix by the author called El Paraíso Perdido (Paradise Lost), which has been posted on ccMixter.org with an invitation to anyone and everyone to remix it, deconstruct it, or reinvent it.

I’ve already received one version from a ccMixter member and I know another is on the way. All mixes are posted on the soundtrack home page.

Here’s one more track to whet your appetite:

____________________

12 Dogware (Zapac, Hungary) 3:40

____________________

Curiosity piqued?

Get it here: Luck + Death: the Soundtrack for the Movie in Your Head.

Posted in freebies, music | Leave a comment

Hum Along to Turing, Part Deux

“A Great science fiction detective story” – Ian Watson, author of The Universal Machine

Luck and Death Banner - click to go to Amazon.com

NOW AVAILABLE for instant download! Click to find out more.

Days since the Centenary: 255

Days to the Bicentennial: 36,269

In a recent post entitled Hum Along to Turing I reported that London-based band Fiction was releasing their debut album The Big Other, including a song called The Apple that was about, and dedicated to, our boy Alan Turing.

Somehow my search for a video at the time of that post failed to come up with one that was posted on YouTube waaaaay back in 2010, possibly because it’s not an official video but a live version of the song captured at the Offset Festival in 2010.

(Although, honestly, if you type in search terms like “Fiction” and “Apple” you’d think that a song by Fiction called Apple would come up, irrespective of whether it was a fan video or an official release.)

So, until an official version comes out, enjoy the performance below.

Posted in music | Tagged , , , , , | Leave a comment