Sometimes The Turing Test Is Not Enough

Days to Centenary: 247

The Turing Test has a strange, multifaceted life on the internet.

For instance the idea is used literally but prospectively, as a benchmark of some impending level of technological development that we are approaching.  In this aspect it’s usually applied to a type of machine that someone claims is approaching the point of passing the test (for the usual model of the Turing Test see here, and for the kind with killing machines see here).

It’s also used, however, as a metaphor (though not always with great precision or clarity of thought).   Journalists, bloggers, and anonymous trolls regularly claim that we need “a Turing Test” for all manner of things, including for editorsartists, “lefties,” (click and scroll down), and parrots (link requires forum membership, screenshot below) .

Forum commenter calls for Turing Test for parrots

Forum commenter calls for Turing Test for parrots (click to expand)

But it’s a Turing Test that is neither prospective nor metaphorical that is the subject of this post: the actual Turing Test as it already functions in day to day life.

This is the ubiquitous “CAPTCHA,” the annoying distorted image of a word that appears on your screen when you try to post a comment, sign up for free email, or otherwise conduct yourself online.  The name is an acronym for “Completely Automated Public Turing Test To Tell Computers and Humans Apart.”

The CAPTCHA is designed to distinguish genuine human users from the automated bots which are sometimes used to sign up for multiple free emails accounts, post spam in the comments sections of blogs, and so forth.  Ironically, a CAPTCHA is an automated process which — if it is working properly — creates a test that it could not pass itself.

A captcha

A captcha

According to the official CAPTCHA site, the term “CAPTCHA” was coined in 2000 by Luis von Ahn, Manuel Blum, Nicholas Hopper and John Langford of Carnegie Mellon University, who first developed the process for use by Yahoo.

The developers have summarized their work in an article called Telling Humans and Computers Apart, How Lazy Cryptographers Do AI and I recommend it as a good history of the idea — it is brief, informative, and not overly technical.

A pdf copy can be downloaded by clicking here: Telling Humans and Computers Apart.pdf (it will open in your browser and can then be saved if you wish).

Telling Humans and Computers Apart from the Communications of the ACM

Telling Humans and Computers Apart from the Communications of the ACM

The CAPTCHA’s utility can be judged by its ubiquity: you can barely sign up for anything on the internet these days without being required to prove that you’re human.  So whatever else can be said about the Turing Test, it is in practical application and has been integrated into today’s online environment in a way that would be difficult to do without by now.

That’s all very well and good, but is the CAPTCHA a real Turing Test? A brief digression into online laughs will help me illustrate the point I want to make.

Nothing can fail on the internet without becoming the subject of an article or page that exists solely to mock it, and the CAPTCHA is no exception.   The text fragments used by the CAPTCHA process are generated randomly, a process which inevitably produces the occasional inappropriate, offensive, or otherwise funny result, and these have been collected for our amusement and timewasting, as all internet fails are (for instance here).

Inappropriate CAPTCHA: Humping grandchildren

Inappropriate CAPTCHA: Humping grandchildren

But these fails highlight an underlying fact: CAPTCHAs may usefully allow us to distinguish humans from bots online, but the ones we have at this point only test character recognition. Indeed, apart from hacks that bypass the test altogether, improved character recognition is one of the main methods used to defeat CAPTCHAs (see this articlethis page, and this paper in pdf format, as well as the video below).

If a homo artificialis were competing in a real Turing Test, character recognition might be useful (depending on the user interface), but it would hardly be sufficient by itself for a pass.  Put another way, using character recognition alone to contruct a test is not enough to cause an artificial intelligence to fail that test.

Now, the theory of the CAPTCHA is broad enough to encompass a semantic element.  Inevitably such an element will be included into some new version of the CAPTCHA, bringing it in line with the “imitation game” actually proposed by Turing.

But even having a real, fully functional Turing Test may not be sufficient for some real world applications.  In addition to bypassing CAPTCHAs and improving character recognition, there’s a third method being used right now to defeat the CAPTCHA: using cheap human labour (or free human labour fueled by lust).  No Turing Test in the world is going to be proof against the human use of human beings — some other solution is needed for that.

Sidebar for Webmasters:

Webmasters should note that there is a free CAPTCHA service available called reCAPTCHA that not only provides web security for your site, but also assists in the digitization of books at the same time.

Details on how the two sides of the project relate are available here, but in essence the idea is that fragments of scanned text which cannot be correctly read by computers (and therefore cannot be integrated into a digital version of the scanned book) is used as the basis for CAPTCHA images, allowing human users to decipher the words that a computer can’t.

Update re Turing Test as Metaphor

A day after writing this post I noticed an article on in which Daniel Drezner critiques the foreign policy platform of GOP presidential hopeful Herman Cain.

Drezner says “I don’t think it passes the Turing Test” (link in the original).  He doesn’t explain this assessment anywhere and there’s nothing in his article to give it context except an overall assessment of Cain’s foreign policy as falling somewhere between being non-existent and being a loose agreggate of amateurish fumbling.

Is he saying that Cain isn’t human?  That he’s a computer?  I doubt he means either of those things.

Here “Turing Test” seems to morph into a generic standard one must meet to qualify as convincing as something, in this case convincing as a candidate for President of the United States of America.  Once it’s used this way, though, the term has little real meaning any more.

It’s an otherwise well written — and even amusing — analysis of a man taking an amateurish stab at the his party’s candidacy for the presidency and doing inexplicably well (at least as this is written — he may flounder yet).  I recommend it to anyone who’s interested, and especially to anyone in the U.S.A., but you might want to skip over the Turing reference.

As always, if you think I’ve missed something or that I’m wrong, you are invited to write me.  If you have an interesting point (whether or not I agree with it) then I’ll update this post.  To be fair to Mr. Drezner I’ll also drop him an email and see if he wants to say anything about it.

This entry was posted in CAPTCHA. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s