Georgia Tech professor proposes another alternative to the Turing test

The Lovelace 2.0 Test of Artificial Creativity and Intelligence assesses a computer's capacity for human-level intelligence by its ability to create, rather than to converse or deceive
November 20, 2014

But would mathematician-programmer Countess Ada Lovelace have approved?

Georgia Tech associate professor Mark Ried has developed a new kind of “Turing test” — a test proposed in 1950 by computing pioneer Alan Turing to determine whether a machine or computer program exhibits human-level intelligence.Most Turing test designs require a machine to engage in dialogue and convince (trick) a human judge that it is an actual person. But creating certain types of art also requires intelligence, leading Reid to consider if that approach might lead to a better gauge of whether a machine can replicate human thought.

“It’s important to note that Turing never meant for his test to be the official benchmark as to whether a machine or computer program can actually think like a human,” Riedl said.

“And yet it has, and it has proven to be a weak measure because it relies on deception. This proposal suggests that a better measure would be a test that asks an artificial agent to create an artifact requiring a wide range of human-level intelligent capabilities.”

The Lovelace 2.0 Test

To that end, Riedl has created the Lovelace 2.0 Test of Artificial Creativity and Intelligence.

Here are the basic test rules:

  • The artificial agent passes if it develops a creative artifact from a subset of artistic genres deemed to require human-level intelligence and the artifact meets certain creative constraints given by a human evaluator.
  • The human evaluator must determine that the object is a valid representative of the creative subset and that it meets the criteria. (The created artifact needs only meet these criteria — it does not need to have any aesthetic value.)
  • A human referee must determine that the combination of the subset and criteria is not an impossible standard.

The Lovelace 2.0 Test stems from the original Lovelace* Test as proposed by Bringsjord, Bello and Ferrucci in 2001. The original test required that an artificial agent produce a creative item in such a way that the agent’s designer cannot explain how it developed the creative item. The item, thus, must be created in such a way that is valuable, novel and surprising.

Riedl contends that the original Lovelace test does not establish clear or measurable parameters. Lovelace 2.0, however, enables the evaluator to work with defined constraints without making value judgments such as whether the artistic object created surprise.

Riedl’s paper, available here, will be presented at Beyond the Turing Test, an Association for the Advancement of Artificial Intelligence (AAAI) workshop to be held January 25–29, 2015, in Austin, Texas.

* In honor of Ada Lovelace, considered the world’s first computer programmer.