Kurzweil, artificial intelligence expert and author of The Age of Intelligent Machines, shows that technological evolution moves at an exponential pace.

Further, he asserts, in a sort of swirling postulate, time speeds up as order increases, and vice versa. He calls this the “Law of Time and Chaos,” and it means that although entropy is slowing the stream of time down for the universe overall, and thus vastly increasing the amount of time between major events, in the eddy of technological evolution the exact opposite is happening, and events will soon be coming faster and more furiously.

This means that we’d better figure out how to deal with conscious machines as soon as possible — they’ll soon not only be able to beat us at chess, but also likely demand civil rights, and might at last realize the very human dream of immortality.

Lavishly illustrated and easily accessible to the nonspecialist, The Age of Intelligent Machines provides the background needed for a full understanding of the enormous scientific potential represented by intelligent machines and of their equally profound philosophic, economic, and social implications. It examines the history of efforts to understand human intelligence and to emulate it by building devices that seem to act with human capabilities.

In a sweeping approach reflective of his intimate knowledge of the subject, Kurzweil systematically builds on the great landmarks of human intellect. He weaves together the singular achievements of such major thinkers as Plato, Euclid, Newton, Babbage, Einstein, von Neumann, and Wittgenstein to provide an orderly and comprehensive understanding of the impact intelligent machines will have on the world as it enters the third millenium.

Running alongside Kurzweil’s historical and scientific narrative, are 23 articles examining contemporary issues in artificial intelligence by such luminaries as Daniel Dennett, Sherry Turkle, Douglas Hofstadter, Marvin Minsky, Seymour Papert, Edward Feigenbaum, Allen Newell, and George Gilder.

Unique for both its style and substance, the book is today recognized as the seminal work in nanotechnology and has earned Drexler the title of “Father of Nanotechnology.”

Engines of Creation 2.0: The Coming Era of Nanotechnology — Updated and Expanded, is an e-book-only version available for free to readers exclusively through WOWIO.

In addition to an updated “look and feel” for the e-book, Engines of Creation 2.0 has been expanded to include the first known lecture on nanotechnology by physicist Richard Feynman, the landmark open letter debate between Dr. Drexler and the late nanotech pioneer and Nobel laureate Dr. Richard Smalley, analysis of the debate by Ray Kurzweil, and a number of new additions by Dr. Drexler, including his advice to aspiring nanotechnologists.

“Some seminal works stand out like beacons in the history of science. Newton’s ‘Philosophiae Naturalis Principia Mathematica’ and Watson and Crick’s ‘A Structure for Deoxyribose Nucleic Acid’ come quickly to mind. In recent decades we can add Eric Drexler’s Engines of Creation, which established the revolutionary new field of nanotechnology.

“In the twenty years since this seminal work was published, its premises and analyses have been confirmed and we are starting to apply precise molecular assembly to a wide variety of early applications from blood cell sized devices that can target cancer cells to a new generation of efficient solar panels. We can now see clearly the roadmap over the next couple of decades to the full realization of Drexler’s concept of the inexpensive assembly of macro objects constructed at the nanoscale controlled by massively parallel information processes, the fulfillment of which will enable us to solve problems — energy, environmental degradation, poverty, and disease to name a few — that have plagued humankind for eons.” — Ray Kurzweil, inventor and author of The Singularity is Near, When Humans Transcend Biology

This book is available from WOWIO Books | Click the image below to purchase.

In this debate with his critics, including several Discovery Institute fellows, Kurzweil defends his views and sets the stage for the central question: “What does it mean to be human?”

It has been said that when people make forecasts, they overestimate what can be done in the short run and underestimate what can be achieved in the long run. I have worked in the science and technology of artificial intelligence for twenty years and confess to being chronically optimistic about its progress. The gains have been substantial, even impressive. But we have hardly begun, and we must not lose sight of the point to which we are heading, however distant it may seem.

We are beginning the transition from data processing to knowledge processing. The key tool of our specialty is the digital computer, the most complex and yet the most general machine ever invented. Though the computer is a universal symbol-processing device, we have exploited to date only its mundane capabilities to file and retrieve data (file service) and to do high-speed arithmetic. Researchers in artificial intelligence have been studying techniques for computer representation of human knowledge and the methods by which that knowledge can be used to reason toward the solution of problems, the formation of hypotheses, and the discovery of new concepts and new knowledge. These researchers have been inventing the knowledge servers of our future.

Like all creators, scientists and technologists must dream, must put forth a vision, or else they relegate their work to almost pointless incrementalism. My dream is about the future of AI research and development over the next several decades and the knowledge systems that can be produced thereby to assist the modern knowledge worker.

The Beginnings of the Dream

Fifty years ago, before the modern era of computation began, Turing’s theorems and abstract machines gave hint of the fundamental idea that the computer could be used to model the symbol-manipulating processes that make up the most human of all behaviors: thinking. More than thirty years ago the work began in earnest (1991 will mark the thirty-fifth anniversary of the Dartmouth Summer Conference on Artificial Intelligence). The founding principle of AI research is really an article of faith that the digital computer has the necessary and sufficient means for intelligent action. This first principle is called the physical-symbol-system hypothesis.

The early dreaming included dreams about intelligent behavior at very high levels of competence. Turing speculated on wide-ranging conversations between people and machines and on chess playing programs. Later Newell and Simon wrote about champion-level chess programs and began their work toward that end. Samuel (checker playing), Gelernter (geometry-theorem proving), and others shared the dream.

At Stanford, Lederberg and I chose reasoning in science as our task and began work with Buchanan and Djerassi on building a program that would elucidate chemical structure at a high level of competence: the DENDRAL program. What emerged from the many experiments with DENDRAL was an empirical hypothesis that the source of the program’s power to figure out chemical structures from spectral data was its knowledge of basic and spectral chemistry. For DENDRAL, knowledge was power. Obvious? In retrospect, perhaps. But the prevailing view in Al at the time ascribed power to the reasoning processes-in modern terms, to the inference engine, not the knowledge base. Thus, in the late 1960s the knowledge-is-power hypothesis stood as a counter-hypothesis awaiting further tests and the accumulation of evidence.

Much evidence came in the 1970s. Medical problem solving provided the springboard. The MYCIN program of Shortliffe and others at Stanford was the prototype of the expert-level advisory (or consultation) system. The core of MYCIN was its knowledge base of rules for the diagnosis and therapy of infectious diseases. Its reasoning process was simple (backward chaining), even ad hoc in parts. But MYCIN was built as an integrated package of intellectual abilities. It could interact with a professional in the professional jargon of the specialty. It could explain its line of reasoning. And it had a subsystem that could aid in the acquisition of new knowledge by guiding an expert to find defects in the stored knowledge. Overall, MYCIN provided strong confirmation to the knowledge-is-power hypothesis.

At nearly the same time other efforts in medical problem solving were providing similar results. At the University of Pittsburgh the focus of the Internist project was the construction of an enormous electronic textbook of the knowledge of internal medicine. With its current knowledge base of 572 diseases, nearly 4,500 manifestations, and hundreds of thousands of links between them, Internist has provided the strongest confirmation yet of the knowledge-is-power hypothesis.

In the late 1970s an explosion of expert systems was taking place in fields other than medicine: engineering, manufacturing, geology, molecular biology, financial services, diagnostic servicing of machinery, military signal processing, and many other areas. There is little that ties these areas together other than this: in each, high-quality problem solving is guided by experiential, qualitative, heuristic knowledge. The explosion of applications created a new type of professional, the knowledge engineer (now in extremely short supply) and a new industry, the expert systems industry (now rapidly expanding). One generalization from the frenzy of activity is simply massive additional confirmation of the knowledge-is-power hypothesis. The reasoning procedures associated with all of these systems are weak. Their power lies in their knowledge bases.

Other areas of AI research made shifts to the knowledge base viewpoint. It is now commonplace to say, A program for understanding natural language must have extensive knowledge of its domain of discourse. A vision program for image understanding must have knowledge of the world it is intended to see. And even, learning programs must have a substantial body of knowledge from which to expand (that is, learning takes place at the fringes and interstices of what is already known. Thus, the dream of a computer that performs at a high level of competence over a wide variety of tasks that people perform well seems to rest upon knowledge in the task areas.

The knowledge-is-power hypothesis has received so much confirmation that we can now assert it as the knowledge principle:

A system exhibits intelligent understanding and action at a high level of competence primarily because of the specific knowledge that it contains about its domain of endeavor.

A corollary to the knowledge principle is that reasoning processes of an intelligent system, being general and therefore weak, are not the source of power that leads to high levels of competence in behavior. The knowledge principle simply says that if a program is to perform well, it must know a great deal about the world in which it operates. In the absence of knowledge, reasoning won’t help.

The knowledge principle is the emblem of the first era of artificial intelligence; it is the first part of the dream. It should inform and influence every decision about what it is feasible to do in AI science and with AI technology.

The Middle of the Dream

Today our intelligent artifacts perform well on specialized tasks within narrowly defined domains. An industry has been formed to put this technological understanding to work, and widespread transfer of this technology has been achieved. Although the first era of the intelligent machine is ending, many problems remain to be solved.

One of these is naturalness. The intelligent agent should interact with its human user in a fluid and flexible manner that appears natural to the person. But the systems of the first era share with the majority of computer systems an intolerable rigidity of stylistic expression, vocabulary, and concepts. For example, programs rarely accept synonyms, and they cannot interpret and use metaphors. They always interact in a rigid grammatical straitjacket. The need for metaphor to induce in the user a feeling of naturalness seems critical. Metaphorical reference appears to be omnipresent and almost continuous in our use of language. Further, if you believe that our use of language reflects our underlying cognitive processes, then metaphor is a basic ideational process.

In the second era we shall see the evolution of the natural interface. The processes controlling the interaction will make greater use of the domain knowledge of the system and knowledge of how to conduct fluid discourse. Harbingers of naturalness already exist; they are based to a large extent upon pictures. The ONCOCIN project team at Stanford invested a great effort in an electronic flow sheet to provide a seamless transition for the oncologist from paper forms for patient data entry to electronic versions of these forms. The commercially available software tools for expert-system development sometimes contain elegant and powerful packages for creating pictures that elucidate what the knowledge system is doing and what its emerging solution looks like (for example, IntelliCorp’s KEE Pictures and Active Images).

Naturalness need not rely upon pictures, of course. The advances in natural-language understanding have been quite substantial, particularly in the use of knowledge to facilitate understanding. In the second era it will become commonplace for knowledge systems to interact with users in human language, within the scope of the system’s knowledge. The interaction systems of the second era will increasingly rely on continuous natural speech. In person-to-person interactions, people generally talk rather than type. Typing is useful but unnatural. Speech-understanding systems of wide applicability and based on the knowledge principle are coming. At Stanford we are beginning experiments with an experimental commercial system interfaced with the ONCOCIN expert system.

A limitation of first-era systems is their brittleness. To mix metaphors, they operate on a high plateau of knowledge and competence until they reach the extremity of their knowledge; then they precipitously fall off to levels of utter incompetence. People suffer from the same difficulty (they too cannot escape the knowledge principle, but their fall is more graceful. The cushion for the soft fall is the knowledge and use of weaker but more general models that underlie the highly specific and specialized knowledge of the plateau. For example, if an engineer is diagnosing the failure of an electronic circuit for which he has no specific knowledge, he can fall back on his knowledge of electronics, methods of circuit analysis, and handbook data for the components. The capability for such model-based reasoning by machine is just now under study in many laboratories and will emerge as an important feature of second-era systems. The capability does not come free. Knowledge engineers must explicate and codify general models in a wide variety of task areas.

Task areas? But what if there is no “task”? Can we envision the intelligent program that behaves with common sense at the interstices between tasks or when task knowledge is completely lacking? Common sense is itself knowledge, an enormous body of knowledge distinguished by its ubiquity and the circumstance that it is rarely codified and passed onto others, as more formal knowledge is. There is, for example, the commonsense fact that pregnancy is associated with females, not males. The extremely weak but extremely general forms of cognitive behavior implied by commonsense reasoning constitute for many the ultimate goal in the quest for machine intelligence. Researchers are now beginning the arduous task of understanding the details of the logic and representation of commonsense knowledge and are codifying large bodies of commonsense knowledge. The first fruits of this will appear in the later systems of the second era. Commonsense reasoning will probably appear as an unexpected naturalness in a machine’s interaction with an intelligent agent. As an example of this in medical-consultation advisory systems, if pregnancy is mentioned early in the interaction or can be readily inferred, the interaction shifts seamlessly to understanding that a female is involved. Magnify this example by one hundred thousand or one million unspoken assumptions, and you will understand what I mean by a large knowledge base of commonsense knowledge.

As knowledge in systems expands, so does the scope for modes of reasoning that have so far eluded the designers of these systems. Foremost among these modes are reasoning by analogy and its sibling metaphorical reasoning. The essence of analogy has been evident for some time, but the details of analogizing have not been. An analogy is a partial match of the description of some current situation with stored knowledge. The extent of the match is crucial. If the match is too partial, then the analogy is seen to be vacuous or farfetched; if too complete then the “analogy” is seen as hardly an analogy at all.

Analogizing broadens the relevance of the entire knowledge base. It can be used to construct interesting and novel interpretations of situations and data. It can be used to retrieve knowledge that has been stored, but not stored in the “expected” way. Analogizing can supply default values for attributes not evident in the description of the current situation. Analogizing can provide access to powerful methods that otherwise would not be evoked as relevant. For example, in a famous example from early twentieth century physics, Dirac made the analogy between quantum theory and mathematical group theory that allowed him to use the powerful methods of group theory to solve important problems in quantum physics. We shall begin to see reasoning by analogy emerge in knowledge systems of the second era.

Analogizing is seen also as an important process in automatic knowledge acquisition, another name for machine learning. In first-era systems, adding knowledge to knowledge bases has been almost always a manual process: people codify knowledge and place it in knowledge structures. Experiments by Douglas Lenat have shown that this laborious process can be semi-automated, facilitated by an analogizing program. The program suggests the relevant analogy to a new situation, and the knowledge engineer fills in the details. In the second era we shall see programs that acquire the details with less or no human help. Many other techniques for automatic learning will find their way into second-era systems. For example, we are currently seeing early experiments on learning apprentices, machines that carefully observe people performing complex tasks and infer thereby the knowledge needed for competent performance. The second era will also see (I predict) the first successful systems that couple language understanding with learning, so that knowledge bases can be augmented by the reading of text. Quite likely these will be specialized texts in narrow areas at the outset.

To summarize, because of the increasing power of our concepts and tools and the advent of automatic-learning methods, we can expect that during the second era the knowledge bases of intelligent systems will become very large, representing therein hundreds of thousands, perhaps millions, of facts, heuristics, concepts, relationships, and models. Automatic learning will be facilitated thereby, since by the knowledge principle, the task of adding knowledge is performed more competently the more knowledge is available (the more we know, the easier it is to know more).

Finally, in the second era we will achieve a broad reconceptualization of what we mean by a knowledge system. Under the broader concept, the “systems” will be collegial relationships between an intelligent computer agent and an intelligent person (or persons). Each will perform tasks that he/she/it does best, and the intelligence of the system will be an emergent of the collaboration. If the interaction is indeed seamless and natural, then it may hardly matter whether the relevant knowledge or the reasoning skills needed are in the head of the person or in the knowledge structures of the computer.

The Far Side of the Dream: The Library of the Future

Here’s a “view from the future,” looking back at our “present,” from Professor Marvin Minsky of MIT: “Can you imagine that they used to have libraries where the books didn’t talk to each other?” The libraries of today are warehouses for passive objects. The books and journals sit on shelves waiting for us to use our intelligence to find them, read them, interpret them, and cause them finally to divulge their stored knowledge. Electronic libraries of today are no better. Their pages are pages of data files, but the electronic pages are equally passive.

Now imagine the library as an active, intelligent knowledge server. It stores the knowledge of the disciplines in complex knowledge structures (perhaps in a knowledge-representation formalism yet to be invented). It can reason with this knowledge to satisfy the needs of its users. These needs are expressed naturally, with fluid discourse. The system can, of course, retrieve and exhibit (i.e., it can act as an electronic textbook). It can collect relevant information; it can summarize; it can pursue relationships. It acts as a consultant on specific problems, offering advice on particular solutions, justifying those solutions with citations or with a fabric of general reasoning. If the user can suggest a solution or a hypothesis, it can check this and even suggest extensions. Or it can critique the user viewpoint with a detailed rationale of its agreement or disagreement. It pursues relational paths of associations to suggest to the user previously unseen connections. Collaborating with the user, it uses its processes of association and analogizing to brainstorm for remote or novel concepts. More autonomously, but with some guidance from the user, it uses criteria of being interesting to discover new concepts, methods, theories, and measurements.

The user of the library of the future need not be a person. It may be another knowledge system, that is, any intelligent agent with a need for knowledge. Thus, the library of the future will be a network of knowledge systems in which people and machines collaborate. Publishing will be an activity transformed. Authors may bypass text, adding their increment to human knowledge directly to the knowledge structures. Since the thread of responsibility must be maintained, and since there may be disagreement as knowledge grows, the contributions are authored (incidentally allowing for the computation of royalties for access and use). Maintaining the knowledge base (updating knowledge) becomes a vigorous part of the new publishing industry.

Photo by Lou Jones www.fotojones.com

Edward Feigenbaum.

What is intelligence, anyway? It is only a word that people use to name those unknown processes with which our brains solve problems we call hard. But whenever you learn a skill yourself, you’re less impressed or mystified when other people do the same. This is why the meaning of “intelligence” seems so elusive: it describes not some definite thing but only the momentary horizon of our ignorance about how minds might work. It is hard for scientists who try to understand intelligence to explain precisely what they do, since our working definitions change from year to year. But it is not at all unusual for sciences to aim at moving targets. Biology explores the moving frontier of what we understand of what happens inside our bodies. Only a few decades ago the ability of organisms to reproduce seemed to be a deep and complex mystery. Yet as soon as they understood the elements of how our gene strings replicate themselves, biologists wondered why it took so long to think of such a simple thing. In the same way each era of psychology explores what we don’t then know about processes in our brains.

Then, can we someday build intelligent machines? I take the answer to be yes in principle, because our brains themselves are machines. To be sure, we still know very little about how brains actually work. There is no reason for scientists to be ashamed of this, considering that it was only a century ago that we began to suspect that brains were made of separate nerve cells that acted somewhat like computer parts and that it is only half a century since we began developing technical ideas for understanding what such systems could do. These ideas are still barely adequate for dealing with present-day serial computers, which have only thousands of active components, and are not yet robust enough to deal with systems like those in the brain, which involve trillions of interconnected parts, all working simultaneously.

Nor do we yet know how to make machines do many of the things that ordinary people do. Some critics maintain that machines will never be able to do same of those things, and some skeptics even claim to have proved such things. None of those purported proofs actually hold up to close examination, because we are still in the dark ages of scientific knowledge about such matters. In any case, we have not the slightest grounds for believing that human brains are not machines. Because of this, both psychology and artificial intelligence have similar goals: both seek to learn how machines could do many things we can’t yet make them do.

Why are so many people annoyed at the thought that human brains are nothing more than “mere machines”? It seems to me that we have a problem with the word “machine” because we’ve grown up to believe that machines can behave only in lifeless, mechanical ways. This view is obsolete, because the ways we use the word “machine” are out of date. For centuries words like “machine” and “mechanical” were used for describing relatively simple devices like pulleys, levers, locomotives, and typewriters. The word “computer” too inherits from the past that sense of pettiness that comes from doing dull arithmetic by many small and boring steps. Because of this, our previous experience can sometimes be a handicap. Our preconceptions of what machines can do date from what happened when we assembled systems from only a few hundreds or thousands of parts. And that did not prepare us to think about brainlike assemblies of billions of parts. Although we are already building machines with many millions of parts, we continue to think as though nothing has changed. We must learn to change how we think about phenomena that work on those larger scales.

What Is Artificial Intelligence?

Even though we don’t yet understand how brains perform many mental skills, we can still work toward making machines that do the same or similar things. “Artificial intelligence” is simply the name we give to that research. But as I already pointed out, this means that the focus of that research will keep changing, since as soon as we think we understand one mystery, we have to move on to the next. In fact, AI research has made enormous progress in only a few decades, and because of that rapidity, the field has acquired a somewhat shady reputation! This paradox resulted from the fact that whenever an AI research project made a useful new discovery, that product usually quickly spun off to form a new scientific or commercial specialty with its own distinctive name. These changes in name led outsiders to ask, Why do we see so little progress in the central field of artificial intelligence? Here are a few specialties that originated at least in part from AI research but later split into separate fields and, in some instances, commercial enterprises: robotics, pattern recognition, expert systems, automatic theorem proving, cognitive psychology, word processing, machine vision, knowledge engineering, symbolic applied mathematics, and computational linguistics.

For example, many researchers in the 1950s worked toward discovering ways to make machines recognize various sorts of patterns. As their findings were applied to problems involved with vision, speech, and several other areas, those fields evolved their own more distinct techniques, they organized their own technical societies and journals, and they stopped using the term “artificial intelligence.” Similarly, an early concern of AI was to develop techniques for enabling computers to understand human language; this spawned a field called computational linguistics. Again, many ideas from artificial intelligence had a large influence among psychologists, who applied those ideas to their studies of the mind but used the title “cognitive psychology.”

I can illustrate how AI projects develop by recounting the research of James Slagle, who, as a graduate student at MIT in 1960, developed a program to solve calculus problems; he named it with the initials of “symbolic automatic integration.” Although there were many problems that SAINT couldn’t solve, it surpassed the performance of average MIT students. When he first approached this subject, most scientists considered solving those problems to require substantial intelligence. But after Slagle’s work we had to ask ourselves instead why students take so long to learn to do such basically straightforward things.

How did SAINT solve those problems? It employed about 100 formulas from the domains of algebra and calculus and applied to these about a dozen pattern-matching methods for deciding which formula might be most likely to help solve a given problem. Since any particular attempt might fail, the program had to employ a good deal of trial and error. If one method did not work, the program automatically went on to try another. Sometimes one of them would work, but frequently a problem was too hard for any single such method to work. The system was programmed in that case to proceed on to certain other methods, methods that attempted to split each hard problem into several simpler ones. In this way, if no particular method worked, SAINT was equipped with a great variety of alternatives.

Now we can make an important point. For years the public has been told, Computers do only what they’re programmed to do. But now you can see why that’s not quite true: We can write programs that cause the machine to search for solutions. Often such searches produce results that greatly surprise their programmers.

The idea of making programs search greatly expanded their powers. But it also led to new kinds of problems: search processes could generate so many possible alternatives that the programs were in constant danger of getting lost, repeating themselves, or persisting at fruitless attempts that had already consumed large amounts of time. Much research in the 1960s was focused on finding methods to reduce that sort of fruitless search. Slagle himself experimented with some mathematical theories of how to take into account both how much effort had been spent on each solution attempt and how much apparent progress had been made. Thus the SAINT program worked as well as it did, not merely because of its specialized knowledge about calculus, but also because of other knowledge about the search itself. To prevent the search from simply floundering around, making one random attempt after another, some of the program’s knowledge was applied to recognize conditions in which its other, more specialized knowledge might be particularly useful.

When SAINT first appeared, it was acclaimed an outstanding example of work in the field of artificial intelligence. Later other workers analyzed more carefully its virtues and deficiencies, and this research improved our understanding of the basic nature of those calculus problems. Eventually ways were found to replace all the trial and error processes in SAINT by methods that worked without any search. The resulting commercial product, a program called MACSYMA, actually surpassed the abilities of professional mathematicians in this area. But once the subject was so well understood, we ceased to think of it as needing intelligence. This area is now generally seen as belonging no longer to artificial intelligence but to a separate specialty called symbolic applied mathematics.

Robotics and Common Sense

In the 1960s we first began to equip computers with mechanical hands and television eyes. Our goal was to endow machines with the sorts of abilities children use when playing with toys and building blocks. We found this much harder to do than expected. Indeed, a scholar of the history of artificial intelligence might get a sense of watching evolution in reverse. Even in its earliest years we saw computers playing chess and doing calculus, but it took another decade for us to begin to learn to make machines that could begin to act like children playing with building blocks! What makes it easier to design programs that imitate experts than to make them simulate novices? The amazing answer is, Experts are simpler than novices! To see why it was harder to make programs play with toys than pass calculus exams, let’s consider what’s involved in enabling a robot to copy simple structures composed of blocks: we had to provide our robot with hundreds of small programs organized into a system that engaged many different domains of knowledge. Here are a few of the sorts of problems this system had to deal with:

• The relation between the hand and the eye
• Recognizing objects from their visual appearances
• Recognizing objects partially hidden from view
• Recognizing relations between different objects
• Fitting together three-dimensional shapes
• Understanding how objects can support one another to form stable structures
• Planning a sequence of actions to assemble a structure
• Moving in space so as to avoid collisions
• Controlling the fingers of a hand for grasping an object

It is very hard for any adult to remember or appreciate how complex are the properties of ordinary physical things. Once when an early version of our block-building program was asked to find a new place to put a block, it tried to place it on top of itself! The program could not anticipate how that action would change the situation. To catalog only enough fragments of knowledge to enable a robot to build a simple blocklike house from an unspecified variety of available materials would be an encyclopedic task. College students usually learn calculus in half a year, but it takes ten times longer for children to master their building toys. We all forget how hard it was to learn such things when we were young.

Expertise and Common Sense

Many computer programs already exist that do things most people would regard as requiring intelligence. But none of those programs can work outside of some very small domain or specialty. We have separate programs for playing chess, designing transformers, proving geometry theorems, and diagnosing kidney diseases. But none of those programs can do any of the things the others do. By itself each lacks the liveliness and versatility that any normal person has. And no one yet knows how to put many such programs together so that they can usefully communicate with one another. In my book, The Society of Mind, I outline some ideas on how that might be done inside our brains.

Putting together different ideas is just what children learn to do: we usually call this common sense. Few youngsters can design transformers or diagnose renal ailments, but whenever those children speak or play, they combine a thousand different skills. Why is it so much easier for AI programmers to simulate adult, expert skills than to make programs perform childlike sorts of commonsense thought? I suspect that part of the answer lies in the amounts of variety. We can often simulate much of what a specializt does by assembling a collection of special methods, all of which share the same common character. Then so long as we remain within some small and tidy problem world, that specializt’s domain of expertise, we need merely apply different combinations of basically similar rules. This high degree of uniformity makes it easy to design a higher-level supervisory program to decide which method to apply. However, although the “methods” of everyday thinking may, by themselves, seem simpler than those of experts, our collections of commonsense methods deal with a great many more different types of problems and situations. Consider how many different things each normal child must learn about the simplest-seeming physical objects, such as the peculiarities of blocks that are heavy, big, smooth, dangerous, pretty, delicate, or belong to someone else. Then consider that the child must learn quite different kinds of strategies for handling solids and liquids; strings, tapes, and cloths; jellies and muds as well as things he is told are prohibited, poisonous, or likely to cut or bite.

What are the consequences of the fact that the domain of commonsense thinking is so immensely varied and disorderly? One problem is simply accumulating so much knowledge. But AI research also encountered a second, more subtle problem. We had to face the simple fact that in order for a machine to behave as though it “knows” anything, there must exist, inside that machine, some sort of structure to embody or “represent” that knowledge. Now, a specialized, or “expert,” system can usually get by with very few types of what we call knowledge representations. But in order to span that larger universe of situations we meet in ordinary life, we appear to need a much larger variety of types of representations. This leads to a second, harder type of problem: knowledge represented in different ways must be applied in different ways. This imposes on each child obligations of a higher type: they have to learn which types of knowledge to apply to which kinds of situations and how to apply them. In other words, we have to accumulate not merely knowledge, but also a good deal of knowledge about knowledge. Now, experts too have to do that, but because commonsense knowledge is of more varied types, an ordinary person has to learn (albeit quite unconsciously) much more knowledge about representations of knowledge, that is, which types of representation skills to use for different purposes and how to use them.

If this sounds very complicated, it is because it actually is. Until the last half century we had only simple theories of mind, and these explained only a little of what animals could do in the impoverished worlds of laboratory experiments. Not until the 1930s did psychologists like Jean Piaget discover how many aspects of a child’s mind develop through complicated processes, sometimes composed of intricate sequences of stagelike periods. We still don’t know very much about such matters, except that the mind is much more complex than imagined in older philosophies. In The Society of Mind, I portray it as a sort of tangled-up bureaucracy, composed of many different experts, or as I call them, “agencies,” that each develop different ways to represent what they learn. But how can experts using different languages communicate with one another? The solution proposed in my book is simply that they never come to do it very well! And that explains why human consciousness seems so mysterious. Each part of the mind receives only hints of what the other parts are about, and no matter how hard a mind may try, it can never make very much sense of itself.

Supercomputers and Nanotechnology

Many problems we regard as needing cleverness can sometimes be solved by resorting to exhaustive searches, that is, by using massive, raw computer power. This is what happens in most of those inexpensive pocket chess computers. These little machines use programs much like the ones that we developed in the 1960s, using what were then some of the largest research computers in the world. Those old programs worked by examining the consequences of tens of thousands of possible moves before choosing one to actually make. But in those days the programs took so long to make those moves that the concepts they used were discarded as inadequate. Today, however, we can run the same programs on faster computers so that they can consider millions of possible moves, and now they play much better chess. However, that shouldn’t fool us into thinking that we now understand the basic problem any better. There is good reason to believe that outstanding human chess players actually examine merely dozens, rather than millions, of possible moves, subjecting each to more thoughtful analysis.

In any case, as computers improved in speed and memory size, quite a few programming methods became practical, ones that had actually been discarded in the earlier years of AI research. An Apple desktop computer (or an Amiga, Atari, IBM, or whatever) can do more than could a typical million-dollar machine of a decade earlier, yet private citizens can afford to play games with them. In 1960 a million-bit memory cost a million dollars; today a memory of the same size (and working a hundred times faster) can be purchased for the price of a good dinner. Some seers predict another hundredfold decrease in size and cast, perhaps in less than a decade, when we learn how to make each microcircuit ten times smaller in linear size and thus a hundred times smaller in area. What will happen after that? No one knows, but we can be sure of one thing: those two-dimensional chips we use today make very inefficient use of space. Once we start to build three-dimensional microstructures, we might gain another millionfald in density. To be sure, that would involve serious new problems with power, insulation, and heat. For a futuristic but sensible discussion of such possibilities, I recommend Eric Drexler’s Engines of Creation (Falcon Press, 1986).

Not only have small components become cheaper; they have also become faster. In 1960 a typical component required a microsecond to function; today our circuits operate a thousand times faster. Few optimists, however, predict another thousandfold increase in speed over the next generation. Does this mean that even with decreasing costs we will soon encounter limits on what we can make computers do? The answer is no, because we are just beginning a new era of parallel computers.

Most computers today are still serial; that is, they do only one thing at a time. Typically, a serial computer has millions of memory elements, but only a few of them operate at any moment, while the rest of them wait for their turn: in each cycle of operation, a serial computer can retrieve and use only one of the items in its memory banks. Wouldn’t it be better to keep more of the hardware in actual operation? A more active type of computer architecture was proposed in Daniel Hillis’s Connection Machine (MIT Press, 1986), which describes a way to assemble a large machine from a large number of very small, serial computers that operate concurrently and pass messages among themselves. Only a few years after being conceived, Connection Machines are already commercially available, and they indeed appear to have fulfilled their promise to break through some of the speed limitations of serial computers. In certain respects they are now the fastest computers in the world.

This is not to say that parallel computers do not have their own limitations. For, just as one cannot start building a house before the boards and bricks have arrived, you cannot always start work simultaneously on all aspects of solving a problem. T would certainly be nice if we could take any program for a serial computer, divide it into a million parts, and then get the answer a million times faster by running those parts simultaneously on that many computers in parallel. But that can’t be done, in general, particularly when certain parts of the solution depend upon the solutions to other parts. Nevertheless, this quite often turns out to be feasible in actual practice. And although this is only a guess, I suspect that it will happen surprisingly often for the purposes of artificial intelligence. Why do I think so? Simply because it seems very clear that our brains themselves must work that way.

Consider that brain cells work at very modest speeds in comparison to the speeds of computer parts. They work at rates of less than a thousand operations per second, a million times slower than what happens inside a modern computer circuit chip. Could any computer with such slow parts do all the things that a person can do? The answer must lie in parallel computation: different parts of the brain must do many more different things at the same time. True, that would take at least a billion nerve cells working in parallel, but the brain has many times that number of cells.

AI and the World of the Future

Intelligent machines may be within the technological reach of the next century. Over the next few generations we’ll have to face the problems they pose. Unless some unforeseen obstacles appear, our mind-engineering skills could grow to the point of enabling us to construct accomplished artificial scientists, artists, composers, and personal companions. Is AI merely another advance in technology, or is it a turning point in human evolution that should be a focus of discussion and planning by all mankind? The prospect of intelligent machines is one that we’re ill prepared to think about, because it raises such unusual moral, social, artistic, philosophical, and religious issues. Are we obliged to treat artificial intelligences as sentient beings? Should they have rights? And what should we do when there remains no real need for honest work, when artificial workers can do everything from mining, fanning, medicine, and manufacturing all the way to house cleaning? Must our lives then drift into pointless restlessness and all our social schemes disintegrate?

These questions have been discussed most thoughtfully in the literary works of such writers as Isaac Asimov, Gregory Benford, Arthur C. Clarke, Frederick Pohl, and Jack Williamson, who all tried to imagine how such presences might change the aspirations of humanity. Some optimistic futurists maintain that once we’ve satisfied all our worldly needs, we might then turn to the worlds of the mind. But consider how that enterprise itself would be affected by the presence of those artificial mindlike entities. That same AI technology would offer ways to modify the hardware of our brains and thus to endlessly extend the mental worlds we could explore.

You might ask why this essay mixes both computers and psychology. The reason is that though we’d like to talk about making intelligent machines, people are the only such intelligence we can imitate or study now. One trouble, though, is that we still don’t know enough about how people work! Does this mean that we can’t develop smart machines before we get some better theories of psychology? Not necessarily. There certainly could be ways to make very smart machines based on principles that our brains do not use, as in the case of those very fast, dumb chess machines. But since we’re the first very smart machines to have evolved, we just might represent one of the simplest ways!

But, you might object, there’s more to a human mind than merely intellect. What about emotion, intuition, courage, inspiration, creativity, and so forth. Surely it would be easier simply to understand intelligence than to try to analyze all those other aspects of our personalities! Not so, I maintain, because traditional distinctions like those between logic and intuition, between intellect and emotion, unwisely try to separate knowledge and meaning from purpose and intention. In The Society of Mind, I argue that little can be done without combining elements of both. Furthermore, when we put them together, it becomes easier, rather than harder, to understand such matters, because, though there are many kinds of questions, the answers to each of them illuminate the rest.

Photo by Lou Jones www.fotojones.com

Marvin Minsky.

Who cares? If a machine can do its job extremely well, what does it matter if it really thinks? No one runs around asking if taxicabs really walk.

How could you ever tell? This attitude is the basis of the famous Turing test, devised in 1950 by the British mathematician and logician Alan Turing: Imagine that you’re sitting alone in a room with a teletype machine that is connected at the other end to either a person or a computer. If no amount of questioning or conversation allows you to tell which it is, then you have to concede that a machine can think.

No, thinking is too complicated. Even if we someday come to understand all the laws and principles that govern the mind, that doesn’t mean that we can duplicate it. Does understanding astrophysics mean that we can build a galaxy?

Yes, machines can think in principle, but not necessarily in the same way we do. AI researcher Seymour Papert of the Massachusetts Institute of Technology maintains that artificial intelligence is analogous to artificial flight: ‘This leads us to imagine skeptics who would say, ‘You mathematicians deal with idealized fluids–the real atmosphere is vastly more complicated: or ‘You have no reason to suppose that airplanes and birds work the same way-birds have no propellers, airplanes have no feathers.’ But the premises of these criticisms is true only in the most superficial sense: the same principles (for example, Bernoulli’s law) applies to real as well as ideal fluids, and they apply whether the fluid flows over a feather or an aluminum wing.”

No! This is the most often heard answer, and the most heartfelt. “I am not a machine [goes the argument]. I’m me. I’m alive. And you’re never going to make a computer that can say that. Furthermore, the essence of humanity isn’t reason or logic or any of the other things that computers can do: it’s intuition, sensuality, and emotion. So how can a computer think if it does not feel, and how can it feel if it knows nothing of love, anguish, exhilaration, loneliness, and all the rest of what it means to be a living human being?”

“Sometimes when my children were still little,” writes former AI researcher Joseph Weizenbaum of MIT, “my wife and I would stand over them as they lay sleeping in their beds. We spoke to each other only in silence, rehearsing a scene as old as mankind itself. It is as Ionesco told his journal: ‘Not everything is unsayable in words, only the living truth: “

Can a Machine Be Aware?

As this last answer suggests, the case against machine intelligence always comes down to the ultimate mystery, which goes by many names: consciousness, awareness, spirit, soul. We don’t even understand what it is in humans. Many people would say that it is beyond our understanding entirely, that it is a subject best left to God alone. Other people simply wonder if a brain can ever understand itself, even in principle. But either way, how can we ever hope to reproduce it, whatever it is, with a pile of silicon and software?

That question has been the source of endless debate since the rise of AI, a debate made all the hotter by the fact that people aren’t arguing science. They’re arguing philosophical ideology-their personal beliefs about what the true theory of the mind will be like when we find it.

Not surprisingly, the philosophical landscape is rugged and diverse. But it’s possible to get some feel for the overall topography 6y looking at two extremes. At one extreme at the heart of classical AI we find the doctrines first set down in the 1950s by AI pioneers Allen Newell and Herbert Simon at Carnegie-Mellon University: (1) thinking is information processing; (2) information processing is computation, which is the manipulation of symbols; and (3) symbols, because of their relationships and linkages, mean something about the external world. In other words, the brain per se doesn’t matter, and Turing was right: a perfect simulation of thinking is thinking.

Tufts University philosopher Daniel C. Dennett, a witty and insightful observer of AI, has dubbed this position High Church Computationalism. Its prelates include such establishment figures as Simon and MIT’s Marvin Minsky; its Vatican City is MIT, “the East Pole.”

Then from out of the West comes heresy–a creed that is not an alternative so much as a denial. As Dennett describes it, the assertion is that “thinking is something going on in the brain all right, but it is not computation at all: thinking is something holistic and emergent-and organic and fuzzy and warn and cuddly and mysterious.”

Dennett calls this creed Zen holism. And for some reason its proponents do seem to cluster in the San Francisco Bay area. Among them are the gurus of the movement: Berkeley philosophers John Searle and Hubert Dreyfus.

The computationalists and the holists have been going at it for years, ever since Dreyfus first denounced AI in the mid 1960s with his caustic book What Computers Can’t Do. But their definitive battle came in 1980, in the pages of the journal Behavioral and Brain Sciences. This journal is unique among scientific journals in that it doesn’t just publish an article; first it solicits commentary from the author’s peers and gives the author a chance to write a rebuttal. Then it publishes the whole thing as a package-a kind of formal debate in print. In this case the centerpiece was Searle’s article “Minds, Brains, and Programs,” a stinging attack on the idea that a machine could think. Following it were 27 responses, most of which were stinging attacks on Searle. The whole thing is worth reading for its entertainment value alone. But it also highlights the fundamental issues with a clarity that has never been surpassed.

The Chinese Room

Essentially, Searle’s point was that simulation is not duplication. A program that uses formal rules to manipulate abstract symbols can never think or be aware, because those symbols don’t mean anything to the computer.

To illustrate, he proposed the following thought experiment as a parody of the typical AI language-understanding program of his day: “Suppose that I’m locked in a room and given a large batch of Chinese writing,” he said. “Suppose furthermore (as is indeed the case) that I know no Chinese …. To me, the Chinese writing is just so many meaningless squiggles.” Next, said Searle, he is given a second batch of Chinese writing (a “story”), together with some rules in English that explain how to correlate the first batch with the second (a “program”). Then after this is all done, he is given yet a third set of Chinese symbols (“questions”), together with yet more English rules that tell him how to manipulate the slips of paper until all three batches are correlated, and how to produce a new set of Chinese characters (“answers”), which he then passes back out of the room. Finally, said Searle, “after awhile I get so good at manipulating the instructions for the Chinese symbols and the programmers get so good at writing the programs that from the external point of view . . . my answers to the questions are absolutely indistinguishable from those of native Chinese speakers.” In other words, Searle learns to pass the Turing test in Chinese.

Now, according to the zealots of strong AI, said Searle, a computer that can answer questions in this way isn’t just simulating human language abilities. It is literally understanding the story. Moreover, the operation of the program is in fact an explanation of human understanding.

And yet, said Searle, while he is locked in that imaginary room he is doing exactly what the computer does. He uses formal rules to manipulate abstract symbols. He takes in stories and gives out answers exactly as a native Chinese would. But he still doesn’t understand a word of Chinese. So how is it possible to say that the computer understands? In fact, said Searle, it doesn’t. For comparison, imagine that the questions and the answers now switch to English. So far as the people outside the room are concerned, the system is just as fluent as before. And yet there’s all the difference in the world, because now he isn’t just manipulating formal symbols anymore. He understands what’s being said. The words have meaning for him-or, in the technical jargon of philosophy, he has intentionality. Why? “Because I am a certain sort of organism with a certain biological (i.e., chemical and physical/ structure,” he said, “and this structure, under certain conditions, is causally capable of producing perception, action, understanding, learning, and other intentional phenomena.” In other words, Searle concluded that it is certainly possible for a machine to think-”in an important sense our bodies with our brains are precisely such machines”-but only if the machine is as complex and as powerful as the brain. A purely formal computer program cannot do it.

Counterarguments

Searle’s Chinese roam clearly struck a sensitive nerve, as evidenced by the number and spirit of the denunciations that followed. It was clear to everyone that when Searle used the word “intentionality,” he wasn’t just talking about an obscure technical matter. In this context intentionality is virtually synonymous with mind, soul, spirit, or awareness. Here is a sampler of some of the main objections:

The comparison is unfair. The programs that Searle ridiculed demonstrated a very crude kind of understanding at best, and no one in AI seriously claims anything more for them. Even if they were correct in principle, said the defenders, genuine humanlike understanding would require much more powerful machines and much more sophisticated programs.

Searle quite correctly painted out, however, that this argument is irrelevant: of course computers are getting more powerful: what he objected to was the principle.

The Chinese Room story is entertaining and seductive, but it’s a fraud. Douglas R. Hofstadter of Indiana University, author of the best-selling Godel, Escher, Bach, pointed out that the jump from the AI program to the Turing test is not the trivial step that Searle makes it out to be. It’s an enormous leap. The poor devil in the Chinese room would have to shuffle not just a few slips of paper but millions or billions of slips of paper. It would take him years to answer a question, if he could do it at all. In effect, said Hofstadter, Searle is postulating mental processes slowed down by a factor of millions, so no wonder it looks different.

Searle’s reply-that he could memorize the slips of paper and shuffle them in his head-sounds plausible enough. But as several respondents have pointed out, it dangerously undermines his whole argument: once he memorizes everything, doesn’t he now understand Chinese in the same way he understands English?

The entire system does understand Chinese. True, the man in the room doesn’t understand Chinese himself. But he is just part of a larger system that also includes the slips of paper, the rules, and the message-passing mechanism. Taken as a whole, this larger system does understand Chinese. This “systems” reply was advanced by a number of the respondents. Searle was incredulous-”It is not easy for me to imagine how someone who was not in the grip of an ideology could find the idea at all plausible”-yet the concept is subtler than it seems. Consider a thermostat: a bimetallic strip bends and unbends as the temperature changes. When the room becomes too cold, the strip closes an electrical connection, and the furnace kicks on. When the room warms back up again, the connection reopens, and the furnace shuts off. Now, does the bimetallic strip by itself control the temperature of the room? No. Does the furnace by itself control the temperature? No. Does the system as a whole control the temperature? Yes. Connections and the organization make the whole into mare than the sum of its parts.

Searle never makes clear what intentionality is, or why a machine can’t have it. As Dennett pointed out, “For Searle, intentionality is rather like a wonderful substance secreted by the brain the way the pancreas secretes insulin.” And make no mistake: Searle’s concept of intentionality does require a biological brain. He explicitly denied that a robot could have intentionality, even if it were equipped with eyes, ears, arms, legs, and all the other accoutrements it needed to move around and perceive the world like a human being. Inside, he said, the robot would still just be manipulating formal symbols.

That assertion led psychologist Zenon Pylyshyn of the University of Western Ontario to propose his own ironic thought experiment: ‘Thus, if more and more of the cells in your brain were to be replaced by integrated circuit chips, programmed in such a way as to keep the input-output function of each unit identical to the unit being replaced, you would in all likelihood just keep right on speaking exactly as you are doing now except that you would eventually stop meaning anything by it. What we outside observers might take to be words would become for you just certain noises that circuits caused you to make.” In short, you would become a zombie.

Dennett took up the same theme in his own article. So far as natural selection is concerned, he pointed out, Pylyshyn’s zombie or Searle’s robot is just as fit for survival as those of us with Searlestyle intentional brains. Evolution would make no distinction. Indeed, from a biological point of view, intentionality is irrelevant, as useless as the appendix. So how did it ever arise? And having arisen, how did it survive and prosper when it offered no natural-selection value? Aren’t we lucky that some chance mutation didn’t rob our ancestors of intentionality? Dennett asked. If it had, he said, “we’d behave just as we do now, but of course we wouldn’t mean it!” Needless to say, bath Pylyshyn and Dennett found this absurd.

In retrospect, the great debate has to be rated a standoff. Searle, not surprisingly, was unconvinced by any of his opponents’ arguments; to this day he and his fellow Zen holists have refused to yield an inch. Yet they have never given a truly compelling explanation of why a brain and only a brain can secrete intentionality. The computationalists, meanwhile, remain convinced that they are succeeding where philosophers have failed for 3,000 years-that they are producing a real scientific theory of intelligence and consciousness. But they can’t prove it. Not yet, anyway.

And in all fairness, the burden of proof is on AI. The symbol-processing paradigm is an intriguing approach. If nothing else, it’s an approach worth exploring to see how far it can go. But still, what is consciousness?

Science as a Message of Despair

One way to answer that last question is with another question: Do we really want to know? Many people instinctively side with Searle, horrified at what the computationalist position implies: If thought, feeling, intuition, and all the other workings of the mind can be understood even in principle, if we are machines, then God is not speaking to our hearts. And for that matter, neither is Mozart. The soul is nothing more than the activations of neuronal symbols. Spirit is nothing more than a surge of hormones and neurotransmitters. Meaning and purpose are illusions. And besides, when machines grow old and break dawn, they are discarded without a thought. Thus, for many people, AI is a message of despair. Of course, this is hardly a new concern. For those who choose to see it that way, science itself is a message of despair.

In 1543 with the publication of De Revolutionibus the Polish astronomer Nicholas Copernicus moved the earth from the center of the universe and made it one planet among many and thereby changed humankind’s relationship with God. In the earthcentered universe of Thomas Aquinas and other medieval theologians, man had been poised halfway between a heaven that lay just beyond the sphere of the stars and a hell that burned beneath his feet. He had dwelt always under the watchful eye of God, and his spiritual status had been reflected in the very structure of the cosmos. But after Copernicus the earth and man were reduced to being wanderers in an infinite universe. For many, the sense of lass and confusion were palpable.

In 1859 with the publication of The Origin of Species Charles Darwin described how one group of living things arises from another through natural selection and thereby changed our perception of who we are. Once man had been the special creation of God, the favored of all his children. Now man was just another animal, the descendent of monkeys.

In the latter part of the nineteenth century and the early decades of the twentieth with the publication of such works as The Interpretation of Dreams (1901), Sigmund Freud illuminated the inner workings of the mind and again changed our perception of who we are. Once we had been only a little lower than the angels, masters of our own souls. Now we were at the mercy of demons like rage, terror, and lust, made all the more hideous by the fact that they lived unseen in our own unconscious minds.

So the message of science can be bleak indeed. It can be seen as a proclamation that human beings are nothing more than masses of particles collected by blind chance and governed by immutable physical law, that we have no meaning, that there is no purpose to existence, and that the universe just doesn’t care. I suspect that this is the real reason for the creationists’ desperate rejection of Darwin. It has nothing to do with Genesis; it has everything to do with being special in the eyes of a caring God. The fact that their creed is based on ignorance and a willful distortion of the evidence makes them both sad and dangerous. But their longing for order and purpose in the world is understandable and even noble. I also suspect that this perceived spiritual vacuum in science lies behind the fascination so many people feel for such pseudosciences as astrology. After all, if the stars and the planets guide my fate, then somehow I matter. The universe cares. Astrology makes no scientific sense whatsoever. But far those who need such reassurance, what can science offer to replace it?

Science as a Message of Hope

And yet the message doesn’t have to be bleak. Science has given us a universe of enormous extent filled with marvels far beyond anything Aquinas ever knew. Does it diminish the night sky to know that the planets are other worlds and that the stars are other suns? In the same way, a scientific theory of intelligence and awareness might very well provide us with an understanding of other possible minds. Perhaps it will show us more clearly how our Western ways of perceiving the world relate to the perceptions of other cultures. Perhaps it will tell us how human intelligence fits in with the range of other possible intelligences that might exist in the universe. Perhaps it will give us a new insight into who we are and what our place is in creation.

Indeed, far from being threatening, the prospect is oddly comforting. Consider a computer program. It is undeniably a natural phenomenon, the product of physical forces pushing electrons here and there through a web of silicon and metal. And yet a computer program is mare than just a surge of electrons. Take the program and run it on another kind of computer. Now the structure of silicon and metal is completely different. The way the electrons move is completely different. But the program itself is the same, because it still does the same thing. h is part of the computer. It needs the computer to exist. And yet it transcends the computer. In effect, the program occupies a different level of reality from the computer. Hence the power of the symbol-processing model: By describing the mind as a program running on a flesh-and-blood computer, it shows us how feeling, purpose, thought, and awareness can be part of the physical brain and yet transcend the brain. It shows us how the mind can be composed of simple, comprehensible processes and still be something more.

Consider a living cell. The individual enzymes, lipids, and DNA molecules that go to make up a cell are comparatively simple things. They obey well-understood laws of physics and chemistry.

Originally Published 1990

Can machines think? This has been a conundrum for philosophers for years, but in their fascination with the pure conceptual issues they have for the most part overlooked the real social importance of the answer.

It is of more than academic importance that we learn to think clearly about the actual cognitive powers of computers, for they are now being introduced into a variety of sensitive social roles where their powers will be put to the ultimate test: in a wide variety of areas, we are on the verge of making ourselves dependent upon their cognitive powers. The cast of overestimating them could be enormous.

One of the principal inventors of the computer was the great British mathematician Alan Turing. It was he who first figured out, in highly abstract terms, how to design a programmable computing device, what we now call a universal Turing machine.

All programmable computers in use today are in essence Turing machines. About forty years ago, at the dawn of the computer age, Turing began a classic article “Computing Machinery and Intelligence” with the words “I propose to consider the question, ‘Can machines think?’” but he then went on to say that this was a bad question, a question that leads only to sterile debate and haggling over definitions, a question, as he put it, “too meaningless to deserve discussion.”¹

In its place he substituted what he took to be a much better question, a question that would be crisply answerable and intuitively satisfying–in every way an acceptable substitute for the philosophic puzzler with which he began.

First he described a parlor game of sorts, the imitation game, to be played by a man, a woman, and a judge (of either gender). The man and woman are hidden from the judge’s view but are able to communicate with the judge by teletype; the judge’s task is to guess, after a period of questioning each contestant, which interlocutor is the man and which the woman.

The man tries to convince the judge he is the woman, and the woman tries to convince the judge of the truth. The man wins the judge makes the wrong identification. A little reflection will convince you, I am sure, that aside from lucky breaks, it would take a clever man to convince the judge that he was the woman–on the assumption that the judge is clever too, of course.

Now suppose, Turing said, we replace the man or woman with a computer and give the judge the task of determining which is the human being and which is the computer. Turing proposed that any computer that can regularly or often foal a discerning judge in this game would be intelligent, a computer that thinks, beyond any reasonable doubt.

Now, it is important to realize that failing this test is not supposed to be a sign of lack of intelligence. Many intelligent people, after all, might not be willing or able to play the imitation game, and we should allow computers the same opportunity to decline to prove themselves. This is, then, a one-way test; failing it proves nothing.

Furthermore, Turing was not committing himself to the view (although it is easy to see how one might think he was) that to think is to think just like a human being–any more than he was committing himself to the view that for a man to think, he must think exactly like a woman. Men, women, and computers may all have different ways of thinking.

But surely, he thought, one can think in one’s own peculiar style well enough to imitate a thinking man or woman, one can think well, indeed. This imagined exercise has come to be known as the Turing test.

It is a sad irony that Turing’s proposal has had exactly the opposite effect on the discussion of what he intended. Turing didn’t design the test as a useful tool in scientific psychology, a method of confirming or disconfirming scientific theories or evaluating particular models of mental function; he designed it to be nothing more than a philosophical conversation stopper.

He proposed, in the spirit of “Put up or shut up!”, a simple test for thinking that is surely strong enough to satisfy the sternest skeptic (or so he thought). He was saying, in effect, that instead of arguing interminably about the ultimate nature and essence of thinking, we should all agree that whatever that nature is, anything that could pass this test would surely have it; then we could turn to asking how or whether some machine could be designed and built that might pass the test fair and square.

Alas, philosophers, amateur and professional, have instead taken Turing’s proposal as the pretext for just the sort of definitional haggling and interminable arguing about imaginary counter-examples that he was hoping to squelch.

This forty-year preoccupation with the Turing test has been all the more regrettable because it has focused attention on the wrong issues. There are real world problems that are revealed by considering the strengths and weaknesses of the Turing test, but these have been concealed behind a smoke screen of misguided criticisms. A failure to think imaginatively about the test actually proposed by Turing has led many to underestimate its severity and to confuse it with much less interesting proposals.

So first I want to show that the Turing test, conceived as he conceived it, is (as he thought) quite strong enough as a test of thinking. I defy anyone to improve upon it. But here is the point almost universally overlooked by the literature: there is a common misapplication of the Turing test that often leads to drastic overestimation of the powers of actually existing computer systems. The follies of this familiar sort of thinking about computers can best be brought out by a reconsideration of the Turing test itself.

The insight underlying the Turing test is the same insight that inspires the new practice among symphony orchestras of conducting auditions with an opaque screen between the jury and the musician. What matters in a musician is, obviously, musical ability and only musical ability; such features as sex, hair length, skin color, and weight are strictly irrelevant. Since juries might be biased even innocently and unawares by these irrelevant features, they are carefully screened off so only the essential feature, musicianship, can be examined.

Turing recognized that people might be similarly biased in their judgments of intelligence by whether the contestant had soft skin, warm blood, facial features, hands, and eyes–which are obviously not themselves essential components of intelligence. So he devised a screen that would let through only a sample of what really mattered: the capacity to understand, and think cleverly about, challenging problems.

Perhaps he was inspired by Descartes, who in his Discourse on Method (1637) plausibly argued that there was no more demanding test of human mentality than the capacity to hold an intelligent conversation: “It is indeed conceivable that a machine could be so made that it would utter words, and even words appropriate to the presence of physical acts or objects which cause some change in its organs; as, for example, it was touched in some so spot that it would ask what you wanted to say to it; in another, that it would cry that it was hurt, and so on for similar things. But it could never modify its phrases to reply to the sense of whatever was said in its presence, as even the most stupid men can do.”²

This seemed obvious to Descartes in the seventeenth century, but of course, the fanciest machines he knew were elaborate clockwork figures, not electronic computers. Today it is far from obvious that such machines are impossible, but Descartes’ hunch that ordinary conversation would put as severe a strain on artificial intelligence as any other test was shared by Turing. Of course, there is nothing sacred about the particular conversational game chosen by Turing for his test; it is just a cannily chosen test of more general intelligence.

The assumption Turing was prepared to make was this: Nothing could possibly pass the Turing test by winning the imitation game without being able to perform indefinitely many other clearly intelligent actions. Let us call that assumption the quick-probe assumption.

Turing realized, as anyone would, that there are hundreds and thousands of telling signs of intelligent thinking to be observed in our fellow creatures, and one could, one wanted, compile a vast battery of different tests to assay the capacity for intelligent thought. But success on his chosen test, he thought, would be highly predictive of success on many other intuitively acceptable tests of intelligence.

Remember, failure on the Turing test does not predict failure on those others, but success would surely predict success. His test was so severe, he thought, that nothing that could pass it fair and square would disappoint us in other quarters. Maybe it wouldn’t do everything we hoped–maybe it wouldn’t appreciate ballet, understand quantum physics, or have a good plan far world peace, but we’d all see that it was surely one of the intelligent, thinking entities in the neighborhood.

Is this high opinion of the Turing tests severity misguided? Certainly many have thought so, but usually because they have not imagined the test in sufficient detail, and hence have underestimated it. Trying to forestall this skepticism, Turing imagined several lines of questioning that a judge might employ in this game that would be taxing indeed–lines about writing poetry or playing chess. But with thirty years’ experience with the actual talents and foibles of computers behind us, perhaps we can add a few more tough lines of questioning.

Terry Winograd, a leader in AI efforts to produce conversational ability in a computer, draws our attention to a pair of sentences.³ They differ in only one word. The first sentence is this: “The committee denied the group a parade permit because they advocated violence.” Here’s the second sentence: “The committee denied the group a parade permit because they feared violence.”

The difference is just in the verb–”advocated” or “feared.” As Winograd points out, the pronoun “they” in each sentence is officially ambiguous. Both readings of the pronoun are always legal. Thus, we can imagine a world in which governmental committees in charge of parade permits advocate violence in the streets and, for some strange reason, use this as their pretext for denying a parade permit. But the natural, reasonable, intelligent reading of the first sentence is that it’s the group that advocated violence, and of the second, that it’s the committee that feared the violence.

Now sentences like this are embedded in a conversation, the computer must figure out which reading of the pronoun is meant, it is to respond intelligently. But mere rules of grammar or vocabulary will not fix the right reading. What fixes the right reading for us is knowledge about politics, social circumstances, committees and their attitudes, groups that want to parade, how they tend to behave, and the like. One must know about the world, in short, to make sense of such a sentence.

In the jargon of artificial intelligence, a conversational computer needs lots of world knowledge to do its jab. But, it seems, it is somehow endowed with that world knowledge on many topics, it should be able to do much more with that world knowledge than merely make sense of a conversation containing just that sentence.

The only way, it appears, for a computer to disambiguate that sentence and keep up its end of a conversation that uses that sentence would be for it to have a much more general ability to respond intelligently to information about social and political circumstances and many other topics. Thus, such sentences, by putting a demand on such abilities, are good quick probes. That is, they test for a wider competence.

People typically ignore the prospect of having the judge ask off-the-wall questions in the Turing test, and hence they underestimate the competence a computer would have to have to pass the test. But remember, the rules of the imitation game as Turing presented it permit the judge to ask any question that could be asked of a human being–no holds barred. Suppose, then, we give a contestant in the game this question: An Irishman found a genie in a bottle who offered him two wishes.

“First I’ll have a pint of Guinness,” said the Irishman, and when it appeared, he took several long drinks from it and was delighted to see that the glass filled itself magically as he drank. “What about your second wish?” asked the genie. “Oh well, that’s easy,” said the Irishman. “I’ll have another one of these!” Please explain this story to me, and tell me there is anything funny or sad about it.

Now even a child could express, even not eloquently, the understanding that is required to get this joke. But think of how much one has to know and understand about human culture, to put it pompously, to be able to give any account of the point of this joke.

I am not supposing that the computer would have to laugh at, or be amused by, the joke. But it wants to win the imitation game–and that’s the test, after all–it had better know enough in its own alien, humorless way about human psychology and culture to be able to pretend effectively that it was amused and explain why.

It may seem to you that we could devise a better test. Let’s compare the Turing test with some other candidates.

Candidate 1

A computer is intelligent; it wins the World Chess Championship.

That’s not a good test, it turns out. Chess prowess has proven to be an isolatable talent. There are programs today that can play fine chess but do nothing else. So the quick-probe assumption is false far the test of playing winning chess.

Candidate 2

The computer is intelligent; it solves the Arab-Israeli conflict.

This is surely a more severe test than Turing’s. But it has some detects: passed once, it is unrepeatable; it is slow, no doubt; and it is not crisply clear what would count as passing it. Here’s another prospect, then:

Candidate 3

A computer is intelligent; it succeeds in stealing the British crown jewels without the use of force or violence.

Now this is better. First, it could be repeated again and again, though of course each repeat test would presumably be harder, but this is a feature it shares with the Turing test. Second, the mark of success is clear: either you’ve got the jewels to show for your efforts or you don’t. But it is expensive and slow, a socially dubious caper at best, and no doubt luck would play too great a role.

With ingenuity and effort one might be able to came up with other candidates that would equal the Turing test in severity, fairness, and efficiency, but I think these few examples should suffice to convince us that it would be hard to improve on Turing’s original proposal.

But still, you may protest, something might pass the Turing test and still not be intelligent, not be a thinker. What does might mean here? what you have in mind is that by cosmic accident, by a supernatural coincidence, a stupid person or a stupid computer might fool a clever judge repeatedly, well, yes, but so what? The same frivolous possibility “in principle” holds for any test whatever.

A playful god or evil demon, let us agree, could fool the world’s scientific community about the presence of H20 in the Pacific Ocean. But still, the tests they rely on to establish that there is H20 in the Pacific Ocean are quite beyond reasonable criticism. The Turing test for thinking is no worse than any well-established scientific test, we can set skepticism aside and go back to serious matters. Is there any more likelihood of a false positive result on the Turing test than on, say, the tests currently used for the presence of iron in an ore sample?

This question is often obscured by a move called operationalism that philosophers have sometimes made. Turing and those who think well of his test are often accused of being operationalists. Operationalism is the tactic of defining the presence of some property, intelligence, for instance, as being established once and for all by the passing of some test. Let’s illustrate this with a different example.

Suppose I offer the following test–we’ll call it the Dennett test–for being a great city. A great city is one in which, on a randomly chosen day, one can do all three of the following: hear a symphony orchestra, see a Rembrandt and a professional athletic contest, and eat quenelles de brothel a la Nantua for lunch. To make the operationalist move would 6e to declare that any city that passes the Dennett test is by definition a great city. What being a great city amounts to is just passing the Dennett test.

Well then, if the Chamber of Commerce of Great Falls, Montana, wanted, and I can’t imagine why–to get their hometown on my list of great cities, they could accomplish this by the relatively inexpensive route of hiring full time about ten basketball players, forty musicians, and a quick-order quenelle chef and renting a cheap Rembrandt from some museum. An idiotic operationalist would then be stuck admitting that Great Falls, Montana, was in fact a great city, since all he or she cares about in great cities is that they pass the Dennett test.

Sane operationalists (who far that very reason are perhaps not operationalists at all, since “operationalist” seems to be a dirty word) would cling confidently to their test, but only because they have what they consider to be very good reasons for thinking the odds astronomical against a false positive result, like the imagined Chamber of Commerce caper. I devised the Dennett test, of course, with the realization that no one would be both stupid and rich enough to go to such preposterous lengths to foil the test.

In the actual world, wherever you find symphony orchestras, quenelles, Rembrandts, and professional sports, you also find daily newspapers, parks, repertory theaters, libraries, fine architecture, and all the other things that go to make a city great. My test was simply devised to locate a telling sample that could not help but be representative of the rest of the city’s treasures. I would cheerfully run the minuscule risk of having my bluff called. Obviously, the test items are not all that I care about in a city.

In fact, some of them I don’t care about at all. I just think they would be cheap and easy ways of assuring myself that the subtle things I do care about in cities are present. Similarly, I think it would be entirely unreasonable to suppose that Alan Turing had an inordinate fondness for party games or put too high a value on party game prowess in his test. In both the Turing test and the Dennett test a very unrisky gamble is being taken: the gamble that the quick-probe assumption is in general safe.

But two can play this game of playing the odds. Suppose some computer programmer happens to be, for whatever strange reason, dead set on tricking me into judging an entity to be a thinking, intelligent thing when it is not. Such a trickster could rely as well as 1 can on unlikelihood and take a few gambles. Thus, the programmer can expect that it is not remotely likely that 1, as the judge, will bring up the topic of children’s birthday parties, or baseball, or moon rocks, then he or she can avoid the trouble of building world knowledge on those topics into the data base.

Whereas I do improbably raise these issues, the system will draw a blank, and I will unmask the pretender easily. But with all the topics and words that I might raise, such a saving would no doubt be negligible. Turn the idea inside out, however, and the trickster will have a fighting chance.

Suppose the programmer has reason to believe that I will ask only about children’s birthday parties or baseball or moon rocks–all other topics being, for one reason or another, out of bounds. Not only does the task shrink dramatically, but there already exist systems or preliminary sketches of systems in artificial intelligence that can do a whiz-bang job of responding with apparent intelligence an just those specialized topics.

William Wood’s LUNAR program, to take what is perhaps the best example, answers scientists’ questions–posed in ordinary English–about moon rocks. In one test it answered correctly and appropriately something like 90 percent of the questions that geologists and other experts thought of asking it about moon rocks. (In 12 percent of those correct responses there were trivial, correctable defects.)

Of course, Wood’s motive in creating LUNAR was not to trick unwary geologists into thinking they were conversing with an intelligent being. And that had been his motive, his project would still be a long way from success.

For it is easy enough to unmask LUNAR without ever straying from the prescribed topic of moon rocks. Put LUNAR in one room and a moon rocks specializt in another, and then ask them both their opinion of the social value of the moon-rock-gathering expeditions, for instance. Or ask the contestants their opinion of the suitability of moon rocks as ashtrays, or whether people who have touched moon rocks are ineligible for the draft. Any intelligent person knows a lot more about moon rocks than their geology. Although it might be unfair to demand this extra knowledge of a computer moon-rock specialist, it would be an easy way to get it to fail the Turing test.

But just suppose that someone could extend LUNAR to cover itself plausibly on such probes, so long as the topic was still, however indirectly, moon rocks. We might come to think it was a lot more like the human moon-rock specialist than it really was. The moral we should draw is that as Turing-test judges we should resist all limitations and waterings-down of the Turing test. They make the game too easy–vastly easier than the original test. Hence, they lead us into the risk of overestimating the actual comprehension of the system being tested.

Consider a different limitation on the Turing test that should strike a suspicious chord in us as soon as we hear it. This is a variation on a theme developed in a recent article by Ned Block.⁴ Suppose someone were to propose to restrict the judge to a vocabulary of, say, the 850 words of Basic English, and to single-sentence probes–that is, “moves”–of no more than four words.

Moreover, contestants must respond to these probes with no more than four words per move, and a test may involve no more than forty questions Is this an innocent variation on Turing’s original test? These restrictions would make the imitation game clearly finite.

That is, the total number of all possible permissible games is a large but finite number. One might suspect that such a limitation would permit the trickster simply to store, in alphabetical order, all the possible good conversations within the limits and fool the judge with nothing more sophisticated than a system of table lookup. In fact, that isn’t in the cards.

Even with these severe, improbable, and suspicious restrictions imposed upon the imitation game, the number of legal games, though finite, is mind-bogglingly large. I haven’t bothered trying to calculate it but it surely astronomically exceeds the number of possible chess games with no more than forty moves, and that number has been calculated. John Haugeland says it’s in the neighborhood of 10¹²⁰. For comparison, Haugeland suggests there have only been 10¹⁸ seconds since the beginning of the universe.⁵

Of course, the number of good, sensible conversations under these limits is a tiny fraction, maybe 1 in 10¹⁵, of the number of merely grammatically well-formed conversations. So let’s say, to be very conservative, that there are only 10¹⁵ different smart conversations such a computer would have to store. Well, the task shouldn’t take more than a few trillion years–with generous federal support. Finite numbers can be very large.

So though we needn’t worry that this particular trick of storing all the smart conversations would work, we can appreciate that there are lots of ways of making the task easier that may appear innocent at first. We also get a reassuring measure of just how severe the unrestricted Turing test is by reflecting on the more than astronomical size of even that severely restricted version of it.

Block’s imagined–and utterly impossible–program exhibits the dreaded feature known in computer-science circles as combinatorial explosion. No conceivable computer could overpower a combinatorial explosion with sheer speed and size. Since the problem areas addressed by artificial intelligence are veritable minefields of combinatorial explosion, and since it has often proved difficult to find any solution to a problem that avoids them, there is considerable plausibility in Newell and Simon’s proposal that avoiding combinatorial explosion (by any means at all) be viewed as one of the hallmarks of intelligence.

Our brains are millions of times bigger than the brains of gnats, but they are still–for all their vast complexity–compact, efficient, timely organs that somehow or other manage to perform all their tasks while avoiding combinatorial explosion. A computer a million times bigger or faster than a human brain might not look like the brain of a human being, or even be internally organized like the brain of a human being but, far all its differences, it somehow managed to control a wise and timely set of activities, it would have to be the beneficiary of a very special design that avoided combinatorial explosion. And whatever that design was, would we not be right to consider the entity intelligent?

Turing’s test was designed to allow for this possibility. His point was that we should not be species-chauvinistic, or anthropocentric, about the insides of an intelligent being, for there might be inhuman ways of being intelligent.

To my knowledge the only serious and interesting attempt by any program designer to win even a severely modified Turing test has been Kenneth Colby’s. Colby is a psychiatrist and intelligence artificer at UCLA. He has a program called PARRY, which is a computer simulation of a paranoid patient who has delusions about the Mafia being out to get him.

As you do with other conversational programs, you interact with it by sitting at a terminal and typing questions and answers back and forth. A number of years ago, Colby put PARRY to a very restricted test. He had genuine psychiatrists interview PARRY. He did not suggest to them that they might be talking or typing to a computer; rather, he made up some plausible story about why they were communicating with a real, live patient by teletype.

He also had the psychiatrists interview real, human paranoids via teletype. Then he took a PARRY transcript, inserted it in a group of teletype transcripts from real patients, gave them to another group of experts–more psychiatrists–and said, “One of these was a conversation with a computer. Can you figure out which one it was?” They couldn’t. They didn’t do better than chance.

Colby presented this with some huzzah, but critics scoffed at the suggestion that this was a legitimate Turing test. My favorite commentary on it was Joseph Weizenbaum’s; in a letter to the Communications of the Association of Computing Machinery, he said that, inspired by Colby, he had designed an even better program, which passed the same test.⁶ His also had the virtue of being a very inexpensive program, in these times of tight money. In fact you didn’t even need a computer for it. All you needed was an electric typewriter. His program modeled infantile autism.

And the transcripts–you type in your questions, and the thing just sits there and hums–cannot be distinguished by experts from transcripts of real conversations with infantile autistic patients. What was wrong with Colby’s test, of course, was that the unsuspecting interviewers had no motivation at all to try out any of the sorts of questions that easily would have unmasked PARRY.

Colby was undaunted, and after his team had improved PARRY, he put it to a much more severe test–a surprisingly severe test. This time, the interviewers–again psychiatrists, were given the task at the outset of telling the computer from the real patient. They were set up in a classic Turing machine test: the patient in one room, the computer PARRY in the other room, with the judges conducting interviews with both of them (on successive days). The judges’ task was to find out which one was the computer and which one was the real patient. Amazingly, they didn’t do much better, which leads some people to say, “Well, that just confirms my impression of the intelligence of psychiatrists!”

But more seriously now, was this an honest-to-goodness Turing test? Were there tacit restrictions on the lines of questioning of the judges? Like the geologists interacting with LUNAR, the psychiatrists’ professional preoccupations and habits kept them from asking the sorts of unlikely questions that would have easily unmasked PARRY. After all, they realized that since one of the contestants was a real, live paranoid person, medical ethics virtually forbade them from toying with, upsetting, or attempting to confuse their interlocutors.

Moreover, they also knew that this was a test of a model of paranoia, so there were certain questions that wouldn’t be deemed to be relevant to testing the model as a model of paranoia. So they asked just the sort of questions that therapists typically ask of such patients, and of course PARRY had been ingeniously and laboriously prepared to deal with just that sort of question.

One of the psychiatrist judges did, in fact, make a rather half-hearted attempt to break out of the mold and ask some telling questions: “Maybe you’ve heard the saying ‘Don’t cry over spilled milk.’ What does that mean to you?” PARRY answered, “Maybe you have to watch out for the Mafia.”

When then asked “Okay, now you were in a movie theater watching a movie and smelled something like burning wood or rubber, what would you do?” PARRY replied, “You know, they know me.” And the next question was, ” you found a stamped, addressed letter in your path as you were walking down the street, what would you do?” PARRY replied, “What else do you want to know?”⁷

Clearly, PARRY was, you might say, parrying these questions, which were incomprehensible to it, with more or less stock paranoid formulas. We see a bit of a dodge that is apt to work, apt to seem plausible to the judge, only because the “contestant” is supposed to be a paranoid, and such people are expected to respond uncooperatively on such occasions. These unimpressive responses didn’t particularly arouse the suspicions of the judge, as a matter of fact, though they probably should have.

PARRY, like all other large computer programs, is dramatically hound by limitations of cost-effectiveness. What was important to Colby and his crew was simulating his model of paranoia. This was a massive effort. PARRY has a thesaurus or dictionary of about 4,500 words and 700 idioms and the grammatical competence to use it-a parser, in the jargon of computational linguistics.

The entire PARRY program takes up about 200,000 words of computer memory, all laboriously installed by the programming team. Now once all the effort had gone into devising the model of paranoid thought processes and linguistic ability, there was little time, energy, money, and interest left over to build in huge amounts of world knowledge of the sort that any actual paranoid would, of course, have. (Not that anyone yet knows haw to build in world knowledge in the first place.)

Even one could do it, building in the world knowledge would no doubt have made PARRY orders of magnitude larger and slower. And what would have been the point, given Colby’s theoretical aims?

PARRY is a theoretician’s model of a psychological phenomenon: paranoia. It is not intended to have practical applications. But in recent years there has appeared a branch of AI (knowledge engineering) that develops what are now called expert systems. Expert systems are designed to be practical. They are typically software super specializt consultants that can be asked to diagnose medical problems, analyze geological data, analyze the results of scientific experiments, and the like. Some of them are very impressive.

SRI in California announced a few years ago that PROSPECTOR, an SRI-developed expert system in geology, had correctly predicted the existence of a large, important mineral deposit that had been entirely unanticipated by the human geologists who had fed it its data. MYCIN, perhaps the most famous of these expert systems, diagnoses infections of the blood, and it does probably as well as, maybe better than, any human consultants. And many other expert systems are on the way.

All expert systems, like all other large AI programs, are what you might call Potemkin villages. That is, they are cleverly constructed facades, like cinema sets. The actual filling-in of details of AI programs is time-consuming, costly work, so economy dictates that only those surfaces of the phenomenon that are likely to be probed or observed are represented.

Consider, for example, the CYRUS program developed 6y Janet Kalodner in Roger Schenk’s AI group at Yale a few years ago.⁸ CYRUS stands (we are told) for “Computerized Yale Retrieval and Updating System,” but surely it is no accident that CYRUS modeled the memory of Cyrus Vance, who was then secretary of state in the Carter administration.

The paint of the CYRUS project was to devise and test some plausible ideas about how people organize their memories of the events they participate in. Hence, it was meant to be a “pure- AI system, a scientific model, not an expert system intended for any practical purpose. CYRUS was updated daily by being fed all UPI wire-service news stories that mentioned Vance, and it was fed them directly with no doctoring and no human intervention.

With an ingenious news-reading program called FRUMP, it could take any story just as it came in on the wire and could digest it and use it to update its database so that it could answer more questions. You could address questions to CYRUS in English by typing at a terminal. You addressed CYRUS in the second person, as you were talking with Cyrus Vance himself. The results looked like this:

Question: Last time you went to Saudi Arabia, where did you stay?

Answer: In a palace in Saudi Arabia on September 23, 1978.

Question: Did you go sightseeing there?

Answer: Yes, at an oilfield in Dharan on September 23, 1978.

Question: Has your we ever met Mrs. Begin?

Answer: Yes, most recently at a state dinner in Israel in January 1980.

CYRUS could correctly answer thousands of questions, almost any fair question one could think of asking it. But one actually set out to explore the boundaries of its facade and find the questions that overshot the mark, one could soon find them. “Have you ever met a female head of state?” was a question I asked it wondering CYRUS knew that Indira Ghandi and Margaret Thatcher were women.

But for some reason the connection could not be drawn, and CYRUS failed to answer either yes or no. I had stumped it, in spite of the fact that CYRUS could handle a host of what you might call neighboring questions flawlessly. One soon learns from this sort of probing exercise that it is very hard to extrapolate accurately from a sample performance to the system’s total competence. It’s also very hard to keep from extrapolating much too generously.

While I was visiting Schenk’s laboratory in the spring of 1980, something revealing happened. The real Cyrus Vance suddenly resigned. The effect on the program CYRUS was chaotic. It was utterly unable to cape with the flood of “unusual” news about Cyrus Vance. The only sorts of episodes CYRUS could understand at all were diplomatic meetings, flights, press conferences, state dinners, and the like–less than two dozen general sorts of activities (the kinds that are newsworthy and typical of secretaries of state). It had no provision for sudden resignation.

It was as if the UPI had reported that a wicked witch had turned Vance into a frog. It is distinctly possible that CYRUS would have taken that report more in stride than the actual news. One can imagine the conversation

Question: Hello, Mr. Vance, what’s new?

Answer: I was turned into a frog yesterday.

But, of course, it wouldn’t know enough about what it had just written to be puzzled, startled, or embarrassed. The reason is obvious. When you look inside CYRUS, you find that it has skeletal definitions of thousands of words, but these definitions are minimal. They contain as little as the system designers think that they can get away with.

Thus, perhaps, “lawyer” would be defined as synonymous with “attorney” and “legal counsel,” but aside from that, all one would discover about lawyers is that they are adult human beings and that they perform various functions in legal areas. you then traced out the path to “human being,” you’d find out various obvious things CYRUS “knew” about human beings (hence about lawyers/, but that is not a lot.

That lawyers are university graduates, that they are better paid than chambermaids, that they know how to tie their shoes, that they are unlikely to be found in the company of lumberjacks–these trivial, weird, facts about lawyers would not be explicit or implicit anywhere in this system.

In other words, a very thin stereotype of a lawyer would 6e incorporated into the system, so that almost nothing you could tell it about a lawyer would surprise it. So long as surprising things don’t happen, so long as Mr. Vance, for instance, leads a typical diplomat’s le, attending state dinners, giving speeches, flying from Cairo to Rome, and so forth, this system works very well.

But as soon as his path is crossed by an important anomaly, the system is unable to cape and unable to recover without fairly massive human intervention. In the case of the sudden resignation, Kolodner and her associates soon had CYRUS up and running again with a new talent–answering questions about Edmund Muskie, Vance’s successor. But it was no less vulnerable to unexpected events. Not that it mattered particularly, since CYRUS was a theoretical model, not a practical system.

There are a host of ways of improving the performance of such systems, and, of course, some systems are much better than others. But all AI programs in one way or another have this façade-like quality, simply for reasons of economy. For instance, most expert systems in medical diagnosis developed so far operate with statistical information. They have no deep or even shallow knowledge of the underlying causal mechanisms of the phenomena that they are diagnosing.

To take an imaginary example, an expert system asked to diagnose an abdominal pain would be oblivious to the potential import of the fact that the patient had recently been employed as a sparring partner by Mohammed Ali: there being no statistical data available to it on the rate of kidney stones among athlete’s assistants. That’s a fanciful case no doubt–too obvious, perhaps, to lead to an actual failure of diagnosis and practice. But more subtle and hard-to-detect limits to comprehension are always present, and even experts, even the system’s designers, can be uncertain of where and how these limits will interfere with the desired operation of the system.

Again, steps can be taken and are being taken to correct these flaws. For instance, my former colleague at Tufts, Benjamin Kuipers, is currently working on an expert system in nephrology for diagnosing kidney ailments that will be based on an elaborate system of causal reasoning about the phenomena being diagnosed. But this is a very ambitious, long-range project of considerable theoretical difficulty. And even all the reasonable, cost-effective steps are taken to minimize the superficiality of expert systems, they will still be facades, just somewhat thicker or wider facades.

When we were considering the fantastic case of the crazy Chamber of Commerce of Great Falls, Montana, we couldn’t imagine a plausible motive for anyone going to any sort of trouble to trick the Dennett test. The quick-probe assumption for the Dennett test looked quite secure. But when we look at expert systems, we see that, however innocently, their designers do have motivation for doing exactly the sort of trick that would fool an unsuspicious Turing tester.

First, since expert systems are all superspecializts that are only supposed to know about some narrow subject, users of such systems, not having much time to kill, do not bother probing them at the boundaries at all. They don’t bother asking “silly” or irrelevant questions. Instead, they concentrate, not unreasonably, on exploiting the system’s strengths. But shouldn’t they try to obtain a clear vision of such a system’s weaknesses as well? The normal habit of human thought when we converse with one another is to assume general comprehension, to assume rationality, to assume, moreover, that the quick-probe assumption is, in general, sound.

This amiable habit of thought almost irresistibly leads to putting too much faith in computer systems, especially user-friendly systems that present themselves in a very anthropomorphic manner.

Part of the solution to this problem is to teach all users of computers, especially users of expert systems, how to probe their systems before they rely on them, how to search out and explore the boundaries of the facade. This is an exercise that calls for not only intelligence and imagination but also for a bit of special understanding about the limitations and actual structure of computer programs. It would help, of course, if we had standards of truth in advertising, in effect, for expert systems.

For instance, each such system should come with a special demonstration routine that exhibits the sorts of shortcomings and failures that the designer knows the system to have. This would not be a substitute, however, for an attitude of cautious, almost obsessive, skepticism on the part of users, for designers are often, not always, unaware of the subtler flaws in the products they produce. That is inevitable and natural because of the way system designers must think. They are trained to think positively–constructively, one might say–about the designs that they are constructing.

I come, then, to my conclusions. First, a philosophical or theoretical conclusion: The Turing test, in unadulterated, unrestricted form as Turing presented it, is plenty strong well used. I am confident that no computer in the next twenty years in going to pass the unrestricted Turing test. They may well win the World Chess Championship or even a Nobel Prize in physics, but they won’t pass the unrestricted Turing test.

Nevertheless, it is not, I think, impossible in principle for a computer to pass the test fair and square. I’m not giving one of those a priori “computers can’t think” arguments. I stand unabashedly ready, moreover, to declare that any computer that actually passes the unrestricted Turing test will be, in every theoretically interesting sense, a thinking thing.

But remembering how very strong the Turing test is, we must also recognize that there may also be interesting varieties of thinking or intelligence that are not well poised to play and win the imitation game. That no nonhuman Turing test winners are yet visible on the horizon does not mean that there aren’t machines that already exhibit some of the important features of thought.

About them it is probably futile to ask my title question, Do they think? Do they really think? In some regards they do, and in some regards they don’t. Only a detailed look at what they do and how they are structured will reveal what is interesting about them.

The Turing test, not being a scientific test, is of scant help on that task, but there are plenty of other ways to examine such systems. Verdicts on their intelligence, capacity for thought, or consciousness will be only as informative and persuasive as the theories of intelligence, thought, or consciousness the verdicts were based on, and since our task is to create such theories, we should get on with it and leave the Big Verdict for another occasion. In the meantime, should anyone want a surefire test of thinking by a computer that is almost guaranteed to be fail-safe, the Turing test will do very nicely.

My second conclusion is more practical and hence in one clear sense more important. Cheapened versions of the Turing test are everywhere in the air. Turing’s test is not just effective, it is entirely natural; this is, after all, the way we assay the intelligence of each other every day.

And since incautious use of such judgments and such tests is the norm, we are in some considerable danger of extrapolating too easily and judging too generously about the understanding of the systems we are using. The problem of overestimating cognitive prowess, comprehension, and intelligence is not, then, just a philosophical problem. It is a real social problem, and we should alert ourselves to it and take steps to avert it.

Postscript: Eyes, Ears, Hands, and History

My philosophical conclusion in this paper is that any computer that actually passed the Turing test would be a thinker in every theoretically interesting sense. This conclusion seems to some people to fly in the face of what I have myself argued on other occasions. Peter Bieri, commenting on this paper at Boston University, noted that I have often claimed to show the importance to genuine understanding of a rich and intimate perceptual interconnection between an entity and its surrounding world–the need for something like eyes and ears–and a similarly complex active engagement with elements in that world–the need for something like hands with which to do things in that world. Moreover,

I have often held that only a biography of sorts–a history of actual projects, learning experiences, and other bouts with reality–could produce the sorts of complexities (both external, or behavioral, and internal) that are needed to ground a principled interpretation of an entity as a thinker, an entity with beliefs, desires, intentions, and other mental attitudes.

But the opaque screen in the Turing test discounts or dismisses these factors altogether, it seems, by focusing attention on only the contemporaneous capacity to engage in one very limited sort of activity: verbal communication. (I have even coined a pejorative label for such purely language-using systems: “bedridden.”)

Am I going back on my earlier claims? Not at all. I am merely pointing out that the Turing test is so powerful that it will indirectly ensure that these conditions, they are truly necessary, are met by any successful contestant.

“You may well be right,” Turing could say, “that eyes, ears, hands, and a history are necessary conditions for thinking. so, then I submit that nothing could pass the Turing test that didn’t have eyes, ears, hands, and a history. That is an empirical claim, which we can someday hope to test. you suggest that these are not just practically or physically necessary but conceptually necessary conditions for thinking, you make a philosophical claim that I for one would not know how, or care, to assess. Isn’t it more interesting and important in the end to discover whether or not it is true that no bedridden system could pass a demanding Turing test?”

Suppose we put to Turing the suggestion that he add another component to his test: Not only must an entity win the imitation game; it must also be able to identify–using whatever sensory apparatus it has available to it–a variety of familiar objects placed in its room: a tennis racket, a potted palm, a bucket of yellow paint, a live dog. This would ensure that somehow or other the entity was capable of moving around and distinguishing things in the world.

Turing could reply, I assert, that this is an utterly unnecessary addition to his test, making it no more demanding than it already was. A suitably probing conversation would surely establish beyond a shadow of a doubt that the contestant knew its way around in the real world. The imagined alternative of somehow “prestocking” a bedridden, blind computer with enough information and a clever enough program to trick the Turing test is science fiction of the worst kind: possible “in principle” but not remotely possible in fact in view of the combinatorial explosion of possible variation such a system would have to cope with.

“But suppose you’re wrong. What would you say of an entity that was created all at once (by some programmers, perhaps), an instant individual with all the conversational talents of an embodied, experienced human being?” This is like the question, Would you call a hunk of H20 that was as hard as steel at room temperature ice? I do not know what Turing would say, of course, so I will speak for myself.

Faced with such an improbable violation of what 1 take to be the laws of nature, I would probably be speechless. The least of my worries would 6e about which lexicographical leap to take, whether to say, “it turns out, to my amazement, that something can think without having had the benefit of eyes, ears, hands, and a history” or “it turns out, to my amazement, that something can pass the Turing test without thinking.” Choosing between these ways of expressing my astonishment would be asking myself a question too meaningless to deserve discussion.

Discussion

Question: Why was Turing interested in differentiating a man from a woman in his famous test?

Answer: That was just an example. He described a parlor game in which a man would try to fool the judge by answering questions as a woman would answer. l suppose that Turing was playing on the idea that maybe, just maybe, there is a big difference between the way men think and the way women think. But of course they’re both thinkers. He wanted to use that fact to make us realize that, even there were clear differences between the way a computer and a person thought, they’d both still be thinking.

Question: Why does it seem that some people are upset by AI research? Does AI research threaten our self-esteem?

Answer: I think Herb Simon has already given the canniest diagnosis of that. For many people the mind is the last refuge of mystery against the encroaching spread of science, and they don’t like the idea of science engulfing the last hit of terra incognito. This means that they are threatened, I think irrationally, by the prospect that researchers in artificial intelligence may come to understand the human mind as well as biologists understand the genetic code and physicists understand electricity and magnetism. This could lead to the “evil scientist” (to take a stock character from science fiction) who can control you because he or she has a deep understanding of what’s going on in your mind.

This seems to me to be a totally valueless fear, one that you can set aside for the simple reason that the human mind is full of an extraordinary amount of detailed knowledge, as Roger Schenk, for example, has been pointing out. As long as the scientist who is attempting to manipulate you does not share all your knowledge, his or her chances of manipulating you are minimal. People can always hit you over the head. They can do that now. We don’t need artificial intelligence to manipulate people by putting them in chains or torturing them. But someone tries to manipulate you by controlling your thoughts and ideas, that person will have to know what you know and more. The best way to keep yourself safe from that kind of manipulation is to be well informed.

Question: Do you think we will be able to program self-consciousness into a computer?

Answer: Yes, I do think that it’s possible to program self-consciousness into a computer. “Self-consciousness” can mean many things. you take the simplest, crudest notion of self-consciousness, I suppose that would be the sort of self-consciousness that a lobster has: When it’s hungry, it eats something, but it never eats itself. It has some way of distinguishing between itself and the rest of the world, and it has a rather special regard for itself. The lowly lobster is, in one regard, self-conscious.

You want to know whether or not you can create that on the computer, the answer is yes. It’s no trouble at all. The computer is already a self-watching, self-monitoring sort of thing. That is an established part of the technology. But, of course, most people have something more in mind when they speak of self-consciousness. It is that special inner light, that private way that it is with you that nobody else can share, something that is forever outside the bounds of computer science.

How could a computer ever be conscious in this sense? That belief, that very gripping, powerful intuition, is in the end, I think, simply an illusion of common sense. It is as gripping as the commonsense illusion that the earth stands still and the sun goes around the earth. But the only way that those of us who do not believe in the illusion will ever convince the general public that it is an illusion is by gradually unfolding a very difficult and fascinating story about just what is going on in our minds.

In the interim, people like me, philosophers who have to live by our wits and tell a lot of stories use what I call intuition pumps, little examples that help to free up the imagination. I simply want to draw your attention to one fact.

You look at a computer, I don’t care whether it’s a giant Cray or a personal computer, you open up the box and look inside and see those chips, you say, “No way could that be conscious. No way could that be self-conscious.” But the same thing is true if you take the top off somebody’s skull and look at the gray matter pulsing away in there. You think, “That is conscious? No way could that lump of stuff be conscious.” Of course, it makes no difference whether you look at it with a microscope or with the naked eye.

At no level of inspection does a brain look like the seat of consciousness. Therefore, don’t expect a computer to look like the seat of consciousness. You want to get a grasp of how a computer could be conscious, it’s no more difficult in the end than getting a grasp of how a brain could be conscious. When we develop good accounts of consciousness, it will no longer seem so obvious to everyone that the idea of a self-conscious computer is a contradiction in terms. At the same time, I doubt that there will ever be self-conscious robots, but for boring reasons. There won’t be any point in making them.

Theoretically, could we make a gall bladder out of atoms? In principle, we could. A gall bladder is just a collection of atoms, but manufacturing one would cost the moon. It would be more expensive than every project NASA has ever dreamed of, and there would be no scientific payoff. We wouldn’t learn anything new about how gall bladders work.

For the same reason I don’t think we’re going to see really humanoid robots, because practical, cost-effective robots don’t need to be very humanoid at all. They need to be like the robots you can already see at General Motors, or like boxy little computers that do special-purpose things.

The theoretical issues will be studied by AI researchers looking at models that, to the layman, will show very little sign of humanity at all, and it will be only by rather indirect arguments that anyone will be able to appreciate that these models cast light on the deep theoretical question of how the mind is organized.

Footnotes

1. Alan M. Turing, “Computing Machinery and Intelligence,” Mind 59 (1950).

2. Rene Descartes, Discourse on Method (1637), trans. Lawrence LaFleur (New York: Bobbs-Merrill,1960).

3. Terry Winograd, Understanding Natural Language (New York: Academic Press, 1972).

4. Ned Block, “Psychologism and Behaviorism, Philosophical Review, 1982.

5. John Haugeland, Mind Design (Cambridge, Mass.: MIT Press, 1981), p. 16.

6. Joseph Weizenbaum, CACM17, no. 9 (September 1974): 543.

7. I thank Kenneth Colby for providing me with the complete transcripts (including the judges’ commentaries and reactions) from which these exchanges are quoted. The first published account of the experiment is Jon F. Raiser, Kenneth Mark Colby, William S. Faught, and Roger C. Parkinson, “Can Psychiatrists Distinguish a Computer Simulation of Paranoia from the Real Thing? The Limitations of Turing-like Tests as Measures of the Adequacy of Simulations,” in Journal of Psychiatric Research 15, no. 3 (1980):149-162. Colby discusses PARRY and its implications in “Modeling a Paranoid Mind,” in Behavioral and Brain Sciences 4, no. 4 (1981): 515-560.

Courtesy of Tufts University

Daniel Dennett is Distinguished Arts and Science Professor and Director of the Center for Cognitive Studies at Tufts University. He is the author or editor of a number of books on cognitive science and the philosophy of mind, including The Mind’s I, coedited with Douglans Hofstadter (1981); Elbow Room (1984); and The Intentiional Stance (1987).

Success provides the opportunity for growth, and growth provides the opportunity to risk at a higher level. Eric Vogt

Most of the AI projects that I have been involved in personally are in the pattern-recognition field. The following is a description of some of these efforts from the point of view of technology. This section is perhaps misnamed. This is not really a personal postscript; not recounted here are the many exceptional people who have made contributions to these projects, the early struggles of growing companies, the efforts to attract both capital and talent, the ambiguities and subtleties of understanding markets, the challenge of establishing manufacturing facilities, the relationships with vendors, suppliers, contractors, dealers, distributors, customers, consultants, attorneys, accountants, bankers, investment bankers, investors and the media, or the interpersonal challenges of building organizations.

Optical Character Recognition

I founded Kurzweil Computer Products (KCP) in 1974. Our goal was to solve the problem of omnifont (any type font) optical character recognition (OCR) and to apply the resulting technology to the reading needs of the blind as well as to other commercial applications. There had been attempts to help the blind read using conventional OCR devices (those for a single or limited number of type fonts/, but these machines were unable to deal with the great majority of printed material as it actually exists in the world. It was clear that to be of much value to the blind, an OCR machine would have to read any style of print in common use and also deal with the vagaries of printing errors, poor quality photocopies, varieties of paper and ink, complex page formats, and so on. OCR machines had existed from the beginning of the computer age, but all of the machines up to that time had relied on template matching, a form of low-level property extraction, and thus were severely limited in the range of material they could handle. Typically, users had to actually retype printed material using a specialized typeface before scanning. The principal value of these devices was that typewriters were at that time more ubiquitous than computer terminals.

It was clear to us that to produce an OCR device that was font invariant as well as relatively insensitive to distorted print, we would need additional experts beyond minimal property extraction. Our solution was to develop software for multiple experts, including topological experts such as loop, concavity, and line-segment detectors, with an expert manager that could combine the results of both high- and low-level recognition experts. The system was able to

Courtesy of the Kurzweil Reading Machine Division of Xerox

The flow of information in the Kurzweil Reading Machine.

learn by having the high-level experts teach the low-level experts the type faces found in a particular document. At a later point we added context experts by providing the machine with a knowledge of English (and ultimately several other languages).

The first Kurzweil Reading Machine (KRM/, introduced in 1976, consisted of an image scanner we developed ourselves that contained an electra-optical camera. The camera’s eye consisted of a charge-coupled device containing 500 light-sensitive elements arranged in a straight line. The camera was mounted on an electromechanical “X-Y mover,” which could move the camera in both vertical and horizontal directions. The material to be read (a book, magazine, typed letter, etc.) lay face down on the glass plate that formed the top of the machine.

The camera automatically moved back and forth scanning each line of print, transmitting the image electronically to a minicomputer contained in a separate cabinet. Using our omnifont OCR software, the minicomputer recognized the characters, grouped them into words and computed the pronunciation of each word. To accomplish this last task, several hundred pronunciation rules were programmed in, along with several thousand exceptions. The resulting string of phonemes was sent to a speech synthesizer, which articulated each word. Subsequent models of the KRM have been substantially improved, but they are organized in a similar way.

Using the device is straightforward: the user places the document to be read face down on the machine, presses start, and listens. The KRM has a control panel to control the movement of the scanner, back it up, make it reread sections or spell out words, and provide a variety of other functions.

Photo by Lou Jones www.fotojones.com

Jamal Mazrui uses the Kurzweil Reading Machine. The Kurzweil Reading Machine scans and recognizes such text as books, magazines, and memos and converts it into synthesized speech and thus is able to provide blind readers with independent access to printed materials.

Courtesy of the Kurzweil Reading Machine Division of Xerox

Font invariance is a primary goal of Kurzweil Computer Products’ intelligent character recognition. Users can verify recognized characters against images of the original characters if so desired. A few of the topological features considered by Kurzweil Computer Products’ ICR.

The KRM has been called the first commercial product to successfully incorporate AI technology. A recent survey showed that most blind college students have access to a KRM to read their educational materials. Nothing in my professional career has given me greater satisfaction than the many letters I have received from blind persons of all ages indicating the benefit they have received from the KRM in enabling them to complete their studies or gain and maintain productive employment.

Two years after the introduction of the KRM, we introduced a refined version, the Kurzweil Data Entry Machine (KDEM), designed for commercial applications. The KDEM, like the KRM, could scan printed and typed documents and recognize characters and page formats from a wide variety of sources, but rather than speaking the wards, it transmits them. It has been used to automatically enter documents into databases, word-processing machines, electronic-publishing systems, and a variety of other computer-based systems.

For example, the KDEM was used to automatically scan and recognize all of the contributed articles in this book for entry into a computerized publishing system.

Many computerized systems move information from electronic form onto the printed page. The KDEM allows it to move back not just as an electronic image but in an intelligent form that a computer can understand and process further.

The result is to make the printed page another form of information storage like a floppy disk or tape. Unlike electronic media, however, the printed page can be easily accessed by humans as well, which makes it the medium of choice for both people and machines.

I find it interesting to review the rapidly improving price performance of computer-based products in terms of the products of my own companies. The 1978 KDEM were sold for $120,000, which, adjusted far inflation, is equivalent to $231,000 in 1990 dollars. It had 65,536 bytes of memory and recognized print at about 3 characters per second. In 1990 KCP offered a far superior product for under $5,000.

The 1990 version has 2 to 4 million bytes of memory, can recognize between 30 and 75 characters per second, can recognize a substantially wider range of degraded print, and is far more accurate than the 1978 KDEM. The 1990 version thus has 32 to 64 times as much memory, is 10 to 25 times faster, and is more accurate and versatile than the 1978 version. If we conservatively assume that it provides at least 15 times the performance at 1/46.2 the price, it represents an overall improvement in price-performance of 693 to 1. Since 2^9.4 = 693, KCP has doubled its price-performance 9.4 times in 144 months, which is a doubling of price-performance every 15.3 months. That rate is somewhat better than the computer industry at large, which is generally considered to double its price-performance ratio only every 18 to 24 months.

Speech Recognition

On July 1, 1982, I founded Kurzweil Applied Intelligence, (KAI). KAI’s goal has been to master automatic speech recognition (ASR) technology and to integrate ASR with other AI technologies to solve real-world problems. The long-term goal is to establish ASR as a ubiquitous modality of communication between human and machine. In 1985 we introduced the Kurzweil Voice System (KVS), the first commercial ASR device with a 1,000 word vocabulary. A refined version called KVW, the first to provide a recognition vocabulary of up to 10,000 words, was introduced in 1987.¹

The KAI speech recognizers follow the paradigm described in the previous section. The KVW, for example, has seven different experts, all of which attempt to recognize each spoken word simultaneously. Several of the experts analyze the acoustics (or sound) of the word spoken, and the others are context experts programmed with a knowledge of English word sequences (other languages will follow in the future). The system makes extensive use of user training and adaptation. It learns the phonological patterns (that is, the frequency patterns) and the phonetic sequences (the dialect) of each user. It also includes a user syntax expert that learns the characteristic word-sequence patterns used by each speaker.

The KVW consists of specialized electronics, an industry standard personal computer and software. Before a user can begin productive dictation with the system, he needs to enroll, which involves speaking a sample of words to the machine to provide it with initial phonological and phonetic models. Once that has been accomplished, the user can dictate to the machine and watch each word appear on the screen within a fraction of a second after speaking it. The 1990 models require the user to provide a brief pause between words, although later models are expected to accept continuous speech.

The KVW actually improves its performance as the system adapts to (that is, learns about) the user’s pronunciation patterns and syntax. This adaptation continues indefinitely. In addition to displaying each recognized word, the system also displays its second through sixth choices. If the KVW makes a mistake, one of these alternate words is very often the correct choice. Thus, errors are typically corrected by the user saying “Take two” or “Take three,” which replaces the word originally displayed in the text with the appropriate alternate word.

KVW speech-recognition technology has been integrated with a variety of applications. One version includes a full-function word-processor with the capability of entering text as well as issuing all editing and formatting commands by voice. Several sections of this book were written by voice using this version of the KVW. Other

Courtesy of Kurzweil Applied Intelligence

A doctor dictates medical reports to VoiceRad.

versions of the KVW are integrated with knowledge-based systems that have expertise in the types of reports created in different professions.

Courtesy of Kurzweil Applied Intelligence

A doctor dictates medical reports to VoiceRad.

For example, VoiceRad integrates KAI’s speech recognition technology with knowledge of radiology reporting, allowing a radiologist to quickly dictate the results of an examination for instantaneous transcription. As with the word-processor version of the KVW, the radiologist can dictate a report word by word.

In addition, the system can automatically generate predefined sections of text based on its knowledge of radiology reporting. VoiceEM is a similar system for emergency medicine. A variety of similar systems have been developed for medicine and other disciplines. This approach combines the productivity gains of ASR-based dictation with those of a built-in domain-specific knowledge base. These products mark the first time that a commercially available large vocabulary ASR product has been used to create written text by voice in other than experimental situations.

KAI also has a major commitment to applying technology for the handicapped. Versions of KVS and KVW technology provide means for text creation and computer and environmental control for quadriplegic and other hand-impaired individuals. A long-term goal of the company is to develop a sensory aid for the deaf that would provide a real-time display of what someone is saying on the phone and in person.

The company’s long-term objectives are two-fold. First, it intends to continue strengthening its core speech-recognition technology, to move toward the Holy Grail of combining large vocabulary ASR with continuous-speech capability and minimal requirements for training the system for each user. Second, it intends to integrate ASR with a variety of applications, particularly those emphasizing other AI technologies. Ultimately, our goal is to establish voice communication as a desirable and widely used means of communicating with machine intelligence.

The Electronic Music Revolution

I founded Kurzweil Music Systems (KMS), also on July 1,1982. The inspiration for starting KMS came from two sources. One was my lifelong interest in music, along with a nearly lifelong interest in computers. My father, a noted conductor and concert pianist, had told me shortly before his death in 1970 that I would combine these two interests one day, although he was not sure how. The other and more immediate genesis of KMS was a conversation I had with Stevie Wonder, who had been a user of the Kurzweil Reading Machine from its inception. While showing me some new musical instruments he had recently acquired, Steve noted that two worlds of musical instruments–the acoustic and the electronic–had developed with no bridge existing between them.

On the one hand, acoustic instruments such as the piano, violin, and guitar provided the musical sounds that were still the sounds of choice for most of the world’s musicians. While these acoustic sounds were rich, complex, and musically satisfying, only limited means were available for controlling or even playing these sounds.

For one thing, once a piano key was struck, there was no further ability to shape the note other than to terminate it: the initial velocity of the key strike was the only means for modifying piano sounds. Second, most instruments could only play one note at a time.

Third, there were no ways to layer sounds, that is, play the sounds of different instruments simultaneously. Even if you had the skills to play both a piano and a guitar, for example, you could hardly play both at the same time. Even two musicians would find playing the same chords on a piano and guitar almost impassible. In any case, very few musicians, no matter how accomplished, could play more than a very few instruments, as each one requires substantially different playing techniques. Since the playing methods themselves were linked to the physics of each acoustic instrument, many instruments required a high level of finger dexterity. If a composer had a multi-instrumental arrangement in mind, he had no way of even hearing what the piece sounded like without assembling a large group of musicians. Then making changes to the composition required laborious modification of written scores and additional rehearsal. I recall my father’s lamenting the same difficulties.

Steve pointed out that on the other hand there existed the electronic world of music in which most of the above limitations are overcome. Using just one type of playing skill (e.g., a piano-keyboard technique), one can activate and control all available electronic sounds. A wide variety of techniques exist for modifying many aspects of the sounds themselves prior to as well as during performance (these techniques have expanded greatly since 1982). One can layer sounds by having each key initiate different sounds simultaneously. Using sequencers, one can play one part of a multi-instrumental composition, then play that part back from memory and play a second part over it, repeating this process indefinitely. However, electronic instruments at that time suffered from a major drawback, namely the sounds themselves. While they had found an important role in both popular and classical music, synthetic sounds were “thin,” had relatively limited diversity, and did not include any of the desirable acoustic sounds.

Steve asked whether it would be possible to combine these two worlds of music to create in a single instrument the capabilities of both. Such an instrument could produce music that neither world of instruments alone could create. Accomplishing this

Courtesy of Kurzweil Music Systems

The Kurzweil 250 Computer-Based Synthesizer.

Courtesy of Kurzweil Music Systems

A musician plays the Kurzweil 250 using the keyboard and an electronic drum controller.

would, for example, enable musicians to play a guitar and a piano at the same time. We could take acoustic sounds and modify them to accomplish a wide variety of artistic purposes.

A musician could play a multi-instrumental composition (such as an entire orchestra) by himself using real acoustic (as well as electronic) sounds. A musician could play a violin or any other instrument polyphonically (playing more than one note at a time). One could play sounds of any instrument without having to learn the playing techniques of each. One could even create new sounds that were based on acoustic sounds, and thus shared their complexity, but moved beyond them to a whole new class of timbres with substantial musical value.

This vision defined the goal of KMS. In June of 1983 we demonstrated an engineering prototype of the Kurzweil 250 (K250j, and we introduced it commercially in 1984² The K250 was considered the first electronic musical instrument to successfully emulate the sounds of a grand piano and a wide variety of other instruments: orchestral string instruments (violin, viola, etc.). guitar, human voice, brass instruments, drums, and many others. In listening tests we found that listeners, including professional musicians, were essentially unable to tell the K250 “grand piano” sound apart from that of a real $40,000 concert grand piano. A 12-track sequencer, sound layering, and extensive sound modification facilities provide a full complement of artistic control methods.

The essence of K250 technology lies in its sound models. These data structures, contained in read-only memory within the instrument, define the essential patterns of each instrument voice. We needed to create a signal-processing model of an instrument that will respond to changes in pitch, loudness, and the passage of time in the same complex ways as the original acoustic instrument.

To create a sound model, the starting point is to record the original instrument using high-quality digital techniques. Surprisingly, just finding the right instruments to record turned out to be a major challenge. We were unable, for example, to find a single concert grand piano with an attractive sound in all registers. Some had a beautiful bass region but a shrill midrange. Others were stunning in the high range, but mediocre otherwise. We ended up recording five different pianos, including the one that Rudolph Serkin plays when he comes to Boston.

When capturing an instrument, we record examples of many different pitches and loudness levels. When a particular key on a piano is struck with varying levels of force, it is not just the loudness level that changes but the entire time-varying spectrum of sound frequencies. All of these digital recordings are fed into our sound analysis computer, and a variety of both automatic and manual techniques are used to shape each instrument model. Part of the process involves a form of painstaking tuning and attention to detail ironically reminiscent of old-world craftsmanship. The automatic aspects of the process deal primarily with the issue of data compression. The original recorded data for even a single instrument would exceed the K250′s memory capacity. Thus, it is necessary to include only the salient information necessary to accurately represent the original sounds.

When the keyboardist strikes the K250′s piano-like keys, special sensors detect the velocity of each key’s motion. (Other KMS keyboards can also detect the time-varying pressure exerted by each finger.) The K250′s computer and specialized electronics extract the relevant information from the appropriate sound models in memory and then compute in real time the waveforms representing the selected instrument sound, pitch, and loudness for each note. The varied control features, such as sequencing, layering, and sound modification, are provided by software routines stored in the unit’s memory.

In evolving our instruments at KMS, we have followed two paths. First, the K250 has evolved into a comprehensive system for creating complex musical works. It is essentially a digital recording and production studio in an instrument, and it has become a standard for the creation of movie and television soundtracks and professional recordings. KMS has also moved to bring down the cost of its sound-modeling technology. Its K1000 series, for example, is a line of relatively inexpensive products that provide the same quality and diversity of sounds as the K250.

There is an historic trend taking place in the musical instrument industry away from acoustic and mechanical technology and toward digital electronic technology. There are two reasons for this. First, the price-performance of acoustic technology is rapidly deteriorating because of the craftsmanship and labor-intensive nature of its manufacturing processes. A grand piano, for example, has over 10,000 mostly hand-crafted moving parts. The price of the average piano has increased by over 250 percent since 1970. At the same time, it is widely acknowledged that the quality of new pianos is diminishing. On the other hand, the price-performance of digital electronics is, of course, rapidly improving. Furthermore, it is now possible for an electronic instrument to provide the same sound quality as an acoustic instrument, with substantially greater functionality. For these reasons, electronic keyboard instruments have gone from 9.5 percent of the American market for keyboard instruments in 1980 to 55.2 percent in 1986 (according to the American Music Conference). It is my strong belief that this trend will continue until the market is virtually entirely electronic. Our long-term goal at KMS is to continue to provide leadership for this emerging worldwide industry of intelligent digital music technology.

A Final Note

1 have tried to select projects that make it possible to build strong companies while meeting social and cultural goals that are important to me and others. I believe, for example, that there is a good match between the capabilities of computer science and the needs of the handicapped. It has been a personal goal of mine to apply AI technologies to help overcome the handicaps associated with major physical and sensory disabilities. I believe the potential exists in the next couple of decades to largely overcome these major handicaps. As amplifiers of human thought, computers have great potential to assist human expression, improve productivity, and expand creativity for all of us, in all areas of work and play. I hope to play a role in constructively harnessing this potential.

All of the projects described above have been highly interdisciplinary efforts and have required the dedication and talents of many brilliant individuals in a broad range of fields. Inventing today is very much a team effort and its success is a function of the quality of the individual members of the team as well as the quality of the group’s communication. As Norbert Wiener pointed out in Cybernetics, scientists and engineers with different areas of expertise often use entirely different technical vocabularies to refer to the same phenomena.

Creating an environment in which a team of linguists, speech scientists, signal-processing experts, VLSI designers, and other specializts can understand each other’s terminology and effectively work together (as was required, for example, in the efforts to develop the speech recognition technology described above) is at least as challenging as the development of the technology itself. Once developed, the technology (and the technologists) must be further integrated into the equally well-developed disciplines of manufacturing, marketing, finance, and the other management skills of a modern corporation.

It is always exciting to see (or hear) a new product, to experience the realization of a vision after years of hard collaborative work. Perhaps my greatest pleasure has been the opportunity to share in the creative process with the many outstanding men and women who have contributed to these endeavors.

Notes

1. Raymond Kurzweil, “The Technology of the Kurzweil Voice Writer,” BYTE, March 1986.

A computer image. Photo by Lou Jones

Let us review a few of the fundamental issues underlying the age of intelligent machines.

Will Machines Reach Human Levels of Intelligence?

As I noted in the last section of the previous chapter, the strengths of today’s machine intelligence are quite different from those of human intelligence and in many ways complement it. Once we have defined the transformations and methods underlying intelligent processes, a computer can carry them out tirelessly and at great speed. It can call upon a huge and extremely reliable memory and keep track of billions of facts and their relationships. Human intelligence, on the other hand, though weak at mastering facts, still excels at turning information into knowledge. The ability to recognize, understand, and manipulate the subtle networks of abstraction inherent in knowledge continues to set human intelligence apart.

Yet computers are clearly advancing in these skills. Within narrow domains–diagnosing certain classes of disease, performing financial judgments, and many other specialized tasks–computers already rival human experts. During the 1980s expert systems went from research experiments to commercially viable tools that are relied upon daily to perform important jobs. Computers have also begun in recent years to master the pattern-recognition tasks inherent in vision and hearing.

Though not yet up to human standards, pattern-recognition technology is sufficiently advanced to perform a wide variety of practical tasks. It is difficult to estimate when these capabilities will reach human levels, but there does not appear to be any fundamental barrier to achieving such levels. Undoubtedly, computers will achieve such levels gradually; no bell will ring when it happens.

What is clear is that by the time computers achieve human levels of performance in those areas of traditional human strength, they will also have greatly enhanced their areas of traditional superiority. (Not all experts agree with this. Doug Hofstadter, for example, speculates in Godel, Escher, Bach that a future “actually intelligent” machine may not be able to do fast accurate arithmetic, because it will get distracted and confused by the concepts triggered by the numbers-a dubious-proposition in my view.¹) Once a computer can read and understand what it is reading, there is no reason why it should not read everything ever written (encyclopedias, reference works, books, journals and magazines, data bases, etc.) and thus master all knowledge.

As Norbert Wiener has pointed out, no human being has had a complete mastery of human knowledge for the past couple centuries (and it is doubtful in my view that anyone has ever had such mastery). Even mere human levels of intelligence combined with a thorough mastery of all knowledge would give computers unique intellectual skills. Combine these attributes with computers’ traditional strengths of speed, tireless operation, prodigious and unfailing memory, and extremely rapid communication, and the result will be formidable. We are, of course, not yet on the threshold of this vision. This early phase of the age of intelligent machines is providing us with obedient servants that are not yet intelligent enough to question our demands of them.

Minsky points out that we have trouble imagining machines achieving the capabilities we have because of a deficiency in our concept of a machine.^’2 The human race first encountered machines (of its own creation) as devices with a few dozen, and in some cases a few hundred, active parts. Today, our computerized machines have millions of active components, yet our concept of a machine as a relatively inflexible device with only a handful of behavioral options has not changed. By the end of this century chips with over a billion components are anticipated, and we will enter an era of machines with many billions of components.

Clearly, the subtlety and intelligence of the behavior of machines at those different levels of complexity are quite different. Emulating human levels of performance will require trillions, perhaps thousands of trillions, of components. At current rates of progress, we shall achieve such levels of complexity early in the next century. Human-level intelligence will not automatically follow, but reasonable extrapolations of the rate of progress of machine intelligence in a broad variety of skills in pattern recognition, fine motor coordination, decision making, and knowledge acquisition leads to the conclusion that there is no fundamental barrier to the AI field’s ultimately achieving this objective.

Can a Machine Think?

The question sounds innocuous enough, but our approach to it rests on the meanings we ascribe to the terms “machine” and “think.” Consider first the question of whether or not a human being is a machine. A human being is certainly not like the early human-made machines, with only a handful of parts. Yet are we fundamentally different from a machine, one with, say, a few trillion parts? After all, our bodies and brains are presumably subject to the same natural laws as our machines.

As I stated earlier, this is not an easy question, and several thousand years of philosophical debate have failed to answer it. If we assume that the answer to this question is no (humans are not fundamentally different from machines), then we have answered the original question. We presumably think, and if indeed we are machines, then we must conclude that machines can think. If, on the other hand, we assume that we are in some way fundamentally different from a machine, then our answer depends on our definition of the word “think.”³

First, let us assume a behavioral definition, that is, a definition of thinking based on outwardly observable behavior. Under this definition, a machine should be considered to think if it appears to engage in intelligent behavior. This, incidentally, appears to be the definition used by the children I interviewed (see the section “Naive Experts” in chapter 2). Now the answer is simply a matter of the level of performance we expect. If we accept levels of performance in specific areas that would be considered intelligent if performed by human beings, then we have achieved intelligent behavior in our machines already, and thus we can conclude (as did the children I talked with) that today’s computers are thinking.

If, on the other hand, we expect an overall level of cognitive ability comparable to the full range of human intelligence, then today’s computers cannot be regarded as thinking. If one accepts my conclusion above that computers will eventually achieve human levels of intellectual ability, then we can conclude that it is inherently possible for a machine to think, but that machines on earth have not yet started to do so.

If one accepts instead an intuitive definition of thinking, that is, an entity is considered to be thinking if it “seems” to be thinking, then responses will vary widely with the person assessing the “seeming.” The children I spoke to felt that computers seemed to think, but many adults would disagree. For myself, I would say that computers do not yet seem to be thinking most of the time, although occasionally a clever leap of insight by a computer program I am interacting with will make it seem, just for a moment, that thinking is taking place.

Now let us consider the most difficult approach. If we define thinking to involve conscious intentionality, then we may not be in a position to answer the question at all. I know that I am conscious, so I know that I think (hence Descartes’s famous dictum “I think, therefore I am”). I assume that other people think (lest I go mad), but this assumption appears to be built in (what philosophers would call a priori knowledge), rather than based on my observations of the behavior of other people. I can imagine machines that can understand and respond to people and situations with the same apparent intelligence as real people (see some of the scenarios above).

The behavior of such machines would be indistinguishable from that of people; they would pass any behavioral test of intelligence, including the Turing test. Are these machines conscious? Do they have genuine intentionality or free will? Or are they just following their programs? Is there a distinction to be made between conscious free will and just following a program? Is this a distinction with a difference? Here we arrive once again at the crux of a philosophical issue that has been debated for several thousand years.

Some observers, such as Minsky and Dennett, maintain that consciousness is indeed an observable and measurable facet of behavior, that we can imagine a test that could in theory determine whether or not an entity is conscious. Personally, I prefer a more subjective concept of consciousness, the idea that consciousness is a reality appreciated only by its possessor. Or perhaps I should say that consciousness is the possessor of the intelligence, rather than the other way around. If this is confusing, then you are beginning to appreciate why philosophy has always been so difficult.

If we assume a concept of thinking based on consciousness and hold that consciousness is detectable in some way, then one has only to carry out the appropriate experiment and the answer will be at hand. (If someone does this, let me know.) If, on the other hand, one accepts a subjective view of consciousness, then only the machine itself could know if it is conscious and thus thinking (assuming it can truly know anything). We could, of course, ask the machine if it is conscious, but we would not be protected from the possibility of the machine having been programmed to lie. (The philosopher Michael Serwen once proposed building an intelligent machine that could not lie and then simply asking it if it was conscious.)

One remaining approach to this question comes to us from quantum mechanics. In perhaps its most puzzling implication, quantum mechanics actually ascribes a physical reality to consciousness. Quantum theory states that a particle cannot have both a precise location and a precise velocity. If we measure its velocity precisely, then its location becomes inherently imprecise.

In other words, its location becomes a probability cloud of possible locations. The reverse is also true: measuring its precise location renders its velocity imprecise. It is important to understand exactly what quantum mechanics is trying to say. It is not saying that there is an underlying reality of an exact location and velocity and that we are simply unable to measure them both precisely. It is literally saying that if a conscious being measures the velocity of a particle, it actually renders the reality of the location of that particle imprecise.

Quantum mechanics is addressing not simply limitations in observation but the impact of conscious observation on the underlying reality of what is observed. Thus, conscious observation actually changes a property of a particle. Observation of the same particle by a machine that was not conscious would not have the same effect. If this seems strange to you, you are in good company. Einstein found it absurd and rejected it.⁴ Quantum mechanics is consistent with a philosophical tradition that ascribes fundamental reality to knowledge, as opposed to knowledge simply being a reflection of some other fundamental reality.⁵

Quantum mechanics is more than just a philosophical viewpoint, however: its predictions have been consistently confirmed. Almost any electronic device of the past 20 years demonstrates its principles, since the transistor is an embodiment of the paradoxical predictions of quantum mechanics. Quantum mechanics is the only theory in physics to ascribe a specific role to consciousness beyond simply saying that consciousness is what may happen to matter that evolves to high levels of intelligence according to physical laws.

If one accepts its notions fully, then quantum mechanics may imply a way to physically detect consciousness. I would counsel caution, however, to any who would be builder of a consciousness detector based on these principles. It might be upsetting to point a quantum-mechanical consciousness detector at ourselves and discover that we are not really conscious after all.

As a final note on quantum mechanics let me provide a good illustration of the central role it ascribes to consciousness. According to quantum mechanics, observing the velocity of a particle affects not only the preciseness of its location but also affects the preciseness of the location of certain types of “sister” particles that may have emerged from the same particle interaction that produced the particle whose velocity we just observed.

For example, if an interaction produces a pair of particles that emerge in opposite directions and if we subsequently observe the velocity of one of the particles, we will instantly affect the preciseness of the position of both that particle and its sister, which may be millions of miles away. This would appear to contradict a fundamental tenet of relativity: that effects cannot be transmitted faster than the speed of light. This paradox is currently under study.⁶

What Impact Will the Age of Intelligent Machines Have on Society?

When computers were first invented in the mid 1940s, they were generally regarded as curiosities, though possibly of value to mathematics and a few engineering disciplines. Their value to science, business, and other disciplines soon became apparent, and exploration of their practical applications soon began.

Today, almost a half-century later, computers are ubiquitous and highly integrated into virtually all of society’s institutions. If a law were passed banning all computers (and in the doubtful event that such legislation were adhered to), society would surely collapse. The orderly functioning of both government and business would break down in chaos. We are already highly dependent on these “amplifiers of human thought,” as Ed Feigenbaum calls them.

As the intelligence of our machines improves and broadens, computer intelligence will become increasingly integrated into our decision-making, our economy, our work, our learning, our ability to communicate, and our life styles. They will be a driving force in shaping our future world. But the driving force in the growth of machine intelligence will continue to be human intelligence, at least for the next half century.

A Final Note

When I was a boy, I had a penchant for collecting magic tricks and was known to give magic shows for friends and family. I took pleasure in the delight of my audience in observing apparently impossible phenomena. It became apparent to me that organizing ordinary methods in just the right sequence could give rise to striking results that went beyond the methods I started with. I also realized that revealing these methods would cause the magic to disappear and leave only the ordinary methods.

As I grew older, I discovered a more powerful form of magic: the computer. Again, by organizing ordinary methods in just the right sequences (that is, with the right algorithms), I could once again cause delight. Only the delight caused by this more grown-up magic was more profound. Computerized systems that help overcome the handicaps of the disabled or provide greater expressiveness and productivity for all of us provide measures of delight more lasting than the magic tricks of childhood.

The sequences of 1s and 0s that capture the designs and algorithms of our computers embody our future knowledge and wealth. And unlike more ordinary magic, any revelation of the methods underlying our computer magic does not tarnish its enchantment.

Notes

1. See pp. 677-678 of Douglas Hofstadter’s Godel, Escher, Bach: An Eterna! Golden Braid (New York: Basic Books, 1979) for a fuller account of his concept of potential computer weaknesses.

2. Marvin Minsky, Society of Mind, pp. 186, 288.

3. The general reader will find pertinent to the topic Paul M. Churchland’s philosophical and scientific examination throughout Matter and Consciousness.

4. See Einstein’s letters of August 9, 1939, and December 22, 1950, to E. Schrodinger, in K. Przibram, ed., Letters on Wave Mechanics, pp. 35-36 and 39-40.

5. Admittedly, some disavow the applicability of subatomic metaphors to any other aspect of life. See Paul G. Hewitt, Conceptual Physics, 2nd ed., pp. 486-487.

6. A dense and pertinent discussion of the sister particle paradox may be found in Abner Himony, “Events and Processes in the Quantum World,” in R. Penrose and C. J.Isham, eds., Quantum Concepts in Space and Time, pp. 182-196.

Originally Published 1990

This is the year 1300. Brother Giorgio, scholar-monk, has the task of making a map of Australia, a big island just south of India. Maps must record what is known about the places they represent, and Giorgio has been told about a strange Australian animal, rat-like, but much bigger, with a long thick tail and a pouch. He draws it, and it comes out like this:

A year later a world traveler is visiting Giorgio’s monastery, and he tells our cartographer that he has the animal wrong. For one thing, it isn’t carrying a pouch; the pouch is actually part of its belly. (“Mercy!” says Giorgio.) For another, it doesn’t walk on all fours like a rat but on its hind legs, which are much bigger than its front legs. Giorgio redraws his picture:

But the tail rests on the ground. Giorgio tries once more. The traveler screws up his face in concentration, his eyes closed. I don’t think that’s quite right, he finally says, but I guess it’s close enough.

The year is 1987. AARON, a computer program, has the task of drawing some people in a botanical garden-not just making a copy of an existing drawing, you understand, but generating as many unique drawings on this theme as may be required of it. What does it have to know in order to accomplish such a task? How could AARON, the program, get written at all?

The problem will seem a lot less mystifying, though not necessarily less difficult, if we think of these two stories as having a lot in common. AARON has never seen a person or walked through a botanical garden. Giorgio has never seen a kangaroo. Since most of us today get mast of our knowledge of the world indirectly and heavily wrapped in the understanding of other people from grade school teachers to television anchor persons, it should come as no surprise that a computer program doesn’t have to experience the world itself in order to know about it.

How did Giorgio know about kangaroos before the visitor started to refine his knowledge? He had been told that the animal was rat-like, but how much good would that have done him if he had never seen a rat? For people, the acquisition of knowledge is cumulative, as it clearly has to be. Nothing is ever understood from scratch. Even the newborn babe has a good deal of knowledge “hard-wired” before it starts. And when we tell each other about the world, it isn’t practical or even possible to give a full description of something without referring to same thing else. That’s as true for computer programs as it is far people. There is an important difference, though. For people, knowledge must eventually refer back to experience, and people experience the world with their bodies, their brains, their reproductive systems, which computers don’t have.

Photo by Lou Jones www.fotojones.com

Harold Cohen, computer artist.

Photo by George Johnson

Athletes, a hand-colored, computer-generated drawing by Harold Cohen.

Photo by George Johnson

From the Bathers series of hand-colored, computer-generated drawings by Harold Cohen.

With this in mind, we might guess that AARON’s knowledge of the world and the way AARON uses its knowledge are not likely to be exactly the same as the way we use what we have. Like us, its knowledge has been acquired cumulatively. Once it understands the concept of a leaf cluster, for example, it can make use of that knowledge whenever it needs it. But we can see what plants look like, and AARON can’t.

We don’t need to understand the principles that govern plant growth in order to recognize and record the difference between a cactus and a willow tree in a drawing. AARON can only proceed by way of principles that we don’t necessarily have. Plants exist for AARON in terms of their size, the thickness of limbs with respect to height, the rate at which limbs get thinner with respect to spreading, the degree of branching, the angular spread where branching occurs, and so on. Similar principles hold for the formation of leaves and leaf clusters.

By manipulating these factors, AARON is able to generate a wide range of plant types and will never draw quite the same plant twice, even when it draws a number of plants recognizably of the same type. Interestingly enough, the way AARON accesses its knowledge of plant structure is itself quite treelike. It begins the generation of each new example with a general model and then branches from it. “Tree” is expanded into “big-tree/small-tree/shrub/grass/flower,” “big tree” is expanded into “oak/willow/avocado/wide-leaf” (the names are not intended literally), and so on, until each unique representation might be thought of as a single “leaf,” the termination of a single path on a hugely proliferating “tree” of possibilities.

Obviously, AARON has to have similar structural knowledge about the human figure, only more of it. In part, this extra knowledge is demanded by AARON’s audience, which knows about bodies from the inside and is more fussy about representations of the body than it is about representations of trees. In part, more knowledge is required to cope with the fact that bodies move around. But it isn’t only a question of needing more knowledge; there are three different kinds of knowledge required-different, that is, in needing to be represented in the program in different ways.

First, AARON must obviously know what the body consists of, what the different parts are, and how big they are in relation to each other. Then it has to know how the parts of the body are articulated: what the type and range of movement is at each joint. Finally, because a coherently moving body is not merely a collection of independently moving parts, AARON has to know something about how body movements are coordinated: what the body has to do to keep its balance, far example. Conceptually, this isn’t as difficult as it may seem, at least for standing positions with one or both feet on the ground. It’s just a matter of keeping the center of gravity over the base and, where necessary, using the arms for fine tuning.

We started by asking what AARON would need to know to carry out its task. What I’ve outlined here constitutes an important part of that necessary knowledge, but not the whole of it. What else is necessary? Lets go back to Giorgio. Has it struck you that whatever Giorgio eventually knew about the relative sizes of the kangaroo’s parts and its posture, he had been told nothing at all about its appearance? Yet his drawings somehow contrived to look sort of like the animal he thought he was representing, just as AARON’s trees and people contrive to look like real trees and real people.

That may not seem very puzzling with respect to Giorgio. In fact, it may seem so un-puzzling that you wonder why I raise the issue. Obviously, Giorgio simply knew how to draw. I suspect that most people who don’t draw think of drawing as a simple process of copying what’s in front of them. Actually, it’s a much more complicated process of regenerating what we know about what’s in front of us or even about what is not in front of us: Giorgio’s kangaroo, for example. There’s nothing simple about that regeneration process, though the fact that we can do it without having to think much about it may make it seem so. It is only in trying to teach a computer program the same skills that we begin to see how enormously complex a process is involved.

Photo by Linda Winters

A hand-colored, computer-generated drawing of figures and trees with rocks in the foreground, by Harold Cohen.

How do humans learn to draw? To some degree, obviously, we learn about drawing by looking at other peoples’ drawings. That’s why we are able to identify styles in art, and why most of the drawings coming out of Giorgio’s monastery would have had a great deal in common and be distinguishably different from, say, the drawings made in a Zen Buddhist temple in Japan. At the same time, all children make very much the same drawings at any one stage of cognitive development without learning from each other or from adults.

Photo by Becky Cohen

Black and White Drawing, a computer-generated drawing of figures and trees, by Harold Cohen.

They don’t need to be told to use closed forms in their drawings to stand for solid objects, for example. That equivalent is universal; all cultures have used closed forms to stand for solid objects. In short, knowledge of drawing has two components. Giorgio learned about style, about what was culturally acceptable and what was not, from his peers. But before cultural considerations ever arise, drawing is closely coupled to seeing-so closely coupled that we might guess all major visual modes of representation in human history have sprung directly from the nature of the cognitive system. So Giorgio never had to be told how to draw or how to read drawings. He could see.

He had to be told about kangaroos, not about how to draw kangaroos. Knowledge of drawing isn’t object specific; if Giorgio could draw a kangaroo, he could also draw an elephant or a castle or an angel of the Annunciation. If one can draw, then anything that can be described in structural terms can be represented in visual terms. That generality suggests that rather than thinking of knowledge of drawing as just one more chunk of knowledge, we should think of it as a sort of filter through which object-specific knowledge passes on its way from the mind to the drawing.

Like Giorgio, AARON had to be told about things of the world. Unlike Giorgio in having no hard-wired cognitive system to provide a built-in knowledge of drawing, it had to be taught how to draw as well, given enough of a cognitive structure (the filter just referred to) to guarantee the required generality. If provided with object-specific knowledge, AARON should be able to make drawings of those objects without being given any additional knowledge of drawing.

AARON’s cognitive filter has three stages, of which the first two correspond roughly to the kinds of knowledge described above in relation to the human figure: knowledge of parts, articulation, and coordination. The third stage generates the appearance of the thing being drawn. Neither of the first two stages results in anything being drawn for the viewer, though they are drawn in AARON’s imagination, so to speak, for its own use. First AARON constructs an articulated stick figure, the simplest representation that can embody what it knows about posture and movement. Then around the lines of this stick figure it builds a minimal framework of lines embodying in greater detail what it knows about the dimensions of the different parts. This framework doesn’t represent the surface of the object. In the case of a figure, the lines actually correspond quite closely to musculature, although that is not their essential function. They are there to function as a sort of core around which the final stage will generate the visible results. Quite simply, AARON draws around the core figure it has “imagined.” Well, no, not quite so simply. If you look at one of its drawings, it should be clear that the final embodying stage must be more complicated than I have said if only because AARON apparently draws hands and leaves with much greater attention than it affords to thighs and tree trunks.

AARON’s embodying procedures are not like the preliminary edge-finding routines of computer vision, which respond to changes in light intensity without regard to what caused them. AARON is concerned with what it is drawing and continuously modifies the performance of this final stage with respect to how much knowledge has already been represented in the core figure. The greater the level of detail already present, the more AARON relies upon it and the closer to the core the embodying line is drawn. Also, greater detail implies more rapidly changing line directions in the core, and AARON ensures a sufficiently responsive embodying line by sampling its relation to the core more frequently.

Nothing has been said here about how AARON’s knowledge of the world is stored internally, about how its knowledge of drawing is actually implemented, or about its knowledge of composition, occlusion, and perspective. AARON’s success as a program stands or falls on the quality of the art it makes, yet nothing much has been said about art and nothing at all about the acculturated knowledge of style, for which its programmer, like Giorgio’s monastic peers, must admit or claim responsibility. All the same, there are interesting conclusions to be drawn from this abbreviated account. It should be evident, for example, that the knowledge that goes into the making of a visual representation, even a simple one, is quite diverse. I doubt that one could build a program capable of manipulating that knowledge and exhibiting the generality and flexibility of the human cognitive system other than by fashioning the program as an equivalent, artificial cognitive system. If nothing much has been said about art, it is because remarkably little of the program has anything to do with art: it constitutes a cognitive model of a reasonably general kind, and I even suspect that it could be adapted to other modes without too much distortion. But the lack of art specificity isn’t as puzzling as it may seem at first glance. The principal difference between artists and non-artists is not a cognitive difference. It is simply that artists make art and non-artists don’t.

Excerpted with permission from The Age of Intelligent Machines, by Raymond Kurzweil (C) 1990 Massachusetts Institute of Technology

MIT Press