Soul of a New Machine

February 21, 2001 by James Daly

Business 2.0 editor James Daly interviews Raymond Kurzweil on what happens when machines become conscious.

Originally published January 1, 1999 at Published on February 22, 2001.

In your book, you predict that by 2019 the average $1,000 PC will match the power of the human brain. Should we be excited by that, or scared?

It’s simply a large challenge. And that’s a conservative projection, by the way. At the beginning of the 20th century, we were doubling the power of computation every three years. That dropped to every two years in the ’50s and ’60s. Now we’re doubling it every 12 months and it’s likely to accelerate. This acceleration of computational power won’t automatically give us computers that are as intelligent as the human brain, because the organization of the brain’s resources–the way the matter is connected, its contents–is equally important. Our whole culture and knowledge base also forms a key part of the brain’s intelligence. Yet this advance on the part of our machines provides us with immense and exciting opportunities.

For example?

Think of the Internet as a communications paradigm. It has allowed computational resources to be shared in very dynamic ways. For example, I am sitting in front of my computer here and it’s just flashing a cursor; 99.9 percent of its computational resources are unused. Proposals like Sun Microsystems’ Jeni project, for instance, and others will allow all of those unused computational resources on the Net to be harnessed into highly parallel supercomputers (see “The Power Is Out There,” Premiere, p102). We already have more than the processing power of the human brain on the Net, and soon we’ll be able to call on that power through computational–sharing proposals. You’ll be able to apply the computational ability of the human brain from your PC simply by pulling in power from other computers that aren’t being used at the moment. What does that do? Well, suddenly the harnessing of this vast computer intelligence is a much more important component of an organization’s success.

Let’s look ahead. How will a company need to evolve?

We’re seeing a trend toward what I call a “virtual” company. Increasingly, companies will be associations of individuals who are geographically separate but with a business relationship among each other. People won’t necessarily belong to one company. And you will have flexible work groups that are not at all limited by geographic location. Think about this: Most of the wealth being created today is in the form of knowledge of products with no material form. So increasingly, natural resources are becoming less and less important. Mental resources, the ability to create knowledge, to master knowledge, to control information and have access to information resources, is going to be the way that wealth is created.

What is the role of the big, dominant players of today–Microsoft, Yahoo!, Intel?

An organization could still be large, as a sort of a pool of capital and resources, but it has to really provide a kind of dynamic, entrepreneurial environment within its organization to remain viable. Microsoft already does that well. It gives autonomy to smaller groups. For example, at Word, the developer group is not a lot bigger than little companies. Basically, Microsoft emphasizes having small groups with highly talented people, giving them a lot of autonomy. Large organizations, in and of themselves, are not going to remain viable. They will increasingly represent just an umbrella concentration of information, in the form of companies of all different sizes within.

Sounds like some severe growing pains ahead.

That is what is behind this sort of international financial crisis. The business model that certain countries have been relying on doesn’t work. Japan, for instance, is very manufacturing oriented. Its sophisticated manufacturing is the epitome of the first Industrial Revolution. It’s a business model built on buying inexpensive raw materials and converting them into manufactured products, without an emphasis on the kind of intellectual property creation that is epitomized by Silicon Valley. In Silicon Valley over the past 10 years, the new wealth in terms of market capitalization of companies is about a trillion dollars. That’s not play money. That’s real money. It’s real wealth. I’m not saying there is no innovation in Japan, but the heart of the model is not innovation. The Japanese don’t have Silicon Valley. They don’t have that kind of culture. Their economy and culture don’t reward risk-taking and change.

How will the role of money change in the future?

Money is a wonderful abstraction. It’s really a testament to the human species’ ability to give reality to abstract concepts-the concept of value. Because if visitors from another planet were to come here, they would really have to understand some very deep, abstract notions to recognize this reality of money. Yet it’s obviously a very powerful thing. Everybody respects it. People can have very different political views, ideological views, religious views, and disagree enormously on any subject you might mention, yet everyone respects money. It’s a pretty remarkable phenomenon.

If you look at the trends of what people are paying money for and what money represents, it’s increasingly representing the value of information and knowledge. This goes back to my earlier point. Today, knowledge and information really represent technology and its power over our environments. Increasingly, when people buy products they are paying for the knowledge content of those products, which is embedded in subtle ways. So that which is valuable, that for which we will exchange money, will simply represent the power of knowledge and information.

Let’s get back to the point you raised earlier about machine intelligence. Can something create another thing more intelligent than itself?

There’s no reason why not. During the 21st century, the cutting edge of the creation of intelligent machines will move from humans designing intelligent machines to the more intelligent machines creating their own next generation. But this is not an alien invasion of intelligent creatures. We’ll become smarter by merging with our computers. Machines are already a pretty intimate part of our civilization. If all the computers stopped functioning, society would grind to a halt.

Today, the stock market is already moving to neural networks. I’m not just talking about program trading, which involves fairly simple formulas created by human beings. Today, about 5 percent of the trades are done that way. If it was 2 percent last year and 0.5 percent the year before, at current trends, by the middle of the next decade, the majority of stock trading would be carried out by machine intelligence, not using simple program-trading formulas, but really by doing their own sort of evolved intelligence analysis.

If machines reach this level of consciousness, what will constitute being human?

The issue of consciousness has always been a perplexing question. It will become more vexing when we have machines that have the subtlety, complexity, and depth of human response. The complexity of machines today is very limited compared to a human being. Even our most complex computers are still a million times simpler than the human brain. But it is clearly going to change as we go through the first few decades of this next century. Our more advanced computers will be based on the design of human thinking, and we will have machines that appear to be conscious, will claim to be conscious (see “Big Ideas from Small Creatures,” p100). In fact, they will claim to be human and they will be very convincing and ultimately, they will be more intelligent than we are, so we will end up believing them. That’s still not a scientific demonstration that they are conscious. So from a fundamental, philosophical perspective, we can’t prove that these machines are not zombies, that they are conscious entities; but from a political and social perspective, we will believe that, because they will be very convincing.

Is there an inherent difference between carbon-based human thinking and electronic machine thinking?

Right now, we feel that our uniqueness–our “humanness,” if you will–is simply a function of our intelligence. We see evolution as a billion-year drama that led to human intelligence, which is its grandest creation. But ultimately there is going to be no clear boundary between the human world and the machine world. We’ve taken small steps in that direction already. We already have neural implants designed to rewire parts of the brain to suppress seizures. It’s being used with Parkinson’s disease. We have cochlea implants, which replace the early auditory processing circuits of the brain. One of the people I have an active working relationship with is deaf and has a cochlea implant, and I can hold phone conversations with him. Increasingly, we’re going to be putting machines, computational devices, in our brains to augment, replace, supplement, and bypass neural regions of our brains. Conversely, we’re going to be building our computers based on reverse engineering of the human brain. We have already been able to take clusters of neurons and emulate very precisely their input/output characteristics, creating an electronic device that exactly mimics the information-processing characteristics of a cluster of hundreds of neurons. If we could do that with hundreds of neurons, there is no reason why we cannot scale that up to hundreds of billions of neurons. In fact, scaling systems up from hundreds of something to billions of something is the type of task we do all of the time.

So we will be, certainly in the second and third decades of the 21st century, building machines based on our reverse engineering and our scans of the human brain. By 2029, we will be able to fully scan a human brain, all of that information on the interconnections and the wiring. That re-created brain will claim to be the same guy who was scanned. It will claim to be Joe and will claim to be human.

How do we prepare the next generation for these changes?

What we really need to give to our kids is the ability to create knowledge. That doesn’t mean that everyone should become a computer engineer. In fact, creating knowledge and culture in all of its forms–whether it’s music, writing, art, work–it’s becoming increasingly important. Children need to develop certain fundamental skills and the use of language and the ability to think abstractly. Rote learning is not very useful. But we should really foster children to pursue their own talents and develop them in their own unique ways. The only way to create knowledge is to have a passion for something. People should pursue their passions.

Soul of a New Machine reproduced with permission. Copyright (C) 1999 Business 2.0