 |
|
|
|
|
|
| |
Origin >
The Singularity >
Kurzweil’s Law (aka “the law of accelerating returns”)
Permanent link to this article: http://www.kurzweilai.net/meme/frame.html?main=/articles/art0610.html
Printable Version |
|
| |
|
Kurzweil’s Law (aka “the law of accelerating returns”)
In an evolutionary process, positive feedback increases order exponentially. A correlate is that the "returns" of an evolutionary process (such as the speed, cost-effectiveness, or overall "power" of a process) increase exponentially over time -- both for biology and technology. Ray Kurzweil submitted on essay based on that premise to Edge.org in response to John Brockman's question: "What's your law?"
Published on Edge.org
and KurzweilAI.net Jan. 12, 2003
Evolution applies positive feedback in that the more capable
methods resulting from one stage of evolutionary progress are used
to create the next stage. Each epoch of evolution has progressed
more rapidly by building on the products of the previous stage.
Evolution works through indirection: evolution
created humans, humans created technology, humans are now working
with increasingly advanced technology to create new generations
of technology. As a result, the rate of progress of an evolutionary
process increases exponentially over time.
Over time, the "order" of the information embedded
in the evolutionary process (i.e., the measure of how well the information
fits a purpose, which in evolution is survival) increases.
A comment on the nature of order. The concept of the "order"
of information is important here, as it is not the same as the opposite
of disorder. If disorder represents a random sequence of events,
then the opposite of disorder should imply "not random." Information
is a sequence of data that is meaningful in a process, such as the
DNA code of an organism, or the bits in a computer program. Noise,
on the other hand, is a random sequence. Neither noise nor information
is predictable. Noise is inherently unpredictable, but carries
no information. Information, however, is also unpredictable. If
we can predict future data from past data, then that future data
stops being information. We might consider an alternating pattern
("0101010. . . .") to be orderly, but it carries no information
(beyond the first couple of bits).
Thus orderliness does not constitute order because order requires
information. However, order goes beyond mere information. A recording
of radiation levels from space represents information, but if we
double the size of this data file, we have increased the amount
of data, but we have not achieved a deeper level of order.
Order is information that fits a purpose. The measure of
order is the measure of how well the information fits the purpose.
In the evolution of life-forms, the purpose is to survive. In an
evolutionary algorithm (a computer program that simulates evolution
to solve a problem) applied to, say, investing in the stock market,
the purpose is to make money. Simply having more information does
not necessarily result in a better fit. A superior solution for
a purpose may very well involve less data.
The concept of "complexity" is often used to describe the nature
of the information created by an evolutionary process. Complexity
is a close fit to the concept of order that I am describing, but
is also not sufficient. Sometimes, a deeper order – a better fit
to a purpose – is achieved through simplification rather than further
increases in complexity. For example, a new theory that ties together
apparently disparate ideas into one broader more coherent theory
reduces complexity but nonetheless may increase the "order for a
purpose" that I am describing. Indeed, achieving simpler theories
is a driving force in science. Evolution has shown, however, that
the general trend towards greater order does generally result in
greater complexity.
Thus improving a solution to a problem – which may increase or
decrease complexity – increases order. Now that just leaves the
issue of defining the problem. Indeed, the key to an evolution
algorithm (and to biological and technological evolution) is exactly
this: defining the problem.
We may note that this aspect of "Kurzweil's Law" (the law of accelerating
returns) appears to contradict the Second Law of Thermodynamics,
which implies that entropy (randomness in a closed system) cannot
decrease, and, therefore, generally increases. However, the law
of accelerating returns pertains to evolution, and evolution is
not a closed system. It takes place amidst great chaos, and indeed
depends on the disorder in its midst, from which it draws its options
for diversity. And from these options, an evolutionary process
continually prunes its choices to create ever greater order. Even
a crisis, such as the periodic large asteroids that have crashed
into the Earth, although increasing chaos temporarily, end up increasing
– deepening – the order created by an evolutionary process.
A primary reason that evolution – of life-forms
or of technology – speeds up is that it builds on its own increasing
order, with ever more sophisticated means of recording and manipulating
information. Innovations created by evolution encourage and enable
faster evolution. In the case of the evolution of life forms, the
most notable early example is DNA, which provides a recorded and
protected transcription of life's design from which to launch further
experiments. In the case of the evolution of technology, ever improving
human methods of recording information have fostered further technology.
The first computers were designed on paper and assembled by hand.
Today, they are designed on computer workstations with the computers
themselves working out many details of the next generation's design,
and are then produced in fully-automated factories with human guidance
but limited direct intervention.
The evolutionary process of technology seeks
to improve capabilities in an exponential fashion. Innovators seek
to improve things by multiples. Innovation is multiplicative, not
additive. Technology, like any evolutionary process, builds on
itself. This aspect will continue to accelerate when the technology
itself takes full control of its own progression.
We can thus conclude the following with regard
to the evolution of life-forms, and of technology: the law of accelerating
returns as applied to an evolutionary process: An evolutionary process
is not a closed system; therefore, evolution draws upon the chaos
in the larger system in which it takes place for its options for
diversity; and evolution builds on its own increasing order. Therefore,
in an evolutionary process, order increases exponentially.
A correlate of the above observation is that
the "returns" of an evolutionary process (e.g., the speed, cost-effectiveness,
or overall "power" of a process) increase exponentially over time.
We see this in Moore's law, in which each new generation of computer
chip (now spaced about two years apart) provides twice as many components,
each of which operates substantially faster (because of the smaller
distances required for the electrons to travel, and other innovations).
This exponential growth in the power and price-performance of information-based
technologies – now roughly doubling every year – is not limited
to computers, but is true for a wide range of technologies, measured
many different ways.
In another positive feedback loop, as a particular
evolutionary process (e.g., computation) becomes more effective
(e.g., cost effective), greater resources are deployed towards the
further progress of that process. This results in a second level
of exponential growth (i.e., the rate of exponential growth itself
grows exponentially). For example, it took three years to double
the price-performance of computation at the beginning of the twentieth
century, two years around 1950, and is now doubling about once a
year. Not only is each chip doubling in power each year for the
same unit cost, but the number of chips being manufactured is growing
exponentially.
Biological evolution is one such evolutionary
process. Indeed it is the quintessential evolutionary process.
It took place in a completely open system (as opposed to the artificial
constraints in an evolutionary algorithm). Thus many levels of
the system evolved at the same time.
Technological evolution is another such evolutionary
process. Indeed, the emergence of the first technology-creating
species resulted in the new evolutionary process of technology.
Therefore, technological evolution is an outgrowth of – and a continuation
of – biological evolution. Early stages of humanoid created
technology were barely faster than the biological evolution that
created our species. Homo sapiens evolved in a few hundred thousand
years. Early stages of technology – the wheel, fire, stone tools
– took tens of thousands of years to evolve and be widely deployed.
A thousand years ago, a paradigm shift such as the printing press,
took on the order of a century to be widely deployed. Today, major
paradigm shifts, such as cell phones and the world wide web were
widely adopted in only a few years time.
A specific paradigm (a method or approach to
solving a problem, e.g., shrinking transistors on an integrated
circuit as an approach to making more powerful computers) provides
exponential growth until the method exhausts its potential. When
this happens, a paradigm shift (a fundamental change in the approach)
occurs, which enables exponential growth to continue.
Each paradigm follows an "S-curve," which consists
of slow growth (the early phase of exponential growth), followed
by rapid growth (the late, explosive phase of exponential growth),
followed by a leveling off as the particular paradigm matures.
During this third or maturing phase in the
life cycle of a paradigm, pressure builds for the next paradigm
shift, and research dollars are invested to create the next paradigm.
We can see this in the enormous investments being made today in
the next computing paradigm – three-dimensional molecular computing
– despite the fact that we still have at least a decade left for
the paradigm of shrinking transistors on a flat integrated circuit
using photolithography (Moore's Law). Generally, by the time a
paradigm approaches its asymptote (limit) in price-performance,
the next technical paradigm is already working in niche applications.
For example, engineers were shrinking vacuum tubes in the 1950s
to provide greater price-performance for computers, and reached
a point where it was no longer feasible to shrink tubes and maintain
a vacuum. At this point, around 1960, transistors had already achieved
a strong niche market in portable radios.
When a paradigm shift occurs for a particular
type of technology, the process begins a new S-curve.
Thus the acceleration of the overall evolutionary
process proceeds as a sequence of S-curves, and the overall exponential
growth consists of this cascade of S-curves.
The resources underlying the exponential growth
of an evolutionary process are relatively unbounded.
One resource is the (ever-growing) order of
the evolutionary process itself. Each stage of evolution provides
more powerful tools for the next. In biological evolution, the
advent of DNA allowed more powerful and faster evolutionary "experiments."
Later, setting the "designs" of animal body plans during the Cambrian
explosion allowed rapid evolutionary development of other body organs,
such as the brain. Or to take a more recent example, the advent
of computer-assisted design tools allows rapid development of the
next generation of computers.
The other required resource is the "chaos"
of the environment in which the evolutionary process takes place
and which provides the options for further diversity. In biological
evolution, diversity enters the process in the form of mutations
and ever- changing environmental conditions. In technological evolution,
human ingenuity combined with ever-changing market conditions keep
the process of innovation going.
If we apply these principles at the highest
level of evolution on Earth, the first step, the creation of cells,
introduced the paradigm of biology. The subsequent emergence of
DNA provided a digital method to record the results of evolutionary
experiments. Then, the evolution of a species that combined rational
thought with an opposable appendage (the thumb) caused a fundamental
paradigm shift from biology to technology. The upcoming primary
paradigm shift will be from biological thinking to a hybrid combining
biological and nonbiological thinking. This hybrid will include
"biologically inspired" processes resulting from the reverse engineering
of biological brains.
If we examine the timing of these steps, we
see that the process has continuously accelerated. The evolution
of life forms required billions of years for the first steps (e.g.,
primitive cells); later on progress accelerated. During the Cambrian
explosion, major paradigm shifts took only tens of millions of years.
Later on, Humanoids developed over a period of millions of years,
and Homo sapiens over a period of only hundreds of thousands of
years.
With the advent of a technology-creating species,
the exponential pace became too fast for evolution through DNA-guided
protein synthesis and moved on to human-created technology. Technology
goes beyond mere tool making; it is a process of creating ever more
powerful technology using the tools from the previous round of innovation,
and is, thereby, an evolutionary process. As I noted, the first
technological took tens of thousands of years. For people living
in this era, there was little noticeable technological change in
even a thousand years. By 1000 AD, progress was much faster and
a paradigm shift required only a century or two. In the nineteenth
century, we saw more technological change than in the nine centuries
preceding it. Then in the first twenty years of the twentieth century,
we saw more advancement than in all of the nineteenth century.
Now, paradigm shifts occur in only a few years time.
The paradigm shift rate (i.e., the overall
rate of technical progress) is currently doubling (approximately)
every decade; that is, paradigm shift times are halving every decade
(and the rate of acceleration is itself growing exponentially).
So, the technological progress in the twenty-first century will
be equivalent to what would require (in the linear view) on the
order of 200 centuries. In contrast, the twentieth century saw
only about 20 years of progress (again at today's rate of progress)
since we have been speeding up to current rates. So the twenty-first
century will see about a thousand times greater technological change
than its predecessor.
| | |
|
| |
Mind·X Discussion About This Article:
 |
|
|
 |
Re: Ray needs a book on Kolmogorov complexity
|
|
 |
|
That complexity is defined as the length of the shortest descriptive program About correct.
the purpose is simple record compression. Correct, but not entirely. It's like saying that the purpose of phisics is to make a light bulb.
The problem is that the description language is not included in the definition True, but it does not matter because the language has a finite complexity :-). It's just a constant factor.
length of computation is ignored. It's true for the original studies. But other people study the computational requirements now. For example look up Juergen Schmidhuber work.
For Ray (& for me) order is not simply compressability but environmental correspondence/predictiveness, the difference being that it shouldn't depend on the method of encoding/description. It does not except for a constant factor. Like in integration. Integral of x is 0.5*x^2 + CONST. The presence of the indefined CONST does not invalidate the 0.5*x^2 part.
As for predictiveness, it's not so simple. Here is an example sequence: 2, 4, 6, 8. What's next? 10? Wrong. It's 34 because the sequence is
n^4-10n^3+35n^2-48n+24 :-) (c) Schmidhuber
|
|
 |
|
 |
|
|
|
|
Re: Ray needs a book on Kolmogorov complexity
|
|
|
|
An unusually meaningful conversation for this place:)
>> length of computation is ignored.
> It's true for the original studies. But other people study the computational requirements now. For example look up Juergen Schmidhuber work.
Right, I did look at it before, but it didn't make much sense to me,- he concentrates on computation but ignores record compression, I think there should be a common denominator between the two,- indicating overall "cost".
But it's a secondary issue.
>> being that it shouldn't depend on the method of encoding/description.
> It does not except for a constant factor. Like in integration. Integral of x is 0.5*x^2 + CONST. The presence of the indefined CONST does not invalidate the 0.5*x^2 part.
By encoding I mean not only procedural language but also previous compression of the data set, which would reduce compressabilty but should not, theoretically, affect predictability. The simpliest example is digitization,- variables encoded as binary digits will be less compressible than those encoded as decimal digits.
There must be encoding-neutral way to quantify similarity, initially between two inputs, which would also indicate predictability of subsequent inputs.
> As for predictiveness, it's not so simple. Here is an example sequence: 2, 4, 6, 8. What's next? 10? Wrong. It's 34 because the sequence is
n^4-10n^3+35n^2-48n+24 :-) (c) Schmidhuber
It's actually both, but can you quantify which pattern is stronger? :)
Regards!
Boris
|
|
|
|
|
|
|
|
|
Re: Kurzweil’s Law (aka “the law of accelerating returns”)
|
|
|
|
IMHO, Ray Kurzweil makes the same common mistake that most forecasters make. That is, taking a trend or pattern seen in some slice of data to its extreme conclusion. He attempts to avert us from seeing this by explaining that paradigm shifts allow continued evolutionary processes to accelerate on a roughly exponential curve when one views progress with historical perspective. He concludes that this will continue indefinitely and eventually to a singularity. He is bold enough to call this a law - the Law of Accelerating Returns.
What does a singularity even mean in this context? You can't make sense of such a thing - it is a paradox by itself. Laws break down at a singularity. They are undefined by definition. They cannot be described because they cannot ever truly be reached.
But how does he conclude, given the subjective nature of the quantities attached to progress, that this exponential curve is asymptotic? Can he predict with this curve the approximate timeframe the singularity may be reached? Instead could it not continue infinitely without reaching a limit? Or could it be part of an S-curve itself? Might there be other forces at work that will cause this curve to flatten out or even ultimately be limited by a horizontal asymptote? Perhaps the curve when inspected closely is not smooth at all and contains hills and valleys of significant magnitudes. Couldn't there be a hiccup or two in the evolutionary process? I think if you evaluated the past honestly you would see many of these hiccups in history. How do we know we're not heading for one right now?
IMHO, there are way too many unknowns to make a prediction like he does, let alone claim it to be a law.
Many other possibilities could happen. Its possible, for instance, that we (human race) will not see and correct the folly of our cancerous growth and consumption of the Earth's resources early enough to avert disaster. Its possible that this could throw us back 100's of years requiring us to rebuild up to the advanced level of technology we enjoy today but in sustainable ways. Its possible that we will enter space and embark on colonizing Mars, mining asteroids, and other space exploration and colonization. These activities, although technologically advanced, would put us back into a physical frontier where we just don't have the luxury of chasing surreal disembodied AI or virtual environments, having to use our ingenuity to survive instead. Its possible that some of the imagined technology is simply not feasible at all. Its possible that an asteroid will wipe out life on the earth as we know it (again). One could go on and on.
Back to the topic. So what would make the evolutionary process an S-curve? I have an idea. I call it the Law of Limiting Intelligent Knowledge Absorption. It applies only to technological evolution. It states that as knowledge increases the ability to absorb and apply that knowledge (needed for continued evolutionary progress) by intelligent sentient beings reaches a threshold associated with natural limitations inherent in the current physical embodiment and context of said intelligence. This acts as an opposing force to the law of accelerating returns.
I believe we are reaching this threshold in our current human form, our brains, human nature, our societies, policies, law and environmental disposition. And due to this, it will require a much longer time than Ray imagines- if at all - to move from our constraining biological form to new forms that can continue the exponential potential of evolutionary progress in a new S-curve. During this time, technological progress will continue, but not at the exponential rates required to create the technology and societal changes outlined by Ray in his books by the mid to late 21st century. I believe that due to this growth slowdown he underestimates the difficulty in attaining the kinds of technology and associated societal changes he describes. This is my optimistic outlook. Ray's is certainly fanciful and imaginative, but I don't believe immortality is just around the corner.
Matt
|
|
|
|
|
|
|
|
|
Re: Kurzweil’s Law (aka “the law of accelerating returns”)
|
|
|
|
This thread is discussing the technology aspect of the evolutionary growth. I was trying to introduce policy, human nature, society, and environmental context factors.
As long as we're still human flesh and blood, we still must be born and raised and educated to the current level of common knowledge. Individuals still need to specialize in order to get to the leading edge of a field. This still requires 20-40 years per individual to do. Ethics and politics, law and order still control society. Societal norms do not change at evolutionary speeds like technology does. Technology requires huge amounts of capital and natural resources. What I am saying is these inherent factors related to our current physical reality and environmental/social context are beginning to limit the evolutionary growth themselves.
Try to imagine exponential technological progress resulting in these new paradigms on the order of every year, month, week, day, or hour? It is not possible given our current form and physical reality. Generations of people are required for massive changes. My thought is we are just now or just have started reaching the point where these factors are making a difference and slowing down paradigm shifts. Also that if we "move" to a new form of intelligence that it will have similar limitations inherent within it but on a different level.
Assume for the moment in the next 30 years, the technology for brain scanning and emulation is achieved to the degree needed to copy an intelligence to a computer. What would happen? There would be much debate over the ethics involved and there would be as many or more people that would simply flat out reject the idea of being instantiated in a computer as much as accept it. Laws may restrict its use. The rich early adopters would be needed to begin building the infrastructure needed to make it a common available option. How long would this take? Would masses of people accept it ever? Where will the raw materials needed for millions and billions of new sentient beings come from? How would this help the human condition and the planet?
Personally, I tend to think that a more likely evolutionary scenario is that we over time slowly integrate technology and biology into an evolving humankind that will for 100s if not 1000s of years still bear resemblance to our DNA based roots (like you can find vestiges of DOS in Windows XP code). This is commonly referred to as the post-human or trans-human movement. We will be forced to explore space/colonize other planets so we can continue our population growth and resource utilization (its all just a big pyramid scheme ;-)) This seems a much more realistic prediction than that of a an un-explainable singularity...
Matt
|
|
|
|
|
|
|
|
|
Re: Kurzweil’s Law (aka “the law of accelerating returns”)
|
|
|
|
Dear Matt
Evolution IS exponential. I know what you mean with stagnation: Why has there been a standstill of 1000 years in the Midle ages. The hellenistic civilization was only a few hundred years away from its industrial revolution, as we were at the beginning of the renaissance, why has it stopped. Even much earlier: The brain of stenonychosaurus could learn as fast as that of a ramapithecus. Who knows how close some creatures living just before the Permian mass extinction were to hominide intelligence? There have been many general s-curves in the history of evolution. But why?
Because (don't laugh) an exponential curve in the natural world is different from a mathetical one: unforseen shit can happen. The curve in the cretacious was so extremely flat that a big asteroid had time enough to strike. No dinosauroid was enough developed to go in space and prevent it from happening.
A big asteroid strikes once in 100 million years. So it must happen in this century, or we blow it out of the sky. The chance that this happens is so small, that I don't think, Matt, that you can't integrate this event in your evolutionary model.
Why has technological evolution slowed down after the golden age of hellenism? Because only a comparatively minor military conflict or a famine was enough to unbalance a dawning civilization. Nowadays, it takes a lot more to create such a general s-curve. You would reply: But the ability to unbalance civilization has also evolved.
I know. We have enough nuclear weapons to destroy us, but will we do it? So many years of cold war didn't lead to global destruction, the amount of nuclear weapons will be reduced to zero in a few decenia. North Korea can launch only a few rockets, whereafter it would be immediately and totally destroyed.
So If the evolution of technolgy will slow down, it won't be due to meteors or nukes.
I believe mankind in general has become a lot wiser than 1000 years ago when a certain pope ordered a theological scolar to calculate how mant angels could fit on a pin's head. This is no joke! We have the wisdom to handle dangerous technologies. Accidents will happen, but on a global scale they will be minor nuisances.
Face it: Two thousand years ago, a delay lasted 1000 years, now a delay lasts 10 years. The singularity will happen in this century and we will have to face it.
|
|
|
|
|
|
|
|
|
Re: Kurzweil’s Law (aka “the law of accelerating returns”)
|
|
|
|
But getting back to my question. Describe the singularity - what does it really mean?
What kind of meaning are you looking for- shit happens- is that good enough? How about- continuation of technological growth at a faster and faster rate that is well documented with Moores Law as well as all the other paridigms behind itsuch as vacuum tube computer and transistor machines- its quite easy to see that this is likely to continue for some time with the predicted 3d chips using nanotubes etc. As for the limit thing you mentioned earlier, i have not read anything describing a limit on technological growth once the singularity is ignited. The only limit Ray describes is the limit imposed by a limit of matter and energy availiable- im not sure if this restricts technological evolution- i dont really care at this point either.
what does it imply for the human race or sentient race that reaches it?
again, i dont know what you are seeking here... read the Kurzweil stuff all over this forum and critise whatever you feel is wrong or produced a false implication or lack of one... It seems clear to me that it implies a merge between human and computer tech/nanotech, upgrading ourselves... you know post human-post post human etc. Ray makes it clear that our human quality is maintained, that is our emotions, consciousness etc i guess...
what does it imply for the universe?
saturation of intelligence-there an essay on this read that first...
Can you really put any meaning to such a thing? Rules break down at a singularity. I don't believe it will come. What meaning, you mean like a meaning of life or something? Do you have a meaning, a purpose? Ray bases his predictions of what he observes from the facts of exponential growth, how does this break down Laws of the universe, but dont tell me, point this out to subtillion who knows the universe like its his left hand, if he says its not feasible then i'll agree with you then...
|
|
|
|
|
|
|
| |