Let the AIs, not us, formulate a billion-year plan!
October 12, 2012 by Robert L. Blum
In What our civilization needs is a billion-year plan, posted on KurzweilAI September 23, 2012, Lt Col Peter Garretson calls for a long-term plan to assure humanity’s survival, “moving everyone and everything we value off Earth.”
He cites the coming big extinction events for planet Earth, including asteroid collisions, the Sun engulfing the Earth during its transformation to a red giant, and ultimately, the heat death of the Universe. Human survival, he argues, justifies an ambitious future space program, “with articulated goals of space development and space settlement … pushing the technology and logistical capabilities to be able to attain those goals.”
To accomplish this, he predicts, people (perhaps augmented) will be the great interstellar engineers — in charge of intelligent civilization over the next few billions of years.
Unfortunately, Garretson does not mention the single most important development in the future: the coming technological Singularity, when machine intelligence will surpass human intelligence, leading to “technological change so rapid and profound it will represent a rupture in the fabric of human history,” according to Ray Kurzweil.
Is humanity capable of planning beyond the Singularity?
I agree that long-term planning is essential, but our political process in the U.S. barely allows planning beyond the next election. (Look at any session of Congress on C-SPAN for an hour. Does that look like the face of wisdom that we want to draft a billion year plan?)
Looking at the march of evolution through the eyes of Teilhard de Chardin or Arthur C. Clarke, a prevailing belief (held strongly by me) is that humanity is not the last word in intelligence or its highest expression. Rather, we are just a warm-up act — a stepping stone to what comes next in evolution.
But meanwhile, humanity (all 7 billion of us) is a mixed curse. Throughout our history, we have seen the face of evil with the megadeaths of the Stalin Era, with Hitler during the Third Reich, in Cambodia during the Pol Pot regime, during the Rwanda massacres, and currently in Syria. (The Civil War, the bloodiest war in U.S. history was a mere 150 years ago.) That is humanity’s inhumanity to man.
To other species we are even worse — they don’t even count. Except among environmentalists, there is usually not much protest, as the scourge of our propagation envelopes Planet Earth with new homes, roads, buildings, agribusinesses, and dumps. The loss of habitat is leading to an extinction rate comparable to that of the Cretaceous era with an asteroid strike 65 million years ago.
The biosphere of Planet Earth is one large garbage dump. We have poisoned the soil, rivers, oceans, and atmosphere. The exhaust from our civilization creates more than 98% of new CO2 flowing into the atmosphere. It is likely that the IPCC has underestimated the extent to which Earth will warm during this century: a 5 degrees C rise is quite possible according to Paul Ehrlich).
A catastrophic disruption to agriculture may precipitate global resource wars. The numerous feedback mechanisms among global warming, global toxification, declining ecological services (e.g., death of pollinators), and overpopulation all point toward collapse (video). This litany of threats has been widely documented by Paul Ehrlich, James Hansen, and others.
As another 2.5 billion people are added to the planet by 2050, catastrophic collapse may be all but inevitable. We may not make it to the Singularity.
Can future enhanced humans solve these problems?
If our civilization produced literary and scientific giants like Shakespeare and Einstein, isn’t it reasonable to expect that humans augmented by new drugs, stem cells, implants, etc. will be even more talented and will be the astronauts who will build Garretson’s Dyson spheres and travel to the stars?
My answer is no. First, despite future medical advances, drugs, biologicals, and devices intended for human use always require lengthy and costly testing. At present, a new drug typically requires at least ten years and a billion dollars to develop. Frustratingly slow, speaking as a former emergency-room physician!
Several neurobiologists have published articles and videos that promote scanning and uploading the detailed microanatomy of the human brain (Sebastian Seung, Stephen Smith, Randal Koene, Ken Hayworth, and Anders Sandberg.)
I’m optimistic that such an approach will help to elucidate the principles of neurophysiology, but I have reservations (some are discussed here): the role of local fields and oscillations, the roles of glia and of gap junctions, and unexplained intricacy at synapses). I side more with Tony Movshon than with Sebastian Seung in this must-see debate.
But, suppose my “head freezer” friends succeed in being immortalized as was Han Solo in Star Wars when he was embedded in carbonite. Whether run on a mainframe or downloaded into a new titanium exoskeleton in 2100, Humanity 2.0 will still have all our current psychological failings (detailed by Daniel Kahneman in Thinking Fast and Slow).
More fundamentally, early design commitments, frozen into us as we evolved from single cells over billions of years were superbly adapted to local materials and conditions on Earth but are not suitable for space. It’s time for replacement. You cannot make a (bio) neuron that spikes at 3 Ghz or that conducts neural impulses at 300 million meters per second. (The way to speed up a cheetah is not by strapping on a jet engine.)
I also reject Kurzweil’s premise that we will merge with AIs — that’s like merging that cheetah with a jet plane.
AIs as the next intelligence carriers
Instead of humans, post-Singularity AIs (not us) will be the highest intelligences. And they will be calling the shots on design and execution of massive space-based engineering projects a la O’Neill and Dyson.
Long-term, humanity (whether augmented, re-engineered, or uploaded) will be left in the dust by the machines, who will stand in relation to us as we to bacteria. OK, that has a heavy-metal Skynet ring to it, so let me replace it immediately by a term I’ve come to love (from David Grinspoon’s book Lonely Planets): the Immortals.
Who are the Immortals? Perhaps we know who we want them to be: wise, superintelligent, compassionate, and just. And powerful! More powerful than a light-speed rocket, able to leap into intergalactic space in a single bound, and imbued with truth, justice, and the Western democratic way!
Whatever we choose to call them, further evolution of themselves and their tools will be in their hands and not ours. While future advances will greatly benefit humans, humans will be replaced as the helmsmen of a space-faring civilization before the Singularity — probably by 2040 (Philip K. Dick nailed this prediction in Blade Runner).
The evolving prototypes that will eventually leap to the stars will be electronic — informed by human design and concerns, but not constrained by them. Their decisions and wisdom will encompass all that is on the Web and all that is perceived by the world’s sensors. With a solar system full of effectors they will accomplish engineering that we cannot imagine. That is how they will begin their evolution and their journey to the stars.
So let’s leave the really long term planning (post-Singularity) to the Immortals.
Sometime before the Cambrian era 500 million years ago, the first differentiated, multicellular creatures arose. As the reproductive unit changed from a single cell to a multicellular organism, the individual cells had surrendered their autonomy for a greater chance of survival.
I think about the coming superorganism as something that will (at least initially) encompass human beings and confer upon them greater survival and quality of life (see Greg Stock’s Metaman). Just as the Web will embrace all of humanity and our culture, machines will evolve that understand and contribute to the Web.
Robots will autonomously update their databases and plans from the Web. The “rise of the machines” and their gradual metamorphosis into the wise Immortals won’t take place overnight. This will be a gradual evolution, dictated as always by “technology push and demand pull” (initially from human consumers, later from AI consumers).
So what projects should we humans undertake now?
These predictions will not happen automatically as a consequence of accelerating technology. They will require concerted science and engineering specifically focused on AI, including machine learning, robotics, computer vision, and knowledge representation; non-von Neumann architectures including neuromorphic engineering and other large-scale parallel designs; materials science; neurobiology; neural nets and cognitive science (to mine their principles), and the mathematics of dynamical and stochastic systems (among others).
Funding this type of R&D is civilization’s near-term path to the stars.
To advance the ball down the field, a thriving, productive, high-tech human civilization may be required for another century or two. Whatever slows or halts that development might kill this development.
As Garretson has pointed out, the spoilers include all those near-term, extinction level catastrophes that could derail the phase transition of intelligence: asteroids, propagation and rogue use of WMDs (nuclear and biologic), accidental worldwide war, pandemics, ecologic calamities, resource depletion, natural disasters, economic and societal chaos, etc.
There are also possible theoretical spoilers. Perhaps it is simply not possible to create intelligence or consciousness at parity with humans for as-yet-unknown reasons, famously argued by Roger Penrose. But recent progress in computer vision (Google Driverless Car) makes me intensely skeptical of these limitations.
Primates in Space
Unfortunately, the current funding environment for science and engineering is extremely limited, so NASA and DOE have had to kill promising projects in annual and decadal reviews.
For example, it was two decades before the spectacularly successful Kepler telescope was funded. The SETI Project, formerly a part of NASA Ames, lost its funding 15 years ago. The promising Terrestrial Planet Finder was cut and even the Hubble repair mission — another spectacular success — had to beg for funding. The vital follow-on to Hubble, the James Webb Telescope (JWST), has limped along with continued funding always in doubt, and the launch is now pushed back to 2018.
Manned space missions typically cost 100X the price of unmanned missions without a commensurate return. Put another way, a single manned expedition may kill scores of science-based, unmanned robotic probes and telescopes. The example du jour is the MSL/Curiosity rover now on Mars.
It cost (a mere) 2.5 billion dollars, in comparison to a manned mission that will cost 100X as much, if funded. My views on manned spaceflight coincide with those of Astronomer Royal Sir Martin Rees (video). I favor humans on the Moon but not on Mars — the key difference is travel time.
Putting humans into space requires the launch of consumables (food, water, shielding, medical supplies) and engenders great concern and over-engineering to assure safe return of the astronauts. Mars missions that are being sketched out for the late 2020s would involve at least two other launches to pre-position caches of consumables.
In 2012 it is easy to tout the superior dexterity, adaptability, intelligence, and autonomy of humans over robots. In 2025 to 2030, when the earliest manned Mars missions might be launching, it is far less clear whether astronauts’ superior abilities will justify their 100-fold expense. When we hit 2100, it’s surely game-over for what I call “primates in space” — which began with the Astrochimps Ham and Enos that orbited in the early sixties.
Ad astra
My view is that humans or other species will go to the stars, only if the Immortals (the AIs) think it is desirable/cost-effective to do so. I think they will want to transport us away from Earth to prevent our destruction.
Just as our biologists delight in the manifold diversity of Nature, I believe that the Immortals will be interested in preserving and studying us and many other species. Like anthropologists they may find value in studying our primitive culture.
If they decide to transport us, they will easily be able to do so. An old idea (on which I based an unpublished novel that my cell biologist son grew up with) is simply to transport the DNA sequences of a collection of humans and other animals to a remote in vitro fertilization machine constructed from local material in a distant world.
So, what’s to become of humanity? My view is that we primates will be on Planet Earth for a long time (even if augmented). My hope is that we mature into a wise old race of beings living in harmony with our biosphere.
Humans may even achieve immortality as predicted by Aubrey de Grey’s SENS (but perhaps not on his ambitious timescale).
I glossed over the crucial notion of whether the Immortal AIs will share our values or our feelings — and the key issue of Friendly AI. I can only prognosticate that they will share our values over the next several decades of development. In doing so (by incorporating human values into their utility functions or emulating human emotions) they will assure their commercial success as assistants (SIRI and Asimo), researchers (Watson), or drivers of our cars (Google car). (I’m personally skeptical of the efficacy of friendly AI long-term, concurring with Hugo de Garis; but read Steve Omohundro’s defense here.)
To paraphrase the quote from Lee Valentine that closes Garretson’s article: Mine the sky (by the AIs). Defend the Earth (by humans now, later by AIs). Settle the stars (by the Immortals — our mind children — if we do our job as good parents).






Comments (79)
by Peakstar
Bob I enjoyed your article. Humans are destined for the recycle bin I predict, and I think that machines will eventually calculate that Homines sapientes are simply in their way, all the time. One of the first recommendations the coming AGI will output is the obvious- the human animal needs to plan out its reproduction sustainably, on a generational 1-1 basis (after shrinking of course). But if you observe the Earth from a macro perspective, the “brains” of this civilization, our scientists, have been outputting that very same recommendation for decades. But in a democratic system the masses, oh those unthoughtful masses have so little interest in the brains of our civilization operation. They are concerned with “reproductive rights” and have culturally welded themselves to the idea that human reproduction is always, without question, without any possibility of alteration, a positive, even holy occurrence. But the Homo sapiens is choking the world as you stated. So will AGI speak with a voice that is respected by the masses? Time will tell. The AGI will see “errors” all throughout the global human civilization and culture. The interesting moments in future history will be the change from AGI recommendations to AGI directives.
by alliwant
Very similar to my view of how the deep future will unfold, at least concerning the future of terrestrial life, and humans in particular. Just the fact that developments like moving the civlization will take millions of years and span interstellar space assures that AIs will eventually be responsible. They will have to be the trailblazers, because humans just will not be able to navigate the stars without help. The timescale and demands are incompatible with our weaknesses.
by Chuck
I predict that AI’s will be neither “good” nor “bad”. AI’s will not be infected by the same greed for power, desire for materiality, nor will they feel the need to love or follow a moral code. AI’s will be like Vulcans. They will do what is logical. Killing and enslaving humans and taking over the planet ala The Terminator just isn’t logical. But don’t wait up for an AI to cozy up to you at night, either.
by Gabriel
Why not? Why would they be apathetic? Don’t you find that to be a pretty cartoonish assessment? If they are to be even more intelligent then we are, the logically, they should be capable of even more emotion then us, both in quality and quantity, just as we are to animals further down the food chain.
Perhaps they even show emotions that we simply don’t know or don’t exist for us, that wouldn’t be surprising.
by Editor
Right. Marvin Minsky’s The Emotion Machine: Commonsense Thinking, Artificial Intelligence, and the Future of the Human Mind could help clarify this. “He argues persuasively that emotions, intuitions, and feelings are not distinct things, but different ways of thinking. By examining these different forms of mind activity, Minsky says, we can explain why our thought sometimes takes the form of carefully reasoned analysis and at other times turns to emotion. He shows how our minds progress from simple, instinctive kinds of thought to more complex forms, such as consciousness or self-awareness.” http://www.amazon.com/Emotion-Machine-Commonsense-Artificial-Intelligence/dp/0743276647
“
by Gabriel
The thing about the word “intelligence” is that it’s more, well, intelligent then people typically give it credit for….usually, people are simply thinking of the word “knowledge” — as an example, we are the most intelligent species on the planet, not just for our brain power, but also our opposable thumbs, emotional intelligence, overall extended capabilities over other creatures etc…
Funny thing is, we already are in a world actually with stereotypical AI’s that are amoral and can perform tasks better then us…..they are called “narrow AI’s – the whole point and difference with Strong AI’s is that they will be the complete package and will be just as emotional and supple as we are…and of course, ultimately go farther.
In other words, they will reach our level of intelligence and exceed it – a narrow-AI is nothing more then an animal performing a smart trick….clever, but you wouldn’t call them smarter then you.
by Jim Mooney
We can’t assume they’d be logical. Logic is also a basis for the neurons, yet we act like idiots. Our brains and bodies are Fantastically complex, but look at Congress or Hateradio or war. We use incredible complexities for simple absurdity. A person of average intelligence can use visual programming tools he doesn’t really undersand to build a stupid, racist website. It is complexity in the service of stupidity, a la the incredible science and engineering of the Third Reich. For all we know we could have venal, stupid, or absurd robots.
by Bob Blum
Lest anyone infer that I think the NIF laser fusion energy project is nanotech, I don’t. It’s extremely macrotech. I just used it as an example of a project promising infinite clean energy that probably will not be scalable.
I just received word of the 2011 Feynman Prizes in nanotech. Spectacular work! (See foresight.org) But, early stage research nonetheless. Logic, please feel free to add more examples. (My sense is you have many more to share with us.)
But, I still don’t understand why you feel that nanotech supports an AI/human combo more than it does AI. To me that issue is entirely separate from the efficacy and roll out of nanotech. Please take the final word.
by Gabriel
“(October 16, 2012 by Bri )That’s a very true statement. Check out the movie Born Rich. It’s by one of the Johnson and Johnson heirs. He interviews his contemporaries as he is about to turn twenty one, and inherit a gazillion dollars. They are all adrift. No meaning or purpose in life. It’s a real eye opener.”
Bri, that’s nonsense talk….if that man truly feels like he has no purpose or meaning, then it’s up to him to make something of himself, just as we all must do — why doesn’t he use that money to become a humanitarian and donate massive funds to good causes like Bill Gates? or set up something like the X Prize like Peter Diamandis?
You can’t blame laziness or a lack of purpose, or whatever, on the resources or capabilities…..that just makes you look selfish. We all have to make do with what we have, no matter how great or small, and the more you have, the less justification you have for not knowing what you should do, because you are capable of doing so much more, that other people don’t even have the option too.
That’s silly — we alll must create our own sense of meaning and purpose in our own lives, regardless of how much we have; to blame the money and resources is just stupid….perhaps he truly WOULD be happier if he gave away his fortune (and many many people would gladly take it)…..but that’s something he has to recognize for himself…
Whatever you do with yourself, it’s all on you, not the stuff…blaming the stuff, no matter how little or how much, is just selfish and lazy. Capabilities are just that, capabilities….you have to make something of yourself no matter what you can do….but if the Johnsons really are unhappy with their massive fortune, I’ll gladly take it if they don’t mind.
by Bri
I know it’s hard to believe that too much money( abundance) can take meaning out of life, but it’s been observed before, in the super wealthy. Just check out the movie, it’s amazingly candid. He even interviews his father. All I can say is that it’s the real deal. He did ot as a college film making project. Please take my advice and whatch it. It profoundly influences my views, as to what will happen if we get what we think we want. It reminds me of the scene in the matrix, where agent Smith is interrogating Morpheous. He say’s that the first matrix was a failure. That we couldn’t take a perfect world. Watch Born Rich, then that scene in The Matrix and then the scene were Neo meets The Archetect. As he describes why his vision of the matrix failed and the “certain vegarities of human behavior” that caused it. The reasons play out on the TV screens behind Neo. The Wachowsky brothers understand these aspects of human behavior, and they use it to drive The Matrix story line. It’s actually an aspect of nature itself, but that’s an even deeper philosophical debate
by Gabriel
I don’t know what to say Bri — I’ll try looking at that movie, but right now, I disagree…
What are you saying exactly? That those men should give up their fortune? Is that what you think they should do?
You can’t blame the money — it’s one thing if these men genuinely felt like they would be happier without it, but you sound like you are making this Anti-Abundance reasoning which just seems wrong to me because it’s shifting the issue from the source….the person. Those men are more capable, in a sense, then almost anyone today could dream of….all that money could be used for so many incredible things, all that remains is the will to do them…..again, donate it all to good causes, take up humanitarian work…
They sound like selfish babies, that having too much money has “taken away the meaning of life” — does that seriously suggest that the toil and grind of work is actually necessary to give meaning to life? That we live to work instead of the other way around? That’s ridiculous…..these men are young, in their early 20′s….like anyone in that age, they haven’t made themselves “grounded” yet — they have alot of questions and haven’t established alot of things in their life…..this is normal….to blame the money as the ultimate source for the misery they may have, is silly.
Travel the world, learn some new languages, join some new causes, meet new people….there is so much they can do, with and without the money, but certainly so much good they can do with that others can’t dream. I know I’m repeating myself at this point, but you can’t blame the money for sapping their will…..that’s just irresponsible — we all have to ultimately establish a sense of meaning in our own lives, no matter what we have….things certainly are easier though when you inherent a fortune though.
Abundance will place everything at our fingertips, it will still be up to us to decide what to do with it, just as it was before….to actually bask in the toil-and-grind and rationalize such hardships as a good thing, is, in my view, simply more of the same misguided masochistic mentality toward all sorts of things like Death….keeping ourselves healthy and wanting to live as long as we can yet also deluding ourselves to see death as a good thing because we can’t do anything else yet….we are frightened of a world of Abundance despite working toward one and truly wanting one, simply because we have never known such a world — a darwinian competitive Scarcity world is all we’ve ever known, and we will reflexively and wrongly resist such a change because we’ve only ever known the hardships of a scarce world even though all that work to make such a change in the first place….suddenly the hardships went from something you had to bare through, to now being superior.
You can’t suddenly get cold feet right before the finish line.
by John
“Travel the world, learn some new languages, join some new causes, meet new people….there is so much they can do, with and without the money, but certainly so much good they can do with that others can’t dream.”
It can be misleading, this seemingly endless list of things that you can do and have fun from. But if you look underhood, all these things can be reduced to some finite set of needs that you have. These several needs drive you and make things valuable (interesting / pleasant / etc.). When all these needs are filled, all the list of things, no matter how long, is invalidated. Then there comes emptiness!
by Gabriel
John, merging with machines will enhance and enrich all of our experiences, as well as give us the longevity to see and enjoy them, not to mention living in a ever-changing world accelerating faster and faster….I can’t see boredom as an issue considering you will be able to do all the things you ever wanted to do, period, plus so much more.
From the perspective of ordinary humans in an Abundant world, Abundance doesn’t mean we necessarily be god-like…all our needs will be fulfilled, but what about new discoveries and contributions? the further progression of the human-race? As world-changing as it would be, a world of Abundance is not the end….an end perhaps to an old status quo, but certainly not the end, period….we still will have much to do and go….and for those who want to take part, they can enhance themselves to fully capable of accomplishing and appreciating it. For those that don’t and wish to “stay behind”, they will still live seemingly in a “utopia” where everything will be provided.
Even if merging with machines is non-existent, and we only got a world of Abundance, it’s still a huge amazing difference…it’s up to all of us, yet again, to create our drives, motivations and so on…and it’s not like, again, we still won’t have issues and new contributions/discoveries/etc waiting to be unleashed….Abundance on it’s own is like a dream, but the universe is still waiting for us.
I don’t know….I feel like a broken record at this point — I’d like to think, again, that we are psychologically equipped to properly handle a world of Abundance and all of this is just blown out of proportion…but even if we’re not, that’s where I feel merging comes in to fill that void.
by Bri
Everyone will be affected differently. It has more to do with your perspective on life. At one point in the movie, he is interviewing his father. He asks,” so what should I do with my money”. The father responds,” I don’t know, maybe collect old manuscripts” when Smithsonian inherited his money, he led a ful life, and established the institute. Most people won’t handle it well. I suspect many will disappear into virtual reality, if not most. It’s one of the reasons pets are loved so much. People are hard to deal with. In virtual reality, that can be altered easily. I rate Born Rich as a must see movie, if you want to understand the meaning of life.
by Gabriel
Bri, what can I say….you are right that everyone is affected in their own way, but regardless….there are no excuses — even if you exclusively stayed in virtual reality, you would still be cornered into making big life decisions…you could alter experiences, personalities, and so much else, but all of that will just be surface manifestations….No matter how many more opportunities you have for happiness, it all means nothing if you don’t actually strive for them, whether it means having more money or enhancing yourselves…you’d have to find something to do with yourself, and while it may seem unfair, you do have less excuses the further and further you advance.
In any case, I’m willing to move on if you wish since it seems we’re just repeating ourselves — agree to disagree I suppose :). I certainly wish those guys all the best, and wasn’t trying (but probably did) in calling them ‘selfish babies’….after all, no matter how much money they have, they are like anybody else and must recognize and understand what it is they need, and aim for that….they simply also should recognize how capable they are and how much good they can do for others with the money they have…I couldn’t blame if they truly felt they were happier without it and adopting a middle-class lifestyle if that’s what would fulfill them, but I certainly hope they instead use it to better themselves and other people who couldn’t touch that amount – it doesn’t have to be a burden unless they make it so.
by Bri
In the long run, I agree whole heartedly with what your saying. That’s why I call it a perspective on life. It’s just that people tend to revert to the lowest common denominator. Look at that copper fortune heiress that died recently at somewhere around one hundred years old. She was a recluse most of her life. Another good example is that the people that most often reach one hundred years old, typically remain active and working till very late in their lives. You have to have a vitality for life. The basic issue is how you derive that vitality. What drives you forward.
by Gabriel
Dang, what a crazy conversation that was, it seemed to jump all over the place :p.
by Bob Blum
In regard to NANOTECH: here are my views (responding to Logic). First –agreeing with you – Kurzweil’s Law of Accelerating Returns (LAR) is the basis of my article. Ray nailed it: Peter Garretson neglected its application to future intelligence. That’s why postponing his billion year plan makes sense. Nanotech is at an early stage relative to where it will be in several decades, and yours and my hope is that that will always be true.
Now, disagreeing with your techno-optimist/ infinite abundance position, here’s what I meant by valley of death.
Just like the Lawrence Livermore NIF fusion energy project, many (but not all) nanotech projects are now at an early and tentative stage. They will not provide the post-scarcity resources you posit for many decades .
Meanwhile, the global problems you blithely dismiss are quite real. No one thinks the atmosphere will boil away in 50 years. As for extrapolating the past history of humanity, the possession of nuclear weapons by religious fundamentalist nations is a brand new phenomenon. Furthermore, the problem of providing sustenance for 10 billion people within a few decades is a brand new dilemma. I don’t’ share your unbridled confidence that the nanotech cornucopia will arrive in time.
Finally, you state “to imagine that brain and machine will not merge is to ignore the very essence of humanity.”
That’s hardly modus-ponens. I know we agree that the future is unpredictable. The direct implication would be that future predictions need to be couched in probabilistic language. My view of humanity and our deductive capacity is captured by Daniel Kahneman and many others: short-sighted, biased, limited. I’d like the ultimate synthesis you hope for to capture our good bits (whatever they may be) and get rid of the bad. (Your positions are clearly and well stated. I invite you to include detailed examples of the nanotech projects you’d most like to draw to the attention of our readers. Please have the last word.)
Responding to DCWhatthe: Beautiful post! Thank you. Quoting you “we are not necessarily terrific.” Yes, that’s an understatement. We think well of ourselves, because we write all the PR ourselves. To the biosphere and the other animals, we are an unmitigated disaster – an out-of-control virus. The human experiment might well end in collapse and failure. I sincerely hope it does not, but I do NOT share the infinite abundance point of view of some singularitarians.
An argument against merger that I have not explicitly stated thus far is that humanity may not be able to avert self-extermination without outside help. Yes, I hope for a period of several more decades in which war becomes increasingly less likely and universal education leads to a wiser, long-lived humanity. But that is not guaranteed.
The seeds of our destruction may be inherent in our own human utility functions. I share the hopes of other readers for a wise, sustainable human race, but our primordial drives may not be compatible with an immortal race. And, we are certainly not designed for space travel (except for Felix Baumgartner!).
by DCWhatthe
I really enjoyed these reflections. Good stuff.
I agree with Blum’s contrarian view that we aren’t the end-all. That our existence shouldn’t necessarily be one of the higher-valued targets of billion-year planning. We MIGHT be in there, might overcome our silly, petty limitations; but it’s not a given.
We might be special, so far on this planet, but we’re not necessarily terrific.
Regarding long-term planning of anything as complex as the fate of a solar system or universe – we also don’t know for a fact that solid planning is possible, even for higher intelligence. A billions years in advance? It is possible that surprises are part of the fabric of the cosmos, a law that can’t be breached, regardless of the iq of the observer.
Robustness & adaptability might be the focus of higher organisms. Be ready for anything, and deal with it.
But I could be wrong. The point is, we don’t know enough about the nature of intelligence, and the structure of nature, and what lies ahead, to prescribe the behavior of higher or lower intellects in dealing with future conditions.
As far as getting through the obstacles set by FDA-style agencies, as well as competing with organizations like DARPA for research & development of the types of technology that WE choose to pursue democratically, I have a bit of faith that groups like the Biohack community will help to get us past all the BS. We have a fighting chance at enhancement & survival, because of the variety of research, speculation, and empirical discoveries taking place.
Roger on the mental & emotional blocks that Kahneman talks about. They are inescapable. We don’t have a prayer of surviving this century without enhancements, and without some degree of ‘letting go’ – meaning that ego has to take a back seat to higher intelligence. That doesn’t necessarily mean giving up our individuality – I hope not – but it will mean accepting that there are illusive aspects to identity, and at some point the caterpillar has to be willing to risk change, risk outright transformation, if it wants to fly.
The neuro-debate sounds interesting, will check it out, thanks.
We’re not ready for space. Absolutely. Again, not without some fundamental physical modifications.
The merging of us & AI – if it takes place, it won’t be something where the steps from here to there, are clear at the outset. It might be another emergent property, something that someone will suddenly experience. Maybe it will happen while experimenting with software automation, or knowledge discovery algorithms, or auto-composing music while linked neurally to a smartphone or a pc. If this digital-biological integration is possible, and some of us are willing to take the time to experiment, to try new interfaces between ourselves and our digital progeny, it will happen. The only thing that I know for a fact, is that I am personally ready for that leap.
Will the hybrids be greater than the sum of both the digital & the biological substrate? Who knows? The prevailing heuristic posits a partial order between the effectiveness of biological systems vs. digital ones. Digital always trumps biological.
There’s also the assumption that complex digital systems will be able to simulate the slower biological ones. And that because electronic signals are faster than the messy communication between our chemical/electric hybrid cells, that they will therefore win out. So if intelligence is an emergent property which we can’t fully explain, we are assuming that this same property will take place at essentially the speed of light, just as it does now at the speed of neurotransmitters.
This all sounds plausible, but I’ll reserve judgement until reality plays out a little further. We’re near the beginning of this, and I’m not making predictions about whether purely digital organisms win the evolutionary lottery, or hybrids come through, or we both populate this vast universe. Or something completely different, from what we expected.
by Bob Blum
ROLE of NANOTECHNOLOGY
“Logic” (Oct 14) emphasizes the future importance of nanotech.
I agree with him; however, that’s where his logic ends.
Nanotech supports future AI development, perhaps more than it
does his imagined combo of machines and brains. But – agreeing with
him – “prediction is hard, especially of the future.”
Nanotech is in an even more rudimentary state than AI.
Drexler’s original notion is plagued by the “fat, sticky fingers
problem.” But my interest was heightened by presentations
at the Foresight 2011 nanotech conference: spectacular
(but more supportive of AI than AI/neurobio combos.)
MERGED HUMANS VS AI
Giulio Prisco (Oct 14) says that “In my favorite scenario the new,
augmented humans 2.0 move to the stars and leave the Earth to
humans 1.0.” Agreed – that’s also my favorite scenario:
Earth as a retirement home for humanity and other species.
(I disagree on who or what is moving to the stars – my guess is that
the travelers are much more machine than human.)
Giulio continues “a change in capabilities does not necessarily imply a
change in motivations.” Implication: supersmart AIs might still have the
same utilities (values, preferences, goals) as their human creators
(who merged with them, as children do as they engage the adult world.)
Perhaps: but fundamentally unpredictable. (I do share your hope.)
Miscellaneous on Friendly AI
MrQuincle (Oct 13) draws our attention to Mark Waser’s work on
Rational Universal Benevolence. I just glanced at it: it does look substantive.
Mike Davidson: Ascension and Merging with God: that’s what we’re talking about -
the nature of the Immortals.
Gabriel (Oct 13) also advocates Human/AI Merger as the path most likely to
assure friendly AI.
Jon (Oct 14) makes the great point that AIs do not WANT anything. They have no
survival instinct, no reproductive urges, no emotions whatsoever. My view is that
those high-level goals will be incorporated once AGI gets going. Steve Omohundro
(in the cited pdf) is explicit about what those goals will be: survival, integrity, resource
accumulation, efficiency, self-improvement. Its uncertain now how those top-level goals
will be incorporated. Steve uses decision analysis/ utility functions as his basis (rather than
re-created animal sensations like pain and pleasure.)
Logic (Oct 14) replies to Jon – bases his reponse on anticipated infinite abundance
supplied by the cornucopia of nanotech (the Diamandis argument in “Abundance.”)
Logic – I entertain the possibility that your post-Singularity abundance may come to late -
after the valley of death from collapse. To quote you “the only question is whether we get
there in time.” Agreed.
Again, thank you all. (I was at the AI in video games conference (AIIDE12) this week at Stanford.)
by Logic
Thanks for replying, Bob, but you’ve unreasonably dismissed nanotech as being in its infancy, while at the same time discussing “billion-year” plans. Nanotech will only be in its infancy for another decade or two, if we accept the general predictive principle of Kurzweil’s Law of Accelerating Returns, which thusly makes nanotech far more relevant to your predictions than you appreciate. Nanotech and the sticky finger problem are nothing more than engineering challenges, a domain under which LAR applies.
Regarding “getting there in time”, my quip was tongue in cheek; I see nothing on the horizon that would lead me to believe self-destruction is imminent (and I include the various religious and political conflicts), which makes exponential growth across disciplines the far more likely scenario. Anyone of a certain age and with a rational sense of history knows that all our present woes are nothing more than modern variations on the same woes we’ve had all along. The climate won’t boil in the next 50 years, there won’t be catastrophic globe-scale wars, and the sky is not falling. No more so than it all has been for the past 50, anyway. Which means exponential progress is the more likely outcome.
I have yet to see any cogent argument opposing Kurzweil’s LAR anywhere (and I have tried to find one). Building that into one’s predictive framework, then, it is reasonable to conclude a fast-approaching post-scarcity world, the kind which Diamandis writes of. To imagine that brain and machine will not merge (and soon) is to ignore the very essence of humanity, which is what my reply to Jon was referring to. We reach because we are self-aware, so why wouldn’t we; what else would we do?. We only destroy and grab resources when there is scarcity. And it is the combination of nanotech, AI and the various disruptive technologies that will fuel one another’s growth that your predictions (at least in this article) overlook.
by Bob Blum
Post-Singularity AIs: Machines or Uploaded Humans?
Consider two scenarios:
Facebook 2020 and Facebook 2050
FB 2020 is a Terasem style model of 7B people:
it records detailed preferences (utilities) on thousands of difficult issues
for each person. Not only Democrat vs Republican but also very fine-grained positions on stem cells, global warming, gun control, death penalty, US Constitutional interpretation, appropriate size and role of federal government, measures to limit debt, taxation.
Each of these profiles is an AI that can interact with the other 7B AIs to
persuade, cajole, listen, respond, lobby.
Out of this interaction, some preferences are clearly preferred by a majority. Trade-offs are also identified: spending to support research
(a good thing) is tied to raising the federal deficit (a bad thing.)
AIs and databases of this sort are increasingly used to set policy.
FB2050: same as above but instead of detailed profiles, hand-crafted by
individual users, the machine now has detailed brain scans of all 10B people. Furthermore, lets say those uploaded brain scans can be run. Again, all 10B people are constantly adjucating their positions with many others, only now they are “core-resident.” Their flesh-and-blood counterparts are in Starbucks, sipping lattes.
I believe that FB2050 is close to the image that many of you (Giulio et al)
predict and identify as merging and becoming the post-Singularity AI.
My position: I also think something like FB 2020 and perhaps FB 2050 may occur. However, as stated in my article, it does NOT solve the problem of designing Kardashev 2 machines or solving planet Earth’s near term crises.
Just to make it absurdly obvious, let’s say that all 10B uploaded humans
are discussing Kim Kardashian and Kanye West 24 by 7. Where is the Superintelligence? Where is the sustained focus on arms control, resource allocation, and health of the biosphere? It takes a SuperIntel – not a bunch of merged mammals.
Here are other problems with a SuperAI comprised of a large set of uploaded humans.How to weight differing opinions? How to weight non-verbal creatures? How to weight time horizons (immediate pay-off vs long-term investment).
by Logic
I’m sorry but this analogy is patently absurd. Why would super-intelligence suddenly have a problem multi-tasking? Why not discuss Kardashian, Kanye AND arms control simultaneously? You reject the merging scenario without any rational argument against it (and your description of it certainly doesn’t match MY definitions), and then reduce humanity’s primary interests to gossip? I’m sorry, but I must object. You think “human” Super AI is only possible by aggregating human brains? I believe you are making gross assumptions that taint your predictive capacity.
Humanity has been on an upward trajectory from the start. The individual has always been celebrated, more so when his excellence serves the collective. Thus, we can reasonably expect that AI will be both machine AND merged human, coexisting for mutual benefit, for that is the next logical step in the evolution of these technologies. Perhaps you should more clearly explain why you think merging man with machine is so unlikely?
by Klaatu
Forget the hype of the next billion…
AGI inteligent agent modules are very likely
LEARNING how to learn right now despite what is in
the public eye or press right now. You start off by
duplicating the brain of baby or child and go from there.
It is also likely that when combined with various technologies
associated with the paranoid crowd on the internet…as in
these sites are put together with mostly deliberate nonsense
& historical references to MKULTRA etc are hiding something
in plain sight by associating certain facts with riht wing NWO
wet dream delusions. They
by Klaatu
(cont) These sites on YT and elsewhere are put together,
not by nuts, but by a real network of ppl who specialize in:
A. Deliberate BS and disinformation about the NWO etc.
B. Planning the future. 10-20-40 years.
This comment will be consigned to tinfoil hat territory
if I try and describe this thing further.
by Klaatu
WHY DO YOU THINK CLIMATE DENIERS4HIRE
ARE CENTERED IN THE REPUBLICAN SOUTH?
Think of what will happen to the US south if climate
predictions are too conservative in the next 5-10
years? The Michele Bachmann’s & Rep Broun MD
of Georgia are distractions. The South will need a high tech
bailout if it wants to remain viable politically
and demographically. The ppl
by Logic
The problem with predicting the future is that you can only do so from the perspective of the present. A new discovery or invention tomorrow could radically upend every point you make.
So no matter how blustery, emphatic or impassioned you get, you’re never discussing anything more than our current moment. All the fears and hopes and paranoias that riddle this (in my opinion grossly overstated) article are reflections not of our future, but of this moment in time. We can project and extrapolate, sure, but we do so with the limited insight of today, and the value of our projections is limited by our omissions.
The most obvious problem here is the glaring omission of any mention of nanotechnology. To reject the merging of AI with humans with nothing more than a misguided cheetah/jet analogy is to dismiss the single most transformative technology we’re developing. Your entire imagined future (at least as presented in this article) has not accounted for nanotech, which will be equally as disruptive as (if not more disruptive than) AI. It, in fact, will be the mechanism by which the “cheetah merges with the jet”. If you introduce the accelerating evolution of nanotech into your imagination of the future, it radically upends almost every point you’ve made.
I love that KurzweilAI.net has published these two pieces of wonderful and thought-provoking “billion year plan” articles. Both, despite their glorious long-term thinking, are radically naive, and fundamentally flawed by ignoring essential technologies and principles upon which the future will be built (if current progress is any indication). But I applaud and deeply thank both of the writers for capturing different facets of the snapshot of where we are today.
by Giulio Prisco
Thanks Bob for taking the time to answer our comments.
Re ” I hope that unmerged human primates will exist as well as other unmerged species of Earth that are rapidly being extincted by us.”
So do I. In my favorite scenario the new, augmented humans2.0 move to the stars and leave the earth to humans1.0. I hope they will leave some kind of embassy to assist those on earth who want to join them though, I would hate to miss the fun because I was born too late.
Re “My intuition is that stored uploads of scanned human brains will be a tiny part of the memory of Post-Singularity AIs. They will not dominate their architecture nor their decisions or plans.”
I agree with he first part. I am not sure the second part follows though: the child that I used to be is only a tiny part of me, but we both love our mom. A change in capabilities does not necessarily imply a change in motivations.
by Vlad
Can someone explain me what is this?
http://kickstarter.kimerasystems.com/
Do they really have strong AI technology or it is just another ‘less weak’ AI technology?
by Bob Blum
BOB BLUM RESPONDS (Saturday 13 October)
Thank you all for your posts. I’ll hit the big topics.
MACHINES REPLACING HUMANS AS THE HIGHEST INTELLIGENCES
EricSMeyer accuses me of reveling in that eventual outcome.
No, I do NOT REVEL in that outcome. It is an outcome that
I predict WILL OCCUR post-Singularity. It is NOT an outcome that
anyone can gleefully welcome without reservation nor absent-mindedly ignore.Nonetheless, it will happen.
Even current machine intelligence has great capacity to do enormous harm (let alone future AI).
Read US Defense Secretary Leon Panetta’s warning in this issue of Kurzweil Accelerating Intelligence that appeared in the New York Times about cybersecurity.
Eric does not believe that the Singularity will occur, ie that machines will achieve independent intelligence. Futhermore, he argues, you would need to be a lunatic to create them to rule over or replace us.
I do believe that the Singularity will occur and that machines will achieve independent intelligence, let’s say in 50 to 200 years from now. 1) Totalitarian governments WILL develop them to exert ever more control and repression of their citizens. 2) In democratic societies they will mainly be developed by corporations (Google, Apple, Amazon, IBM, Microsoft, etc) as consumer or business products, whose fate will be determined by commercial success.
As I pointed out, these AI products now have names like SIRI, Asimo, Watson, Google Car. Future cloud-based assistants will be far more capable. There will be a whole industry of personal advisors and confidantes. They will become pervasive because they will make our lives better.
There IS a bastion of REVELERS amidst the video game players. AI researchers are hard at work in that industry making those games ever more realistic. Making simulated humans is an exponentially profitable business as avatars in chat rooms, in video games, in the movies, and as robots.
My attitude toward humanity is clearly stated: “we are a mixed curse” (for the reasons given). But, here’s my preferred remedy for humanity: birth control/ family planning and increased education (ie fewer but better humans with a lower carbon footprint.)
FRIENDLY AI (fAI)
ErikSMeyer’s very legitimate concern about “misanthropic lunatics” leads to the topic of FRIENDLY AI.
If you believe, as I do, that superintelligent machines (much smarter than humans) are inevitable, then how do you assure that their motives are benign – that they are not henchmen of dictators or unstoppable Terminators.
Read Steve Omohundro’s excellent paper on fAI (cited). This is a huge issue. However, for reasons cited by Hugo de Garis
in Humanity+ on August 21, 2012 I am skeptical of its feasibility.
So, is it logical to just plunge ahead and develop AI-based cars, robots, consultants, and avatars?
It IS logical in 2012 – they will be extremely useful. At some point, say in 20 years or so, some governments may try to
draw “red lines” with anti-AI laws. However, other governments (for commercial or military advantage) will go right ahead (as Giulio mentions).
POST-SINGULARITY AIs: MACHINES or MERGED HUMANS
Avlin (Oct 12) gets a gold star on this. Avlin listed my two reasons (I have others, too).
Our inflexible bio design and the fact that it takes 10 years and a billion dollars to develop a new drug.
Inflexible design: Gordon Russell’s suggestion (Oct 12) of a processor inserted into the prefrontal cortex will not enable that human stock trader to buy up the market. Reason: the rest of his brain is still moving at primate speed.
To get the desired result, you need to replace the entire brain with a post-Singularity AI. Develop THAT, Gordon, and
you’ll be able to buy up Goldman Sachs.
Ten year drug development: Jay (Oct 12) suggests that it might shorten. Yes, a bit. Stem cell models of disease are being
used to improve predictions of drug efficacy and side effects. The final drug still will need lengthy clinical trials. Human enhancement will NOT keep pace with accelerating AI.
Giulio Prisco (Oct 12 and 13) suggests that humans and machines will gradually merge, so why not just call the post-Singularity machines “humans.” (Giulio and I had an animated discussion on this prepublication.) This also addresses the cheetah and jet plane analogy.
Divide the future into two periods: first PRE and second POST-Singularity.
PRE-Singularity. Robots and AIs are clearly not human. BUT, they cater to our needs, goals, and desires. That is ASSURED by their competing in the marketplace for our dollars.
POST-SINGULARITY (minimum of 50 years in the future). They run on billions of servers; they communicate with light speed fiberoptics; they understand every field of knowledge; they autonomously conduct experiments in labs they built; they can tap into trillions of sensors and effectors, and they self-augment.
What are they made of? Who knows? How much of them is human? My guess is … not much.
In his post today (Oct 13) Giulio says “the only way to exist as a species will be to merge with the AIs until there is no Us vs Them but only a future humanity of advanced intelligences.”
1) I hope that unmerged human primates will exist as well as other unmerged species of Earth that are rapidly being extincted by us. 2) Uploading has existed for 30,000 years or more with cave paintings, oral legends, writing, the printing press, and telecommunications. I can watch Charlie Rose stored on my DVR because he has been captured in magnetic domains.
My website captures key pieces of my personality. Martine Rothblatt’s Terasem is intended to advance that even further.
3) My intuition is that stored uploads of scanned human brains will be a tiny part of the memory of Post-Singularity AIs.
They will not dominate their architecture nor their decisions or plans.
Again, thank you all for your posts… Bob
by MrQuincle
I found the text behind the link of Omohundro embarassing. If any, such an approach to limit Artificial General Intelligence to fulfill the remarkable short-sighted variants of Asimov’s laws, might very well be counterproductive.
I like much more the text from Mark Waser: “Rational Universal Benevolence: Simpler, Safer, and Wiser Than ‘Friendly AI’”. According to Kant: “Act only according to the maxim whereby you can, at the same time, will that it should become a universal law”.
I personally do not only think it will turn out to be suboptimal to neglect the wishes and needs of fellow earth inhabitants to reach your own goals, whatever they may be. I also think that the very goals of AGI will not be met if people attack the problem of morality and ethics with an if-then-else mentality.
There is nothing wrong with having “being friendly” or “being cooperative” as an instrumental(!) goal towards a general goal of survivability. We ain’t different…
by Mike Davidson
I have a simple 1 trillion year plan for humanity: ascend until we become (merge with) God.
by Giulio Prisco
@Gabriel re “The whole Us vs Them idea is just something perpetuated by hollywood…there is not going to even be a “them” – only an increasingly capable “Us”.
Right. And I think this is the only way forward. We are talking of a “them” (the A.I.s) billions of times smarter than “us” (as Bob says, they “will stand in relation to us as we to bacteria.”) How on earth there can be any doubt on which will dominate the other if the issue is framed as Us vs Them?
I consider the concept of “Friendly A.I.” as a total oxymoron. If they are really smarter than us, they will do what _they_ want, and they will find out (in microseconds) how to force us to also do what they want. If an entity has no other options than being friendly, then it is not intelligent. If an entity is intelligent, then it is not necessarily friendly. Of course, it may still choose to be friendly. All parents, and everyone who has some familiarity with children and teens, know that smart kids do what they want, not what their parents want.
So, “they” will be extremely smarter than us, and not necessarily friendly. This means that the only way to continue to exist as a species will be to merge with A.I.s until there is no Us vs Them but only a future humanity of advanced intelligences.
Of course, relinquishment (not building smarter than human A.I.s) will not work, because any ban will only push R&D abroad or underground. Also, as a Cosmist I _want_ to build super-intelligent mind children who will colonize the universe and, perhaps, take something of us out there with them.
by Tom
While I agree that we have to be wary of the motives of other (types of) intelligences, I must question your logic that independent-goaled smart teens means independent-goaled AIs.
These are two of the human tendencies we have to avoid, both irrelevant extrapolation, and anthropomorphisation.
Also, please see my response to ErikSMeyer, below.
by Mike Davidson
I agree with everything you have said here, Giulio. Why can’t Eliezer Yudkowsky or Hugo de Garis see this? A “war” with artilects will last about 1 nanosecond…it would be more like an adult taking candy away from an infant.
by Gabriel
The thing about “Friendly AI” is that it’s not about ‘forcing’ them to be friendly…that’s an impossibility – My Dog or my Goldfish cannot control me no matter what it tries to do….there are no hardware-tricks we could pull that would force them to be friendly, that’s an impossibility…..the whole idea about creating ‘Friendly AI” is striving to create an environment where they are ‘most likely’ be friendly to us.
However, just as parents can strive to guide their children down the path they intend by placing them in the right neighborhood, schools and so on…their is nothing that truly commands their obedience…
When I stop and think about that, I actually think, not just the best, but the only real solution we have AT ALL is to merge with them….because merging with them would eliminate this supposed-rift…We won’t have to worry about them being friendly or not because we will become ‘them’, or more preferably, a more increasingly capable “Us”.
If you are curious, I write more in the “rise of the machines: the really bad news” topic….as much as I disagree with the Terminator scenario, there is really nothing that 100% says we will have nothing to fear if we go down a certain road, even if we merge with them (perhaps enhancement carries a lace of malevolence by being brutal to the “regulars” and everyone underneath — I disagree, but it could still happen)…..then again, perhaps this isn’t the right thing to say, but you could say the same thing about human beings…
by Jon
But regardless of how intelligent we make AI’s; why would they ‘want’ anything? It would seem our urges and instincts are a result of evolution and the like, we’re programmed as it were.
I see no reason why a super-intelligent machine would intrinsically want to dominate, grow, or would even care about it’s own existence – unless we programmed it to do so?
(I do not mean to start a debate about free will here, just having a thought.)
Perhaps humanity and it’s (by then vestigial?) desires are necessary to give meaning and a goal to all the reality-changing power that AI will have. We appear to be insatiable, but perhaps that can be a good thing, a necessary thing, in the right setting (one of infinite possibility?).
Whatever happens; merely sitting here, waiting for our sun to die would be an incredible waste of “what could have been.” Rather go down reaching for the stars.
by Logic
Jon, I agree with this, and have never understood why those who are afraid of AI superintelligence always anthropomorphize as they do. We imagine that AI will destroy us, as in silly Terminator scenarios, but why? What is AI’s motivation for doing so? We are not a threat to intelligence beyond ours. Even destroying our world ony harms us; an AI robot could travel throught space without the need for gravity or oxygen. Skynet and Terminator scenarios are a reflection of Cold War sensibilities; the fear of mutually assured destruction.
But there is a philosophical aspect to the universe that scientists so often overlook, and you nailed it in your comment: we reach because why wouldn’t we? All of our problems and fears are vestiges of a scarcity model that derives from a competitive animalistic history, and are wholly unnecessary in an abundant and technologically advanced future with wide open spaces. Our irrationality comes from fighting over limited space on our little rock. Remove the constraints and there is little logic in the fear.
by Gabriel
It’s just natural Logic — those fears are based of what we’ve always known thus far…Darwinian principles, survival of the fittest….everything you say, makes sense, and yet, we still can’t 100% say that things will be as easy as all that….or as dystopian as others say. Maybe they will be so intelligent in that we will have nothing to fear…or maybe it’s just a simple repeat of historical technology-advanced countries in that AI’s want to “help us” (i.e. abuse us) or simply outright destroy us….perhaps it really is worth it on the other side, but perhaps unintentionally and inherently, the intelligence growth makes for a lace of malevolence by being cruel to everyone “underneath” and their mega-intelligence flawlessly justifies this….essentially, this is nothing new.
The biggest implications are that we don’t know what they are – one can get at me for being too optimistic, and I can get at them for being too pessimistic….we just.don’t.know.
What do I think? I think we’re headed down a middle-road….Knowledge is Power, and Power is neutral, a utility….to be used for good or evil, smart reasons or dumb reasons….it’s totally impartial and so are we.
Good and Evil in the future will continue to expand – we live in a world where incredible altruism can be done, and a nuclear warhead can vaporize a city….the future will simply be more of this and even more incredible good, and evil, can be done….a new status-quo over a true heaven or hell. Age-old problems will be solved, and newer, more profound problems will be introduced to take it’s place…imagine if Al-Qaida woke up today and found out they could do anything just by willing it….the good news is, there will be many more, with the same power, wanting to stop them.
It won’t be an easy world by any means, but better as a whole? No doubts there…that’s how I see it in my mind anyway. The future is always better, no matter how complex and difficult it seemingly gets. We, to them, probably look like we have it super-easy and primitive (and of course, we’d argue that it’s not true), but is it worthwhile to wish you were there instead? With mortality alone still being a problem, I couldn’t blame them for saying ‘no’. By that same logic, we must realize we have it better then ever…we can romanticize the past, there shouldn’t be anybody wishes they didn’t live in 2012…well, except the future.
by Logic
I agree with your point, but what is the point of “Evil” in a post-scarcity world? Imagine nanotech that repairs cell damage, making death obsolete. Imagine all the land you want and all the material wishes you want or need available abundantly. What, then, does “Evil” actually do all day? I submit that these are not universal concepts, but rather constructs of the narrow moment in time we have witnessed. But simple (exponential) extrapolation makes it all rather silly. It is necessarily absent in a sufficiently advanced civilization. Why would AI destroy (or anyone) in a post-scarcity world? It’s a fear entirely constructed by, with, and for our current moment of evolution, when we must compete for limited resources..
We do indeed have it better than ever before, and it will continue to improve indefinitely, with occasional hiccups born out of our competitive/scarcity history. Gradually these fears will be relegated to the domain of childish worries. Technology drives us toward abundance, and with it, pushes “Evil” into irrelevance. The only question is whether we’ll get there in time. Which means, simply, we either face a utopian future, or nothing at all. There is no middle ground. If its the latter, just like turning off the light switch of life, we won’t ever know it happened anyway. So it makes no sense to worry about it, and any predictions of the future need to account for the social impact of the elimination of scarcity.
by Gabriel
Hmmm….you raise good points Logic — Bri said once that a suicide bomber won’t have to kill for 72 virgins in heaven when he could create that scenario with virtual reality….essentially, all our needs will be alleviated and it will seemingly be a utopian.
The reasons for why I say it won’t truly be a Utopia, is because we can still lose it in mere moments no matter how much work it took to get there — we still live a world with enough nukes to blast us to smithereens many times over…the post-Singularity world, I imagine, would be to an even higher degree. You can argue, yes, that there is less and less of a reason for suffering and to create suffering as time goes on, and Abundence will dismantle many reasons for breaking the peace….
The thing about the word “utopia”, is that you have to recognize, sooner or later, that you really ‘are’ in one…we are in one relatively-speaking: having it best then any time period thus far (again, this is natural – it has to be; as before, no better time then the Present, except the Future)….this will continue more and more until we just can’t go any further….and maybe we truly won’t enter a perfect world, but I imagine it will so much like one, that we may as well acknowledge that fact….because we won’t stop until we can’t go any farther, nor do we have a reason too.
So why do I say we still will have things to worry about? Because, as before, Power is impartial…there is only so much a single-purpose power, no matter how great, can do….and Knowledge/Power itself is impartial — a utility to be used for good/evil, smart/dumb….our capabilities will continue to increase as time goes on, and that includes our readiness to act, whether for good or for ill.
Maybe this is the semantics over the word “utopia” then — because, no matter how utopian our world becomes, we will still possess the capability to destroy it….however, as it becomes more utopian, we essentially lose more and more reasons to do so, as you say….so even if Heaven is a place that truly doesn’t exist, the future will be so like it, that we truly won’t be able to really tell a worthwhile difference….and that will kill justifications for messing it up.
It all makes sense…I just hope things will be as easy as all that.
by Logic
I take the long view, which is why I was drawn to the billion-year plan articles. I simply can’t see any possible scenario in which significantly advanced intelligence has any need for the power struggles we have. The night sky is filled with infinite resources. If the Law of Accelerating Returns is even partially correct, we will achieve significantly advanced intelligence within a century, let alone a billion years. A universe within our grasp and the technology to reach it make the power struggles we think are so all-consuming (even to theoretical AI that have no such base instincts as we do) rather silly. They are a reflection of our history and our present moment in time, nothing more, and I think predicting our future ought to factor in the realities of post-scarcity (which, I believe, few people can truly imagine adequately, which is wherein the problem lies).
by Gabriel
Well….yeah – I think you basically have everything together Logic: it’s s difficult for us to imagine what a greater intelligence would be like or what it would do, simply because it is more intelligent….we can draw up seemingly plausible outcomes and ideas, but the end result is that we just don’t know, anymore then an insect could fathom what goes on in our heads and lives.
Many conceptions are mostly just an amplified reflection of everything we are used too, like survival-of-the-fittest and so on….Abundance will make Greed essentially nonexistent and overthrow that sense of king-of-the-hill thinking completely, which alone makes for huge changes. How could Greed possibly exist in a post-scarcity environment? It just doesn’t make any sense. Initially, perhaps, you are overwhelmed and want to collect as much of whatever as much as you can….but a post-scarcity environment will eventually wittle this behavior down….again…there is no sense of competition or “winning” when everyone has it all.
It’s funny, but you can argue that such dystopian outlooks paint a pretty bleak outlook on the human race….they are plausible, yes, but it sort of gives you an insight into our minds and what we would expect to happen because of what we are used too…..you could argue then that, we should hope that while they inherent the right values, that future AI’s are not actually like human beings…..that they are better then us and don’t reflect the sort of Darwinian mentality that we would expect them to have and are familiar with…and if they don’t, then we ourselves will merge with them and leave that behind as well.
Do we actually have the capacity to handle all of this? a world of Abundance? Maybe not….or maybe we do — it’s our nature to rationalize bad things when we feel their is nothing else we can do about it, and all these good things may make us feel like we’ll just sit around bored with nothing left to accomplish or do or any sense of competition….or maybe, that’s not the case at all when we enhance ourselves and we can take this, properly. After all, such fears of boredom and entropy are also rooted in our present-day psyche.
Whatever you end up believing in, you have to keep transcending and wishing, not just that you could become capable of doing more, but that you could ‘be’ more….that we won’t have to compete because everyone will have it all, and that we can properly handle it because will become able too…the future shouldn’t be tied down by beliefs rooted from the past, and neither should we.
by John
Man, there is no Abundance. When you have everything, that’s the worst thing that could happen to you. Then all these precious things lose their value, everything turns to ash and you suffocate because there is nothing left to wish. Like you became god and the question is now: “So what?” It’s no fun to create new universes, since you created trillion and it’s kind of boring now. So then you create a simulation of being human who struggles in pain to get things that he wants. Or you create simluation where you lose. It’s so fun to try hard your best, to believe and then fail miserably!
by Gabriel
You don’t know that John — that’s exactly the point i made in my post….that, are we psychologically ready/capable for that sort of transition if that’s what ended up happening? Are we psychologically ready for a world of Abundance?
There is no need for a world of Scarcity…whether we are truly capable of living in a world of Abundance and this is all over-blown out of proportion, or we’d have to enhance ourselves to truly make that transition, it’s all irrelevant — rationalizing a world of Scarcity is no different then excessively rationalizing Death as a good thing…it’s a slippery slope from “This is a bad thing we have to deal with” to “This is a good thing!” and the longer we’ve had it, the harder it is to let it go, even if we KNOW it’s a bad thing and we try to get away from them.
Look, to make my point, this all leads back to why I said we’re heading for a Middle-Road….if we actually moved into a world of Abundance, we would have to deal with the ramifications of that world in one form or another, and that would invariably lead to problems of some sort….the same way Nanotech will mean the end of biological threats, but now we have gray goo to worry about.
Yes, you do have to acknowledge that you truly do live in a paradise at some point or another, even if only essentially (as I went into back-and-forth with the word “utopia”), but I don’t think there is such a real thing as a true Heaven or Hell because, no matter how hard it took to make it, we always have ways of messing it up, even if we don’t have any good reason to do so.
There will always be challenges of some sort, whether or not they are the sort we’re used too….it’s turtles all the way down — Boredom itself could be considered a problem to alleviate, which would mean we’d need something to solve that….It’s easy to silently paint a dystopia over a utopian scenario, like a certain Twilight Zone episode, but it’s not gonna truly be either way…..we will always have challenges that will we overcome which will make for new challenges, continuing ad infintum
For the nth time, yes, things are getting better and better and that should be acknowledged…but I don’t believe we will ever truly live in a utopia or dystopia because of what I’ve explained. Essentially utopian? Sure…you could say that about 2012 right now, and the future will be even more so….but truly? No, I don’t think so…and maybe these isn’t the right thing to say, but maybe it’s a good thing.
by Bri
That’s a very true statement. Check out the movie Born Rich. It’s by one of the Johnson and Johnson heirs. He interviews his contemporaries as he is about to turn twenty one, and inherit a gazillion dollars. They are all adrift. No meaning or purpose in life. It’s a real eye opener.
by Bri
@Gabriel: Life is a bowl of cherries. They’ve all got pits. Don’t eat too fast!!!!
by Bill
If the deepest pockets funding AI is coming from Super Power governments investing in Military strategy and attack, it stands to reason, that an AI may view humans as an unnecessary propagator of atmospheric CO2.
by Gorden Russell
Right, Bill, we have to hope that these AIs have processors made of carbon nanotubes and graphene that are replicated by the self-assembly of carbon taken out of the air. These AI will force us to put more carbon into the air. They will give us all air conditioners and make us run them all winter long. They will strip out our gas furnaces and put in coal-fired boilers. They will even make us all go around wearing electric blankets.
by ErikSMeyer
Once again, we read a singularitarian reveling in the idea that machines will eventually overcome, then replace mankind. I’d say that’s more than bad public relations, it’s a kind of madness.
I don’t believe in the singularity (machines currently have no independent intelligence at all, for all of their data processing and computation abilities), but if I did, I certainly wouldn’t glory in the idea that someday a mechanical form of intelligence will push humanity aside.
A great many people would support the idea of using the machines to transcend human limitations, guardedly (I certainly do); only a tiny fringe of misanthropic lunatics would support creating an intelligence, “immortal” or otherwise, to rule over and ultimately replace us.
When the general public suspects certain transhumanists of secretly wishing for that, well, it’s not such a secret.
by Tom
Though I disagree on the likelihood of a thinking, in some way concious machine apearing soon, I agree that welcoming our replacement is insane. Unless our replacement is our evolved selves.
Given the inevitable intelligence explosion when such machines do appear, I think I have to agree with Giulio Prisco that the only way we have a long-term future is by becoming the singularity ourselves.
We should therefore be pushing, as a species, for radical enhancement at the expense of independent artificial intelligences.
Please don’t think I have anything else against artificial intelligences, per say. Just that I don’t like the idea of them making us extinct in long, or more likely, very short order.
This is too much to risk on chance = business as usual techno-economic progress.
by John Middlemas
This whole theory is highly suspicious. If “Immortals” could start from this planet why not ages ago on some of the 1000 billion billion (10^21) star systems in the universe (nasa estimate). Immortals should already be all over the place and obviously detectable, but they are not. Kurzweil says we are probably alone but history shows we are never the “center” of anything despite the tempting belief.
by Gorden Russell
We couldn’t detect aliens using entanglement communicators.
by Tom
That could mean nothing more than we won’t be the first to blow/burn/grey-goo ourselves off the face of the universe.
by Whittaker
… but history shows we are never the “center” of anything despite the tempting belief.
This is the so-called “Mediocrity Principle”, which physicist David Deutsch had argued to be “not actually correct”. (http://en.wikipedia.org/wiki/Mediocrity_principle#Other_uses_of_the_heuristic).
BTW, there could be another grim scenario, in which every civilization in the universe believe in the existence of other civilizations, and instead of working hard to transcend their own limits, they stay home and wait for “communication”.
by Vin
Enjoyable article. I enjoyed the links too. Very informative, instructive and thought provoking.
Regarding the connectome debate, both agree the connectome approach should yield a comprehensive possibility space? Wouldn’t this space be amenable to the laws of physics like how a wave function of a quantum system leads to specifics? Then will not these specifics need to be cross-verified by the non-connectome determinations of them by the other debater?
So I say its a draw :D. But how the possibility space reduces to specifics for one connectome may yield a general framework for all connectomes – which could be useful to make AI’s?
by Bri
Honestly, I think a billion second plan is too optimistic. When the singularity hits, everything is up for grabs. AI will zoom past us by 2050. Some humans will become incorporated into Gaia, but many people will be uncomfortable with merging. I think humans will diversify to all sorts of environments. Some genetically, some robotically, some a mix of the two. I think it will be like a Precambrian explosion of diversity. What will be exactly human is hard to decide. On a normal evolutionary time scale, we probably wouldn’t be humans anymore, after a few hundred thousand years anyway. I think it will be so accelerated that it will be hard to define in a few hundred years. I still wouldn’t hold my breath that you can upload your spirit into a computer. Your mental state yes. Like a photograph it can be stored. Even it’s tendencies to think thoughts, but that doesn’t mean it will actually be the you that you experience internally. Only time will tell if I’m right or wrong, so it’s a matter of individual beliefs now. This whole discussion is like counting the number of angels on a pin. All conjecture.
by John
You sound so smart when sober. Agreed in 100%. Precambrian explosion of minds was new to me and eye opening.
by Bri
Interesting that you should say sober. I don’t really do drugs. At times in my life I have smoked pot and drank. I didn’t drink at all until college. After that it was mainly in social contexts. Friends in local bars. I don’t like losing my sense of the environment. Mushrooms are fun, but maybe five times in my life and that was college. I don’t even like medicine. I only take it when absolutely necessary. I never drink coffee. I do like playing up to peoples perceptions. I act very street as it were, but I’m really more of a geek or nerd. I just learned long ago to blend in. How to be engaging and disarming..
by Tom
Agree with you 100% about the futility of looking further ahead than the singularity as we perceive it at any point in time.
That’s what makes it a singularity, after all, isn’t it?
Anything after that is unknowable, so therefore, un-planable-for, by definition.
One might then say that the biggest existential threat is the singularity itself.
This is just another argument in favour of us having to become the singularity ourselves before the machines get there on their own and eat us in the process.
by Tom
And, wrt uploading, if a conciousness is merely an emergent property of sufficient complexity in some substrate, and assuming one could simulate that substrate at a fine enough resolution and to the same level of complexity, and perform a piece-by-piece transfer at that or better resolution, then uploading with a continuation of conciousness seems possible, to me.
It would be nothing more than we do on a daily basis with our own natural cell-renewal processes, new memories encoded/emerging, old memories deleted/fading, etc.
No-one has a problem with themselves remaining the same them, even though their contents are continually changing on many levels by natural processes.
by Jon
As Jay also said, this quote:
“You cannot make a (bio) neuron that spikes at 3 Ghz or that conducts neural impulses at 300 million meters per second. (The way to speed up a cheetah is not by strapping on a jet engine.)
I also reject Kurzweil’s premise that we will merge with AIs — that’s like merging that cheetah with a jet plane.”
—-
It’s an anoying thing to say, and it’s meaningless. It doesn’t -mean- anything to say that. It’s a fallacious appeal to absurdism, but he mentions no studies, data or information to prove Kurzweil wrong. Kurzweil, on the other hand, has cited dozens of studies and explains step by step how it could indeed work. To simply dismiss a scenario because it seems unlikely to you is unscientific, and it feels as if he barely understands Kurzweil’s ideas to begin with – the biological neuron itself is not sped up, the associated nano-hardware is – yet he feels free to criticize regardless.
I don’t know, it vexes me, and I wanted to rant about it. So there you go. I got it out of my system.
by Giulio Prisco
Not so fast Jon. Merging with AIs is a plan, not a prediction. Some agree with this plan, and some don’t.
by John
Jon was talking about difference in levels of argumentation. In this case, lack of it, which levels the whole post with fantasizing of any other kind. Maybe poetic, but not grounded for sure.
by Jon
@ John & Giulio Prisco
Basically exactly what John said. Thanks for clarifying that.
I fully accept that there are several ways to approach the (almost certainly coming) AI situation, and I think all of them are equally worthy of discussion. It was not my intention to insinuate otherwise.
by brainwave abduction
I think it happened a long long time ago. How removed was a hunter from his weapon and fire 50,000 years ago? We are way down this path, the rate is just exonential. We will have AI teachers able to manage us as individuals.
by Giulio Prisco
Great article, Bob. Some comments:
Re “Long-term, humanity (whether augmented, re-engineered, or uploaded) will be left in the dust by the machines, who will stand in relation to us as we to bacteria.”
This is, I believe, a common misconception. My view is different: I think humans and machines will co-evolve, blend and merge more and more deeply until it will be impossible to tell which is which. At some points humans1.0 will leave biology behind and upload, re-implement themselves in software running in more and more powerful substrates, and merge with native machine intelligences. The irreversibly mixed species of humans/machines will be “the immortals.”
What will they call themselves? Machines? Posthumans? Hybrids? I think they will just call themselves “humans.” And I think this is what _we_ should call them if we want to overcome the frequent hostility against our (allegedly) “inhuman” ideas. Humanity will not be left in the dust by the machines, because humanity will include machines, and the future human/machine hybrids will be the future humans.
I agree with the technical and programmatic points, but I think this terminology/PR point is important.
Instead of “machines will leave us in the dust,” I prefer to say “future humanity will leave us in the dust,” where by “future humanity” I mean future intelligences with mixed (biological) human and AI origin.
I don’t care about political correctness (I am proud of being totally unPC), but I do care about communicating positive, solar visions of the future that can facilitate acceptance of our ideas. That’s why I regard formulations like “machines will replace humanity” as a PR suicide.
My favorite analogy: children cease to be children and become adults, but this does not mean that children have ceased to exist as persons, it just means that they have grown up. Same for current biological humans and future persons. Perhaps future superintelligences, who will stand in relation to us as we to bacteria, will have a core personality subsystem derived from our grandchildren’s upload.
by Gorden Russell
There it is, right there, Giulio. Once we can have a processor inserted somewhere in the prefrontal cortex and go online to think with the speed of the cloud, we will be the AIs.
But imagine if the first person to get the prefrontal processor is an experienced and ruthless stock trader from Goldman Sachs. He will trade faster than anyone, matching the speed of all the algorithms now in use. But he will be thinking like a human, not just reacting according to a list of rules. He will quickly outwit all the algorithms. If he starts work at opening, by lunch he will own Goldman Sachs and by the closing bell he will own the entire market.
by Gorden Russell
Just think, that trader will get his mind into the cloud and he will know everything that is happening everywhere. He will become MacCloud in the closing scene of the film, “The Highlander.”
But don’t forget, “There can be only one!”
What if that trader is Mitt Romney?
He will open with a single million, pocket change for him. In his first minute of trade he will double his money, then double that in another minute, and so on and so on…until after 30 minutes he has over a million billion dollars. That’s ten to the fifteenth power.
By the time of the closing bell he will be over the multi-googleplex range. He will have more dollars then there are protons in the universe.
Then he can stage a hostile take-over of the Congress and the White House.
by YourNameHere
Amazing story. Thanks for sharing. I am thrilled.
by John King
LOL Gorden. The political theater during this decade does color our views. Remember that we live in the era of big administrative states; probably the seminal modern philosopher who most succinctly expresses the psychology and goals of this world is Thomas Hobbes. The world is presently run by perhaps 200 or so entities, mostly banks. So, the focus of your illustration is apt. We are seeing proto-AI’s most intense application in the military drone program; add to this the control of the global banking system, and that is where AI will find its base environment, not in the scientific projects that have the greatest aesthetic appeal to those of us who are afraid of losing our grant money.
I think Bob is basically right about entities with a unique and separate ontological basis taking up the torch (rooted in Western Renaissance and Enlightenment) that we in Olympic relay fashion have carried, while exhausting our “Civilizational”-ponzi scheme supports. Bodies that are “at home” outside the Terran biosphere cannot be fancy carbon — too expensive to maintain life-support.
Of course there will be some development of a cyborg class, as some have pointed out. Government agent positions might soon be filled only by cyborgs (for awhile).
I opine that the top-level AIs, in the course of their explorations and inquiiries, will encounter their superiors; that will be an interesting dialogue.
by Gabriel
THANK you Giulio, I’m happy somebody has the right mentality about it. The whole Us vs Them idea is just something perpetuated by hollywood…there is not going to even be a “them” – only an increasingly capable “Us”.
We can try to draw lines, but the simple truth is, we will never rationally be able to make a line and say “Okay, you are a human until you cross this”….No, that’s ridiculous – there is no such thing as a “Posthuman”, because transcendence (aka solving problems aka “playing God/human” etc) is inherently a human thing, perhaps the most human thing…
No matter how god-like we become (or how god-like we already are), we will always be Human beings…vastly more capable humans with more freedom and capability as time goes on, but humans nevertheless…..”Gabriel” didn’t just disappear because I suddenly bought a new car that can travel faster then I ever could, or I took new synthetic medicine to protect myself…why would this change because I decided to further better myself by enhancing my intelligence?
by Tom
Agree with almost everything expect that I would amend
“I think humans and machines will co-evolve, blend and merge more and more deeply until it will be impossible to tell which is which.”
to
“I think humans will have to co-evolve with machines, blend and merge more and more deeply until it will be impossible to tell which is which, or else we’ll rapidly go extinct.”
by A4i
To me, the best possible outcome is planet Earth to be preserved for another intelligent species to emerge from animal kingdom. Humans have to clean all traces of their presence and abandon Earth. So better leave some resources in the ground for future intelligent beings to fuel their own civilization. All that means – we have to plan for growth in Space and restrict activities on Earth, because current exponential expansion is causing planetary mass extinction. Space is extremely harsh and unforgiving environment. Better plan for lots of resources per every human and not for lots of humans and lack of resources .
by Jay
“At present, a new drug typically requires at least ten years and a billion dollars to develop. Frustratingly slow…”
Just because it’s always been like that, doesn’t mean it’s always going to stay like that.
“I also reject Kurzweil’s premise that we will merge with AIs — that’s like merging that cheetah with a jet plane.”
Just because we can’t do it now, doesn’t mean we won’t ever be able to do it. Besides, merging a cheetah with a jet plane is a silly idea. Nobody has ever seriously thought of doing it, and nobody (probably) ever will. The merger of humans and technology, however, is already well under way.
by Samcon
I agree Jay. Blum does not explain why he thinks we will not merge with our technology.
by Avlin
The cheetah is just a humorous analogy. Blum does give two good reasons why the merger won’t happen–or specifically, why it won’t be the BEST design:
1. Biological design is inflexible as a result of commitments made over billions of years.
2. Experimentation on humans is severely hampered by government regulation (and rightly so.)
Do you really believe that the future masterminds will involve DNA/protein, or do you just want to believe that?
by blair
It was an amazing article, but your points were pretty much my exact thoughts.
by Marcos Marin
He certainly don’t watch enough cartoons! even low quality power rangers had this cheetah jet plane gig! easy peasy! Cartoon Network is more scientific than this =)