Foreword to ‘Dark Ages II’ (book by Bryan Bergeron)

July 26, 2001 by Ray Kurzweil

Our civilization’s knowledge legacy is at great risk, growing exponentially with the exploding size of our knowledge bases. And that doesn’t count the trillions of bytes of information stored in our brains, which eventually will be captured in the future. How long do we want our lives and thoughts to last?

Originally published September 28, 2001. Published on KurzweilAI.net prior to book publication on July 26, 2001.

My father was one of those people who liked to store all the images and sounds that documented his life. So upon his untimely death at the age of 58 in 1970, I inherited his archives which I treasure to this day. I have my father’s 1938 doctoral dissertation at the University of Vienna containing his unique insights into the contributions of Brahms to our musical vocabulary. There are albums of neatly arranged newspaper clippings of his acclaimed musical concerts as a teenager in the hills of Austria. There are the urgent letters to and from the American music patrons who sponsored his flight from Hitler just before “Krystalnacht” made such escape impossible. These items are among dozens of aging boxes containing a myriad of old remembrances, including photographs, musical recordings on vinyl and magnetic tape, personal letters, and even old bills.

I also inherited his penchant for preserving the records of one’s life, so along with my father’s boxes, I have several hundred boxes of my own. My father’s productivity assisted by the technology of his manual typewriter and carbon paper cannot compare with my own prolificacy, aided and abetted by computers and high speed printers which can reproduce my thoughts in all kinds of permutations.

Tucked away in my own boxes are also various forms of digital media: punch cards, paper tape reels, and digital magnetic tapes and disks of various sizes and formats. I often think about just how accessible this information remains. Ironically, the ease of approaching this information is inversely proportional to the level of advancement of the technology used to create it. Most straightforward are the paper documents, which although showing the signs of age, are imminently readable. Only slightly more challenging are the vinyl records and analog sound tape recordings. Although some basic equipment is required, these are not difficult items of equipment to find or use. The punch cards are somewhat more difficult, but it’s still possible to find punch card readers, and the formats are uncomplicated.

By far, the most difficult information to retrieve is that contained on the digital disks and tapes. Consider the challenges involved. For each one, I have to figure out exactly which disk or tape drive was used. I then have to recreate the exact hardware configuration from many years ago. Try finding an IBM 1620 circa 1960 or Data General Nova I circa 1973 with exactly the right disk drive and controller, and you’ll quickly discover the difficulties involved. Then once you’ve assembled the requisite old equipment, there’s layers of software to deal with: the appropriate operating system, disk information drivers, and application programs. Then just who are you going to call when you run into the inevitable scores of problems inherent in each layer of hardware and software? It’s hard enough getting contemporary systems to work, let alone systems for which the help desks were disbanded decades ago. Even the Computer Museum, which used to be located in Boston, has been disbanded, and even when it was in business, most of the old computers on display had stopped functioning many years earlier.

Assuming that you prevail through all of these obstacles, the actual magnetic data on the disks has probably decayed. So even if we assume that the old hardware and software that you assembled are working perfectly, and that you have aging human experts to assist you with perfect recall of long since obsolete equipment, these old computers would still generate mostly error messages.

So is the information gone? The answer is: not entirely. Even though the magnetic spots may no longer be readable by the original equipment, the faded magnetic regions could be enhanced by suitably sensitive equipment using methods that are analogous to the image enhancement often used on images of the pages of old books. So the information is still there, albeit extremely difficult to get at. With enough devotion and historical research one might actually retrieve it. If we had reason to believe that one of these disks contained secrets of enormous value, we would probably succeed in recovering the information. But the mere motivation of nostalgia is unlikely to be sufficient for this formidable task. I will say that I did largely anticipate this problem, so I do have paper print outs of most of these old files. Invariably, that will be how I solve this problem. The bottom line is that accessing information stored in digital form decades (and sometimes even just years) later is extremely difficult if not impossible.

However, keeping all our information on paper is not the answer. Hard copy archives present a different problem. Although I can readily read even a century-old paper manuscript if I’m holding it in my hand, finding a desired document from among thousands of only modestly organized file folders can be a frustrating and time consuming task. It can take an entire afternoon to locate the right folder, not to mention the risk of straining one’s back from moving dozens of heavy file boxes from one stack to another. Using the more compact form of hard copy known as microfilm or microfiche may alleviate some of the problems, but the difficulties of locating the right document remain.

So I have had a dream of taking all of these archives, scanning them into a massive personal data base, and then being able to utilize powerful contemporary search and retrieve methods on the hundreds of thousands of scanned and OCR’d (Optical Character Recognized) records. I even have a name for this project: DAISI (Document And Image Storage Invention), and I have been accumulating the ideas for this little venture for many years.

DAISI will involve the rather formidable task of scanning and OCR’ing hundreds of thousands of documents, and patiently cataloguing them into a data base. But the real challenge to my dream of DAISI is the one that Bryan Bergeron articulates so eloquently in this volume, namely how can I possibly select appropriate hardware and software layers that will give me the confidence that my archives will be viable and accessible decades from now?

Of course my own archival desires are a microcosm of the exponentially expanding knowledge base that the human civilization is accumulating. It is this shared species-wide knowledge base that distinguishes us from other animals. Other animals communicate, but they don’t accumulate an evolving and growing base of knowledge to pass down to the next generation. Given that we are writing our precious heritage in what Bergeron calls “disappearing ink,” our civilization’s legacy would appear to be at great risk. The danger appears to be growing exponentially along with the exploding size of our knowledge bases. The problem is further exacerbated by the accelerating speed with which we turn over to new standards in the many layers of hardware and software needed to store information.

Is there an answer to this dilemma? Bergeron’s insightful volume articulates the full dimension of the problem as well as a road map to ameliorating its destructive effects. I will summarize my own response to this predicament below, but first we need to consider yet another source of knowledge.

There is another valuable repository of information stored in our own brains. Our memories and skills, although they may appear to be fleeting, do represent information, stored in vast patterns of neurotransmitter concentrations, interneuronal connections, and other salient neural details. I have estimated the size of this very personal data base at thousands of trillions of bytes (per human brain), and we are further along than many people realize in being able to access this data and understand its encoding. We have already “reverse engineered” (i.e., scanned and understood the methods of) several dozen of the hundreds of regions of the brain, including the way in which information is coded and transmitted from one region to another.

I believe it is a conservative scenario to say that within thirty years we will have completed the high resolution scan of the human brain (just as we have completed today the scan of the human genome) and will have detailed mathematical models of the hundreds of information processing organs we call the human brain. Ultimately we will be able to access and understand the thousands of trillions of bytes of information we have tucked away in each of our brains.

This will introduce the possibility of reinstantiating the vast patterns of information stored in our electrochemical neural mechanisms into other substrates (i.e., computational mechanisms) that will be much more capable in terms of speed, capacity, and in the ability to quickly share knowledge. Today, our brains are limited to a mere hundred trillion connections. Later in this century, our minds won’t have to stay so small.

Copying our minds to other mediums raises some key philosophical issues, such as “is that really me,” or rather someone else who just happens to have mastered all my thoughts and knowledge? Without addressing all of these issues in this foreword, I will mention that the idea of capturing the information and information processes in our brains has raised the specter that we (or at least entities that act very much like we do) could “live forever.” But is that really the implication?

For eons, the longevity of our mental software has been inexorably linked to the survival of our biological hardware. Being able to capture and reinstantiate all the details of our information processes would indeed separate these two aspects of our mortality. But the profound implication of Bergeron’s Dark Ages II is that software does not necessarily live forever. Indeed there are formidable challenges to it living very long at all.

So whether information represents one man’s sentimental archive, or the accumulating knowledge base of the human-machine civilization, or the mind files stored in our brains, what can we say is the ultimate resolution regarding the longevity of software? The answer is simply this: information lasts only so long as someone cares about it. The conclusion that I’ve come to with regard to my DAISI project, after several decades of careful consideration, is that there is no set of hardware and software standards existing today, nor any likely to come along, that will provide me with any reasonable level of confidence that the stored information will still be accessible (without unreasonable levels of effort) decades from now. The only way that my archive (or any one else’s) can remain viable is if it is continually upgraded and ported to the latest hardware and software standards. If an archive remains ignored, it will ultimately become as inaccessible as my old 8 inch disk platters.

In this pioneering work, Bergeron describes the full dimensions of this fundamental issue, and also provides a compelling set of recommendations to preserve key sources of information beyond the often short-sighted goals underlying the design of most contemporary information processing systems. The bottom line will remain that information will continue to require continual maintenance and support to remain “alive.” Whether data or wisdom, information will only survive if we want it to.

We are continually recreating our civilization’s trove of knowledge. It does not simply survive by itself. We are constantly rediscovering, reinterpreting, and reformatting the legacy of culture and technology that our forbears have bestowed to us. We will eventually be able to actually access the vast patterns of information in our brains, which will provide the opportunity to back up our memories and skills. But all of this information will be fleeting if no one cares about it. Translating our currently hardwired thoughts into software will not necessarily provide us with immortality. It will simply put the means to determine how long we want our lives and thoughts to last into our own figurative hands.

Dark Ages II