Superintelligence by Nick Bostrom

Superintelligence: Paths, Dangers, Strategies by is a thought-provoking book. The author is a philosopher who has a strong technical background in information technology and science. This work is devoted to the future development of machine intelligence and the potential dangers relating to its development.

First, since this book goes into territory that might initially seem far-fetched, and because “doomsday is coming” type books often lack credibility, I think it is important to discuss some reasons why I believe that Bostrom’s ideas hold some authority. This book is extremely well researched and the author has a strong technical grasp of the relevant subjects. He does not make unsupported contentions. Though a philosopher, Bostrom seems to have a scientific mind. He often speaks in terms of probabilities, not predictions or certainties. He does do a lot of speculation here, but he is careful to point out when it is speculation.

Furthermore, I have done a little research around this book. This work, as well as some other books and essays that have been published in the past several years, has garnered serious attention from a host of scientific, technical and industry experts. Stephen Hawking and Bill Gates are some of the more famous and prominent people who are looking at these issues as credible. In the past several years, these concerns have sparked a conversation among those in the know as it relates to this topic. Some are planning for what may be momentous events related to the possible coming of super intelligence.

Parts of this book are technical, and some of it was bit difficult for me to follow. It helped that I have a passing interest in this topic and in related issues and that I have occasionally read articles on the subject. In addition, the author references various writings and ideas relating to human consciousness and brain function. I have also done a little reading on those subjects, which proved helpful. Nevertheless, a patient lay reader without much background will mostly understand and get a lot out of this book.

In order to comprehend what this work is about, it is important that one understands several important concepts that Bostrom explores. First, this book is mostly concerned with what Bostrom calls general or strong artificial intelligence, or in layman’s terms, computers that will be able to think like humans in multiple ways. This is as opposed to systems that are currently labeled as artificial intelligence that are very good at accomplishing specific tasks, but only specific tasks.  This type of less sophisticated technology is already in use in all sorts of applications, including Internet search engines, navigational aides, medical application, etc.

General artificial intelligence, by definition, will initially be roughly on par with, or slightly superior to, human intelligence. As Bostrom points out, most of the film and book depictions of machine intelligences fall into this category.

Artificial intelligence is not synonymous with  “Suprintelligence.” Though Bostrom spends many pages explaining what he means by Suprintelligence and offers several definitions, it is basically intelligence that is far advanced of human thinking in every important area.

The author writes,

“The magnitudes of the advantages are such as to suggest that rather than thinking of a superintelligent AI as smart in the sense that a scientific genius is smart compared with the average human being, it might be closer to the mark to think of such an AI as smart in the sense that an average human being is smart compared with a beetle or a worm. “

Bostrom’s contentions are as follows: sometime in the next fifteen to ninety years, researchers will likely produce strong artificial intelligence. There will be constant improvement of this artificial intelligence, either through the efforts of its programmers or, more likely, it will be a self-improving system. This improvement will eventually create what Bostrom describes as an “intelligence explosion.” This will be a leap in intelligence of unimaginable magnitude. Suprintelligence will result.

This Suprintelligence will have an overwhelming advantage over the whole of human civilization. Bostrom explains how it might gain easy access to enormous financial resources and manufacturing faculties. Such an entity might be powerful beyond human comprehension. It will likely be driven to expand its intelligence further and further. Such expansion efforts could possibly bring about human extinction. If the results do not end up as dire as the end of humanity, Suprintelligence will at least have an enormous impact on the future of our species and civilization.

Bostrom speculates about many scenarios. Many of the most likely involve a Suprintelligence with seemingly godlike powers. In some of these scenarios, in its drive to become smarter and bigger, the Suprintelligence might begin changing the ecosystem of the Earth as to make human life impossible. Of one of these horrifying possibilities Bostrom writes,

“if the AI is sure of its invincibility to human interference, our species may not be targeted directly. Our demise may instead result from the habitat destruction that ensues when the AI begins massive global construction projects using nanotech factories and assemblers— construction projects which quickly, perhaps within days or weeks, tile all of the Earth’s surface with solar panels, nuclear reactors, supercomputing facilities with protruding cooling towers, space rocket launchers, or other installations whereby the AI intends to maximize the long-term cumulative realization of its values. Human brains, if they contain information relevant to the AI’s goals, could be disassembled and scanned, and the extracted data transferred to some more efficient and secure storage format. “

Though the above seems fantastic, the author bases his speculations upon what are some educated guesses of modern scientific and technical minds as to what technologies are likely to be developed in the future.

Bostrom discusses many possibilities. Some involve human extinction. However, there is an entire range of eventualities are explored. Some involve less maleficent ills, such as a Suprintelligence dominating humanity in more benign but still stiffening ways. Other scenarios are bright, with a passive Suprintelligence helping humanity to avoid extinction and promoting human improvement.

But the author warns,

“Before the prospect of an intelligence explosion, we humans are like small children playing with a bomb.”

The author digs into a lot of detail involving the current state of artificial intelligence, its technical evolution, the revolution that is likely to occur after strong artificial intelligence is developed, as well as post Suprintelligence scenarios.

Bostrom does devote a lot of thought to solutions. He concedes that the development of Suprintelligence is likely inevitable. However, he explores numerous possibilities as to how it can be developed in order to avoid pernicious outcomes. The author digs deep into both technological as well as philosophical issues as they relate to creating favorable outcomes.

There is a lot more to this book than my summary does justice. Bostrom has a keen mind and takes the reader down all sorts of interesting scientific, technological and philosophic paths.

I think that it is important to remember that those who have attempted to forecast the future, even those who are knowledgeable, have often been proven wrong. However, based upon the serious and hardheaded way that this topic is explored, and based upon the fact that these concerns are being given serious consideration by bright people who understand these subjects, these ideas need to be carefully considered. There is much to ponder here.

I found both the hard technology as well as the predictive and philosophical musings contained within these pages fascinating. At the very least it is an excellent primer on the state of artificial intelligence research and its future development. I strongly recommend this book for anyone interested in this type of technology, the future of humanity, or the state of the world in general.

Nick Bostrom’s website is full of interesting thoughts on the ethics, science, philosophy, the future of humanity and all sorts of other topics. It is here.