There is a terrible mismatch between the name of the book and its contents. It could have rightly be called "A Physicist's view on the Future of AI" without anybody complaining. Here's a short summary and I'll let you judge yourself:
The book starts with a fictional scenario where a company, after many years of AI research, succeeds in developing not just human-level but a superintelligent AI (Prelude). He begins by explaining some of the most basic elements necessary to understand a superintelligent AI; intelligence, memory, computation, and learning (Chapter 2: Matter Turns Intelligent). Then the writer goes on (wildly) imagining many sorts of scenarios in the immediate and distant future (Chapter 3, 4 and 5 - Near Future, Intelligence Explosion and Aftermath). In a sufficiently distant future, when superintelligent AI has grown not just on Earth but on a cosmic scale, it becomes important to make most of your resources and gaining resources through cosmic settlements, thereby expanding our civilization (Chapter 6 - Our Cosmic Endowment). At this point, the book becomes more scientific than science fiction, turning its attention to the 'goals'. Again, the author talks about the origins of the goals from the physics perspective and how they are evolved by biology and outsourced by engineers. This discussion leads to the goals of AI (Chapter 7 - Goals). The last chapter is about, as the author puts it, one of the thorniest philosophical topics of all: consciousness. He basically tackles one of the mother-of-all questions; if every object in the universe is just a bunch particles arranged in some patters, what makes some patterns conscious and others not? (Chapter 8 - Consciousness)
Roy Amara, cofounder of Institute for the Future said,
"We tend to overestimate the effect of a technology in the short run and underestimate the effect in the long run."
There is a lot wrapped up in these 21 words. An optimist can read it one way, and a pessimist can read it another.
But Max Tegmark does a wonderful job of not underestimating the effect of AI in the long run. He, in fact, streaches the possibilities offered by a superintelligent AI that knows only the bounds of the physics. Future scenarios projected by him looks almost science-fictional. But I must say, I found the chapters concerning these scenarios (Chapter 3,4 and 5) to be the most boring. Why? Because they have a little to do with "Being Human in the age of AI". This book started out with great promises, became stagnated in the middle portion and, thanks to the last two chapters, rekindeled my interest in this book.
One of the most severe criticism I can think of is that it is pointless to have a discussion on the superintelligent AI at this point. By no means I'm saying that one should not discuss this issue, all I'm saying is that not to this point (thinkning about the effects of an AI 10,000 into the future is senseless because we don't know, with any certainty, how the AI will turn out to be.) For current day and age, it makes sense to have dicussions on privacy, security, narrow AI and to some extent, AGI.
Nevertheless, this book introduced to me to many new ideas. Some of the most interesting ideas are:
1) Heisenberg's Uncertainty Principle forbids anything from being completely boring and uniform.
2) Classification of Life: Life 1.0, Life 2.0 and Life 3.0
3) Idea of Substrate Independence: Information can take life of its own, idependent of physical substrate.
4) Quantum Computers share information with huge numbers of versions of themselves throughout the multiverse. (controversial idea)
5) Hebbian Learning: Fire Together, Wire Together (in the context of Neurons)
6) If you're a robot, life itself can be viewed as a game.
7) As technology grows, we should become more proactive than reactive in our approach to safety engineering.
8) The idea of Cosmic Claustrophobia and Intergalactic Wanderlust.
9) In a cosmos teeming with superintelligence, alsmot the only commodity worth shipping long distances will be information.
10) What feels like goal oriented behavior can emerge from goal-less deterministic laws of physics.
11) Goal Oriented Design versus Goal Oriented Behaviour.
12) Teleological view: Our Universe keeps getting more teleological.
13) What particle patters are consciousness? Emergent behavior and Integrated Information
14) If information processing itself obeys certain principles, it can give rise to higher level emergent phenomenon that we call consciousness.
15) The space of possible AI experiences is huge as compared to what humans can experience.
16) Nested heirarchy of consciousness at all levels from microscopic to cosmic.
17) Which one is correct? "I asked her on a date because I really liked her" or "My particles made me do it by moving according to the laws of physics"?
18) It’s not our Universe giving meaning to conscious beings, but conscious beings giving meaning to our Universe.
19) Dyson: Universe was pointless, life is now filling it with ever more meaning, with the best yet to come if life succeeds in spreading throughout the cosmos.