Robin Hanson’s “The Age of EM”

Work, Love, and Life when Robots Rule the Earth

In summary: Hanson speculates on a world where most or all people are replaced by digital simulations, ‘em’s, of real people who know that they are ems. The emulation is presumed to be faithful and personalities are little changed. Sensory and motor nerves are modified mostly in support of VR. Lakoff would object.

Hanson uses tools of an economist, an engineer, and a few more, to make many conclusions and guesses. He has thought thru many causal chains for a fairly plausible new world view. People run at different rates, depending on wealth, wishes and importance. There is a lot of work to do and emulation is not cheap. Their subjective habitat is mostly VR.

P 5: “Farming environments changed faster than genetic selection could adapt, and the industrial world changes faster than even cultural selection can adapt.”

P 8: “By the way, feel free to skip around to the sections that interest you; only rarely do they depend much on previous sections.” Thank you. Hanson speculates on dates and ems vs. AIs on page 347 in chapter 27.

P 21: There is a section titled “ERA VALUES” which I find opaque. That says more about me than the section. I have no considered opinions on community values. Hanson quotes many scholarly articles to support his claims. I shall have to see how those claims are used to support his conclusions. About my only opinion is that the American economy and politics perform well because Americans mostly govern themselves politely. We do well because we have a democracy but we still have a democracy because we are civil to each other. The advanced economies all do this.

P 27: Section: LIMITS:
Hanson uses the traditional notion of “the economy”. Well, he is an economist. I think we are even today veering away from the physical notion of economy. Google produces great wealth but ships very few goods. The wealth they produce is not constrained by the limits Hanson describes. Each year we produce far more wealth than is counted in our GDP. If mana from Heaven decreases out GDP and that is bewailed then GDP is no longer a figure of merit.

Hanson goes on to mention searching for algorithms, which is sort of like what I suggest above. I think the mathematicians will be amused indefinitely. Hanson explores ideas such as these and dismisses them as providing limited social value.

P 28: Hanson goes onto a crypto Malthusian perspective, with plausible logic. Yet there are many games to play with 1090 particles. Energy is a bigger problem.

P 49: Complexity:
Hanson makes several relevant points about understanding the brain and how that limits us from redesign. If you measure complexity by size of “source code” in bits however, there are several software projects today whose size exceeds our brain DNA. But no one claims to understand such software except in small pieces. Such software indeed displays the inertia that Hanson remarks on.

Another path is learning the DNA language and brain morphogenesis. There are many details, but not many fundamental ideas to be discovered. Then we can improve the brain at least regarding birth canal limitations.

P 51: “A copy can be made at any time of any emulation, after which the two versions diverge because of differing inputs and random fluctuations.” I would imagine deterministic execution but that might not be a good engineering choice. The two versions would have very large mutual information for a substantial time. That bears on cost of storage, perhaps.

P 52: “(Our focus here is on AI that does human jobs well and not passing a “Turing test”.)” Agreed. Such a category includes almost all industrial software since commercial computers appeared in 1953. Computing orbits was once a human occupation—no more. McCarthy lamented that many AI challenges were met only to be relegated to non-AI. Still they are important progress. Neural nets are a new thrust.

There are fascinating reports here on estimates from experts on the arrival of AI and “general AI”. My own non-expert opinion is that general AI will remain elusive because there are many components to it some of which are not profitable to pursue. I think that many think that there is some silver bullet that once found, will make these many components easy, maybe even free. I think they evolved separately and share little mechanism. The very noun phrase “general AI” may mislead. I would be mightily impressed even with general puzzle solving.

Hanson makes similar points on the next two pages. Mixtures of neural nets and conventional software will play a role here. AlphaGo resorts to combinatorial logic to evaluate a board position with a potential ladder. This logic is provided by conventional software. The heavy (novel) lifting is in hardware neural net ASICs.

P 56: “That is, even when ems can read each other’s minds, they may pretend that they cannot.” Just like today.

P 61: Hanson describes several security issues well. I shall study them. Some apps today refuse to work unless they can ‘phone home’. A logic ‘crypto module’ might be strategic. Such would be subjectively (to the em) like a smart phone but with less chance of loosing it.

P 64: Hanson makes many comments about speed-power tradeoffs that seem about right. I shall be nit-picking them. A given em changes state slowly in comparison with its total state. I hope it is feasible to exploit this ratio when saving and transmitting this state. The relation is between processing and memory is not clear to me. That relation has not much changed in von Neumann architectures but will be very different here.

It is a bit more complex than that. There are two costs: capital hardware cost and power. Hardware cost depends on some complex monotonic function of the speed. The subjective minute takes more energy to run fast. The theoretical limit is proportional to speed and the power is the square of the speed. Some flexible hardware now approaches this limit over a range of speeds. The hardware to do the job efficiently in a day is much different from the hardware to do it in a second.

The next page covers this pretty well.

P 65: Brains sleep. Perhaps processing needs are different then. Perhaps we have specialized hardware for sleeping.

P 94: Hanson avoids the common economics pitfall of assuming an equilibrium. He describes the em city more nearly like an explosion than a static configuration.

P 96: Hanson describes consumer demand for variety and the impact of that on production. I agree that ems will demand variety but in what? An em owns data structure stored in real physical memory some where. An em presumably buys electricity the cost of which varies erratically. An em will rely indirectly on a great deal of infrastructure which will surely be multiplexed among many ems, presumably by a corporation. An em will be ‘surrounded’ by virtual reality and someone may figure out how to produce and charge for pleasant virtual surroundings.

An em won’t need a car or a house, or a fancy dinner, except virtually. Hanson has not gotten to the ‘economy’ yet that describes the em’s money flow. I think he is working up to that.

P 104: Hanson describes a group ‘undo’ facility to overcome faux pas etc. Some of the obvious difficulties are explored. I can imagine a movie script already. I think that a perpetual count of how often each participate pushed the button is warranted.

P 113: Hanson raises the issue of giving ems authentic access to the real world. On page 116 protocols are proposed to detect deception. Keykos takes a different approach here but does not deal with ems, yet. This idea arose here in the book. A similar pattern is explored much more extensively beginning on page 172 where quantified information leakage is suggested, like the data muffler.

Law enforcement today wants to be able to see into your smart phone, with a warrant, of course. It has been told that it cannot see into your brain. I would like to be able to do public key crypto in ‘my head’ and keep the long keys there and without law enforcement access even with a warrant. This will be a contentious issue.

P 125: Hanson supposes law enforcement with ‘root access’ at least regarding visibility. It is hard to draw the line so that private keys and ordinary memories are concealed. I think I fear such law enforcement more than lack of such enforcement. I am a 2nd rate anarchist.

P 129:

Sometime in 1999, I recall a cartoon set in a very modern medical setting and a dazed person ‘waking up’. The doctor says: I understand that you know Cobol. There is a calendar on the wall with date 9999.

P 132: Hanson’s ‘ghosts’ are a fun and convincing metaphorical exercise. You can bid such a ghost to resume ‘reality’ if you and it wants.

P 134: Regarding death: I suspect that there is some fear-of-death in our DNA but much more in our culture. My subjective feeling is a wish to preserve my memories in some meaningful way; I am ‘vested in them’. Thus my web site. Old people, especially men, commonly have an urge to tell stories and ‘explain things’, which I suspect has an evolutionary explanation. I have an Alcor ‘neuro’ contract as another hedge.

In the object world we normally see two components: behavior and state. The line between the two if fuzzy, yet the distinction is important. The same goes for ems where the distinction is much fuzzier. The concept of ‘mutual information’ is important here in estimating cost of preserving objects and ems. Here I mean the concept of mutual information as a data compressor would use it. Recent clones or spurs of an em would not require many bits to keep. I figure a good enough connectome requires about 250 bytes. Intel stops at 248 just now but they have room to grow. This promises a new customer every few seconds. Many in the AI world believe that merging learning is difficult in the current neural net paradigm. My note on rats forming memories suggests the same problem. My presumption is that eventually memory formation might be somewhat reverse engineered whereupon these important things might happen: A Smalltalk universe has the same problems for the same reasons. It is also related to applications with linked lists utilizing multiple processors; it can be done but has a bad reputation.

P 136:

If I were trying to prove a math theorem I would like to spawn a spur to try to prove a lemma. If it succeeds then the ‘original’ cedes to the spur. Many tree search schemes work like this.

P 154: Hanson presents and adopts the Malthusian perspective. I have not understood organized descriptions of why we seem not to be trapped in the Malthusian vice today. Vaguely it seems to be related to the fact that we have an unstable economy. Economics models favor stability, since Malthus and earlier. The book’s model is mainly one of stability, but with some rate of growth. Hanson uses a time unit: ‘economic doubling time’ and suggests that intellectual property typically looses value proportionally, due to obsolescence. This allows some reasoning in an unstable era. The opposite to stability is the ‘Singularity’ perspective which abjures questions of ‘what then’. We live in exiting times, already. (Hanson deprecates the Singularity on page 347.)

P 181: Beginning in the section “New Institutions” Hanson has fun suggesting outrageous institutions. I like most of them. That they would be more likely in em-world is still a bit weak.

P 183: “Very secure and anonymous communications between willing parties can be arranged via “public key cryptography” wherein each person publishes a public key for which they can prove that only they know the matching private key.” With ordinary brain function you can’t easily prove that you have not told others the private key. With ordinary brains even remembering a private key is difficult. A very small auxiliary crypto unit closely integrated with the em would solve these problems. Something rather like a wired in smart phone would solve quite a few related problems. Police authorities tend to object unless they can subpoena the content of these.

P 185: I am glad to see a current concise description of Hanson’s “Combinatorial Auctions”. I think they are promising. I have pondered some of these issues.

Ditto “Prediction Markets”. Speaking of hyperdisasters, ems could establish an outpost on Mars, or L5 with enough technology to slowly recover if the Earth were wiped out. This would be a much lower cost than sending and supporting bios.

I think it would be fun to go explore and mine the asteroids as an em. All those clumsy and fragile space suits would stay behind. Life support systems should be much cheaper and reliable. Digital backups would alleviate many worries. Hanson speaks briefly of space travel on page 225. He foresees some but not much.

P 220: “City Auctions” is an interesting proposal for managing a city which is probably superior to today’s practices which involve a great deal of inflexible and time consuming regulation. It would be fun to explore this as a game and contrast it with ordinary property ownership.

P 224: Fast ems find real travel problematic. Sure, they could pause during travel but they go to the expense of being fast because they are in a hurry. For those it would be a throwback to the days when it took a few months to visit Paris.

P 229: Punishment: Prompt termination upon clear evidence of misdeed is rather effective. Lesser malfeasance would suggest life with greatly diminished authority.

P 230: Hanson suggests that clades will be like a single organism in many ways including legal. I think the logic is correct. This is a throwback of a few thousand years. It bears on institutions such as democracy, perhaps negatively.

P 243: Under ‘Conflict’ Hanson seems to implicitly assume that inequality is necessary and sufficient for conflict. I grant that there is some causality there. Plain greed and crime are other conflict sources.

I suspect that large clans will be less common than Hanson assumes. Clan specialties for which large numbers of members are needed to do the job, are just those most subject to automation, of the AI sort—the sort we do today. I have a higher estimate than Hanson of how much diversity there will be in the tasks. There will be exponentially growing space of specialties which will require exponentially large space of training. I am not sure how those changes impact Hanson’s conclusions.

P 257: During meetings of ems of normally different speeds, it may be mutually beneficial to run at more nearly the same speed.

P 260: Hanson addresses totalitarian thrusts— their strengths and weaknesses. Even more than in today’s world this bottoms out in fabrication of infrastructure. It takes, and will take large resources to build hardware for many ems. Not quite so many resources for modest numbers. How to build such hardware might conceivably be a well guarded secret—a practical monopoly. The ability to ‘shallow mind read’ depends greatly on technical details of such infrastructure. In the mind games I play, in light of the digital and logical facts, the defense has the advantage over the offense leaving the world polycentric. While crypto is hard, you can build good crypto.

Hanson speaks well of ‘market speculators’. I like market speculators but they rely on some notion of property rights. In Jane Jacobs’s terminology the guardians must succeed for the merchants to prosper.

“To use decision markets to get good advice, however, such regimes would need to involve outside judges and financial institutions, and allow informed traders to make secret trades and to hold assets outside of the regimes reach.”

P 263: Hanson speaks of governance of clans. I am a non-joiner. I know quite a few other non-joiners. I think I have worked well in companies and enjoy working with people who are different from me. If I were to spawn a clan I imagine it would soon fracture.

P 268: On fashion: Even today in VR a participant has much control on his appearance. That will play a significant role for ems.

Close clan members can do good movie reviews for each other!

P 343: Hanson discusses tinkering with the simulation so as to modify what is wanted or valued. My first reaction is that I would not want that done to an emulation of me. On further consideration it is less than entirely evident. It is a conundrum.

P 348:

This is sometimes called the “silver bullet” AI hypothesis. I agree with Hanson that there is none; not that it is unknown but that it does not exist. It took a few million years to produce human intelligence Darwin’s way, in presence of high evolutionary pressure. And that part is only the part by which we exceed the other primates. We cannot even build an artificial monkey brain yet. That took at least 20 million years. The good news is that the results are recorded in machine language (our DNA) which is smaller than some of our current commercial software projects. It might take 100 years to reverse engineer it but I would guess at least 50 years. Some estimate that like our commercial software it is highly redundant. We will learn to do this rather slowly. I suspect we must grok morphogenesis in the process of understanding our DNA. That is stuff we need to do anyway. I agree with Hanson: ems will come first.

P 363: Hanson quotes Ventura and Voth claiming that old wealth in England failed to invest in industry and lost out. I agree. I have not read that paper but in Institutional Evolution Allen claims that the aristocracy felt that their wealth stemmed from fealty to the king and that the merchant class was a status they could not aspire to and remain in the aristocracy. These values were on display in the movie about Beatrix Potter whose parents were not rich, but aspired to aristocratic values. When Beatrix wanted to invite a potential publisher, who was actually rather more wealthy than the Potters, to visit, her mother objected with the comment that the publisher was a ‘tradesman’ and might soil the furniture. This was rather a cartoon depiction but indicated a dichotomy; wealth is one thing, but status is something else. The aristocrats would loose status if they increased their wealth thru commerce. Their fealty to the king would come into question. Ironically the Potters were well enough off thru merchant ancestors, something that Mrs. Potter wanted to hide.

P 382: In the ‘Finale’ chapter Hanson lists suggestions by others who think some other social transition will be more important than ems. The collection is collectively hilarious, even if thoughtful.

The only scenario that bears on this vision that I give some thought to is a decrease of market value of the skills of an ever larger portion of our population. Ditch diggers are gone. etc, etc.

dense information
Future shock for milli-ems.

Often severe political problems are solved by death. Stalin comes to mind. (see P 259)

Costs: (a framework)
I suggest some physical commodities necessary to support ems. I don’t know what they will cost, but that they will cost something.
I suspect that production of ordinary software will continue to grow in the em world, continuously from where it is now. A class of long lived (subjectively) software will emerge and this will be aided by specialists in that software, frozen until needed. A new class of work will emerge that tries to make this software comprehensible to those who need it. My suspicion is that today software is too often rewritten because of the difficulty in finding extant software that already does the job. Too often today computer languages are choses from among those few that you already know, almost certainly missing the language best for the job.