Who will rule the Earth in 2100?
Scenarios for 2100, with my probability guesstimates:
The shift from (1) to (2) or (3) might happen—or at least get baked in—within 20 years. I guess 10-30% chance of that.
The short story for (2) goes: the capabilities of AI systems surpass those of humans; AI systems pursue their own values.
The main ways that (1) might continue:
(20%) A global agreement prevents the development of the most dangerous AI systems. 4
(10%) A global disaster sets back the development of AI systems (along with much else) by decades. E.g. a major nuclear war.
(<10%) A technological barrier prevents superhuman AI before 2100.
How might we get (3)?
Something causes human extinction before digital minds can sustain themselves without us
Aliens.
Simulation ends.
???
As Holden says, we should not expect business as usual.
By “rule the Earth”, I mean: have the most influence over what happens on Earth, compared to any other kinds of agents. The idea of “agency” is hard to pin down; the key thing is that insofar as humans and animals currently have agency, digital minds will have it too.↩︎
Humans and animals have minds which run on biological hardware. Digital minds run on silicon or other non-biological hardware.↩︎
This might be a desirable outcome. We could think of digital minds as our descendents, and wish them well just as we do our children and grandchildren. They might live flourishing lives and wish us well too, supporting us for a wonderful retirement. It is very unclear whether we should use a “raising children” or “alien invasion” frame when we think about the development of AI.↩︎
If we pull this off, and avoid (1b), then we’ll probably be at least 10x richer and biological humans will be far more “transhuman” than we are now (e.g. enjoy healthy lives for hundreds of years).↩︎