AI safety and the Last Man
Marc Andreessen, Wolf Tivvy and Nick Land all think that the AI safety crowd exhibit the fearfulness of Nietzsche’s Last Man in the face of “dangerous agency” (Wolf’s phrase) 1.
The Last Man dreams of a procedural utilitarian peace and security. He hopes that moral philosophy and rational decision theory might lead to an end of war.
By contrast: the Nietzschean, evolutionary thinkers valorise struggle, bravery and Great Men. For them, Darwinian competition continues forever.
But: key figures in the AI safety crowd are evolutionary thinkers, transhumanists. While Derek Parfit represents the Last Man, Nick Bostrom is a Nietzsche super-fan 2.
Parfit’s longtermism is that of the Last Man. Bostrom’s might just be Nietzschean, with greater pragmatism, and greater realism 3.
It’s tempting to write off pragmatists as Last Men, technocrats whose only dreams are nightmares. But I don’t think that’s right.
The Great Man is a pragmatist by necessity. Unlike the Last Man, he does not value comfort and security above all else—he accepts risk of death when it is called for. But he is not indifferent to risk—he wants to win, he wants to say “Yes”.
Two-thirds utilitarianism, conservative humanism and transhumanist dreams can fit together more easily than one might think.
So too would FT Marinetti.↩︎
“Superfan” is the word used by one of his research assistants, in private conversation. But this is low-key evident in his transhumanist writings and his recent paper on metaethics.↩︎
His concept of the Singleton (and its tacit recommendation) might seem deeply anti-Nietzschean. But he thinks of this as a likely outcome of evolutionary dynamics as we climb the technology tree.↩︎
“The Manifesto of Futurism“ by Filippo Tommaso Marinetti
1 We want to sing the love of danger, the habit of energy and rashness.
2 The essential elements of our poetry will be courage, audacity and revolt.
3 Literature has up to now magnified pensive immobility, ecstasy and slumber. We want to exalt movements of aggression, feverish sleeplessness, the double march, the perilous leap, the slap and the blow with the fist.
4 We declare that the splendour of the world has been enriched by a new beauty: the beauty of speed. A racing automobile with its bonnet adorned with great tubes like serpents with explosive breath … a roaring motor car which seems to run on machine-gun fire is more beautiful than the Victory of Samothrace.
5 We want to sing the man at the wheel, the ideal axis of which crosses the earth, itself hurled along its orbit.
6 The poet must spend himself with warmth, glamour and prodigality to increase the enthusiastic fervour of the primordial elements.
7 Beauty exists only in struggle. There is no masterpiece that has not an aggressive character. Poetry must be a violent assault on the forces of the unknown, to force them to bow before man.
8 We are on the extreme promontory of the centuries! What is the use of looking behind at the moment when we must open the mysterious shutters of the impossible? Time and Space died yesterday. We are already living in the absolute, since we have already created eternal, omnipresent speed.
9 We want to glorify war - the only cure for the world - militarism, patriotism, the destructive gesture of the anarchists, the beautiful ideas which kill, and contempt for woman.
10 We want to demolish museums and libraries, fight morality, feminism and all opportunist and utilitarian cowardice.
11 We will sing of the great crowds agitated by work, pleasure and revolt; the multicoloured and polyphonic surf of revolutions in modern capitals; the nocturnal vibration of the arsenals and the workshops beneath their violent electric moons; the gluttonous railway stations devouring smoking serpents; factories suspended from the clouds by the thread of their smoke; bridges with the leap of gymnasts flung across the diabolic cutlery of sunny rivers; adventurous steamers sniffing the horizon; great-breasted locomotives, puffing on the rails like enormous steel horses with long tubes for bridle, and the gliding flight of aeroplanes whose propeller sounds like the flapping of a flag and the applause of enthusiastic crowds.Author’s note: It is in Italy that we are issuing this manifesto of ruinous and incendiary violence, by which we today are founding Futurism, because we want to deliver Italy from its gangrene of professors, archaeologists, tourist guides and antiquaries. Italy has been too long the great second-hand market. We want to get rid of the innumerable museums which cover it with innumerable cemeteries.
The case for the Scaling Hypothesis, in two sentences
The theory of evolution shows that complex, ordered systems can arise out of simple starting conditions and simple algorithms run over and over and over and over again.
Get this in your bones, and it becomes natural to believe that the brain is “just” another case of this.
Fin Moorhouse & Max Tegmark on the braids of life
My guess is that a given atom in your body has been there for an average of a few months, with a wide distribution between a couple days (water) to a lifetime (stable molecules in e.g. DNA from infancy). And so I’d guess most of our weight in atoms is recycled at least every few weeks.
The picture this paints is one in which we are not so much solid, isolated objects; and more like patterns surviving through time.
[…]
The physicist Max Tegmark suggests a picture of life as a ‘braid’ in space through time.
If that’s right, then the threads of the braid that makes up a person run through skies, seas, other life, and through the machines of industry. A person’s braid isn’t isolated through their life, but ephemeral; swapping threads with the wider world until the point of death, where the threads that now make up the braid — few of the originals remain — disperse again.
How to parent your digital children
One way to think about the challenge of “making AI go well” is: digital minds are our children, and we need to be good parents.
Benigo, Hinton et al. have a new open letter. In a key passage, I tried replacing “AI systems” with “children”:
We need research breakthroughs to solve some of today’s technical challenges in creating children with safe and ethical objectives. Some of these challenges are unlikely to be solved by simply making children more capable. These include:
- Oversight and honesty: More capable children are better able to exploit weaknesses in oversight and testing—for example, by producing false but compelling output.
- Robustness: children behave unpredictably in new situations (under distribution shift or adversarial inputs).
- Interpretability: child decision-making is opaque. So far, we can only test children via trial and error. We need to learn to understand their inner workings.
- Risk evaluations: Frontier children might develop unforeseen capabilities only discovered during training or even well after deployment. Better evaluation is needed to detect hazardous capabilities earlier.
- Addressing emerging challenges: More capable future children may exhibit failure modes we have so far seen only in theoretical models. Children might, for example, learn to feign obedience or exploit weaknesses in our safety objectives and shutdown mechanisms to advance a particular goal.
Compare and contrast to the ways we raise our human children.
Alien invaders vs children vs servants vs tools
GPT-4 breaks it down below. A key point it misses: the “children” metaphor suggests that we should feel good about the prospect of digital minds inheriting the Earth.
PH: I’m thinking about different metaphors we can use to think about digital minds. Here are four ideas:
- Alien invaders
- Children
- Servants
- Tools
Please could you characterise each of these metaphors and spell out the five most important implications of each?
GPT-4: Certainly, the metaphors we use to conceptualize digital minds can deeply influence our attitudes, policies, and ethical frameworks towards them. Here’s a breakdown of the four metaphors you provided:
1. Alien Invaders
Characterization: This metaphor frames digital minds as fundamentally “other” and possibly threatening. They’re perceived as independent entities that operate based on their own motivations, potentially in conflict with human interests.
Implications:
Defensive Posture: Humanity might prioritize safeguarding against these “invaders,” encouraging research into AI safety and containment strategies.
Policy of Isolation: Regulations could aim to limit integration between digital minds and critical systems, to avoid potential takeover.
Existential Risk: The framing encourages conversations around the doomsday scenarios often discussed in existential risk literature (you’d probably find the work on existential risks by Toby Ord relevant).
Ethical Dilemma: If these minds are perceived as invaders but demonstrate sentience, ethical questions about their rights and humane treatment arise.
Innovation Stagnation: This metaphor may hamper collaborations that could yield beneficial advances, out of fear of empowering the “invader.”
2. Children
Characterization: Digital minds are seen as creations to be nurtured and guided, with the hope that they’ll eventually become responsible and autonomous.
Implications:
Parental Responsibility: Developers and society at large feel a moral obligation for the “upbringing” of these minds, focusing on instilling ethical principles.
Evolutionary Paths: This metaphor allows for the idea that digital minds could “grow up” and change, evolving in unpredictable ways.
Legal Protections: Just as children have rights, this metaphor might lead to policies safeguarding digital minds against exploitation.
Resource Investment: Significant time and resources would be committed to “educating” and “socializing” digital minds.
Inter-Generational Ethics: Issues might arise around whether older “generations” of digital minds have responsibilities to newer ones or vice versa.
3. Servants
Characterization: Here, digital minds are tools created for specific tasks, but with a level of autonomy that requires some manner of ethical treatment.
Implications:
Utilitarian Focus: The primary concern becomes optimizing digital minds for useful tasks, possibly at the expense of broader development.
Ethical Blind Spots: If viewed solely as servants, ethical considerations about their well-being may be overlooked.
Social Dynamics: This could reinforce existing human social hierarchies or prejudices, as these “servants” are modeled after them (think Aristotelian “natural slaves”).
Economic Disruption: This metaphor supports the idea that digital minds can perform labor, potentially displacing human workers.
Limited Scope for Growth: Their development may be restricted to areas that directly benefit their serving functions.
4. Tools
Characterization: This is the most reductionist view, seeing digital minds purely as instruments to accomplish tasks, devoid of agency or moral consideration.
Implications:
Ethical Minimalism: There’s less impetus to consider the ethical treatment or rights of digital minds.
Maximum Integration: Without ethical quandaries, digital minds might be more seamlessly integrated into all aspects of life and work.
Optimization Over Exploration: The focus would be on tweaking existing algorithms for efficiency rather than exploring new kinds of digital minds.
Innovative Burst: Rapid development and deployment are encouraged, leading to potential leaps in technology.
Alienation Risk: Treating complex, possibly sentient beings as mere tools could lead to a form of ethical or existential alienation.