Joseph Heath on Kantian evolutionary naturalism (rationality, pragmatism and deontic constraints)

One way to approach the puzzle of deontic constraint is to ask whether rational action necessarily has a consequentialist structure, or whether it can incorporate nonconsequential considerations.

[...]

Unfortunately, many theorists (philosophers and social scientists) have been misled into believing that the technical apparatus of rational choice theory, introduced in order to handle the complications of probabilistic reasoning, is also one that prohibits the introduction of nonconsequential considerations into the agent’s practical deliberations. In other words, it is sometimes thought that decision theorists are necessarily committed to consequentialism, or that consequentialism is simply the expression of Bayesian reasoning, when applied to practical affairs. Deontic constraint, or rule-following behavior, according to this view, is either not mathematically tractable, or else violates some elementary canon of logical consistency.

There is absolutely no reason that a rational choice theorist cannot incorporate deontic constraints—or any other type of rule-following behavior—into a formal model of rational action as utility-maximization (although, in so doing, it would perhaps be prudent to shift away from the vocabulary of utility-maximization toward that of value-maximization, given the close connection in many people’s minds between utility theory and consequentialism). The commitment to consequentialism on the part of many rational choice theorists is the result of a straightforward oversight that arose in the transition from decision theory (which deals with rational choice in nonsocial contexts) to game theory (which deals with social interaction). Early decision theorists adopted a consequentialist vocabulary, but did so in a way that made consequentialism trivially true, and thus theoretically innocuous.

Since I am inclined to put rules on the “preference” rather than the “belief” side of the preference-belief distinction, what really needs to be shown is that the preference through which an agent’s commitment to a rule is expressed may also be rational. In order to do so, it is necessary to challenge the prevailing noncognitivism about preferences, or the view that desires are somewhat less susceptible to rational reevaluation than beliefs.

[...]

My goal is to take what I consider to be some of the best thinking done in the past couple of decades in epistemology and philosophy of language, and show how it “fits” with some of the most important work being done in evolutionary theory, in order to reveal the deep internal connection between rationality and rule-following. One of the major forces aiding and abetting the noncognitive conception of preference, for well over three centuries, has been a commitment to representationalism in the philosophy of mind (i.e., the view that “representation” constitutes a central explanatory concept when it comes to understanding the contentfulness of our mental states).

The alternative strategy, which has recently been developed with considerable sophistication by pragmatist theorists like Robert Brandom, is to start with a set of concepts that are tailor-made for the explanation of human action, and then extend these to explain belief and representation. This is based on the plausible intuition that human action in the world is more fundamental than human thought about the world.

[...]

This analysis serves as the basis for my defense of what I call “the transcendental necessity of morality.”

Reading the philosophical literature, it has come to my attention that “Kantian evolutionary naturalism” is not a particularly well-represented position in the debates over the foundations of human morality. This is a deficiency I hope to remedy. The basic Kantian claim, with respect to moral motivation, is that there is an internal connection between following the rules of morality and being a rational agent.

[...]

I would like to defend the rationality of deontic constraints at the level of action, but am not committed to defending “deontology” as a theory of justification.

[...]

There is also an inclination among moral philosophers to draw a sharp distinction between “moral” and what are called “conventional” obligations, such as rules of etiquette, or “social norms” more generally. I reject this distinction, not because I think morality is conventional, but rather because I follow Emile Durkheim in thinking that all social norms (or “conventions” in this way of speaking) have an implicitly moral dimension.

Sam Altman on The Merge

I think a merge is probably our best-case scenario. If two different species both want the same thing and only one can have it—in this case, to be the dominant species on the planet and beyond—they are going to have conflict. We should all want one team where all members care about the well-being of everyone else.

Although the merge has already begun, it’s going to get a lot weirder. We will be the first species ever to design our own descendants. My guess is that we can either be the biological bootloader for digital intelligence and then fade into an evolutionary tree branch, or we can figure out what a successful merge looks like.

https://blog.samaltman.com/the-merge

Vitalik Buterin on superintelligence: merge or die

Across the board, I see far too many plans to save the world that involve giving a small group of people extreme and opaque power and hoping that they use it wisely. And so I find myself drawn to a different philosophy, one that has detailed ideas for how to deal with risks, but which seeks to create and maintain a more democratic world and tries to avoid centralization as the go-to solution to our problems.

[...]

Unless we create a world government powerful enough to detect and stop every small group of people hacking on individual GPUs with laptops, someone is going to create a superintelligent AI eventually - one that can think a thousand times faster than we can - and no combination of humans using tools with their hands is going to be able to hold its own against that. And so we need to take this idea of human-computer cooperation much deeper and further.

A first natural step is brain-computer interfaces. Brain-computer interfaces can give humans much more direct access to more-and-more powerful forms of computation and cognition, reducing the two-way communication loop between man and machine from seconds to milliseconds. This would also greatly reduce the "mental effort" cost to getting a computer to help you gather facts, give suggestions or execute on a plan.

Later stages of such a roadmap admittedly get weird. In addition to brain-computer interfaces, there are various paths to improving our brains directly through innovations in biology. An eventual further step, which merges both paths, may involve uploading our minds to run on computers directly.

[...]

If we want a future that is both superintelligent and "human", one where human beings are not just pets, but actually retain meaningful agency over the world, then it feels like something like this is the most natural option. There are also good arguments why this could be a safer AI alignment path: by involving human feedback at each step of decision-making, we reduce the incentive to offload high-level planning responsibility to the AI itself, and thereby reduce the chance that the AI does something totally unaligned with humanity's values on its own.

https://vitalik.eth.limo/general/2023/11/27/techno_optimism.html

Scott Alexander on conservatism self-preservation-against-optimality urge

I think that it might be reasonable to have continuation of your own culture as a terminal goal, even if you know your culture is “worse” in some way than what would replace it. There’s a transhumanist joke – “Instead of protecting human values, why not reprogram humans to like hydrogen? After all, there’s a lot of hydrogen.” There’s way more hydrogen than beautiful art, or star-crossed romances, or exciting adventures. A human who likes beautiful art, star-crossed romances, and exciting adventures is in some sense “worse” than a human who likes hydrogen, since it would be much harder for her to achieve her goals and she would probably be much less happy. But knowing this does not make me any happier about the idea of being reprogrammed in favor of hydrogen-related goals. My own value system might not be objectively the best, or even very good, but it’s my value system and I want to keep it and you can’t take it away from me. I am an individualist and I think of this on an individual level, but I could also see having this self-preservation-against-optimality urge for my community and its values.

(I’ve sometimes heard this called Lovecraftian parochialism, based on H.P. Lovecraft’s philosophy that the universe is vast and incomprehensible and anti-human, and you’ve got to draw the line between Self and Other somewhere, so you might as well draw the line at 1920s Providence, Rhode Island, and call everywhere else from Boston all the way to the unspeakable abyss-city of Y’ha-nthlei just different degrees of horribleness.)

https://slatestarcodex.com/2016/07/25/how-the-west-was-won/

Scott Alexander on liberalism and universal culture

On liberalism

Liberalism is a technology for preventing civil war. It was forged in the fires of Hell – the horrors of the endless seventeenth century religious wars. For a hundred years, Europe tore itself apart in some of the most brutal ways imaginable – until finally, from the burning wreckage, we drew forth this amazing piece of alien machinery. A machine that, when tuned just right, let people live together peacefully without doing the “kill people for being Protestant” thing. Popular historical strategies for dealing with differences have included: brutally enforced conformity, brutally efficient genocide, and making sure to keep the alien machine tuned really really carefully.

https://slatestarcodex.com/2017/06/21/against-murderism/

On "Western" culture vs universal culture

I am pretty sure there was, at one point, such a thing as western civilization. I think it included things like dancing around maypoles and copying Latin manuscripts. At some point Thor might have been involved. That civilization is dead. It summoned an alien entity from beyond the void which devoured its summoner and is proceeding to eat the rest of the world.

[...]

“Western medicine” is just medicine that works. It happens to be western because the West had a technological head start, and so discovered most of the medicine that works first. But there’s nothing culturally western about it; there’s nothing Christian or Greco-Roman about using penicillin to deal with a bacterial infection. Indeed, “western medicine” replaced the traditional medicine of Europe – Hippocrates’ four humors – before it started threatening the traditional medicines of China or India. So-called “western medicine” is an inhuman perfect construct from beyond the void, summoned by Westerners, which ate traditional Western medicine first and is now proceeding to eat the rest of the world.

[...]

If western medicine is just medicine that works, soda pop is just refreshment that works.

[...]

Sushi has spread almost as rapidly as Coke. But in what sense has sushi been “westernized”? Yes, Europe has adopted sushi. But so have China, India, and Africa. Sushi is [just] another refreshment that works.

Here’s what I think is going on. Maybe every culture is the gradual accumulation of useful environmental adaptations combined with random memetic drift. But this is usually a slow process with plenty of room for everybody to adjust and local peculiarities to seep in. The Industrial Revolution caused such rapid change that the process become qualitatively different, a frantic search for better adaptations to an environment that was itself changing almost as fast as people could understand it.

The Industrial Revolution also changed the way culture was spatially distributed. When the fastest mode of transportation is the horse, and the postal system is frequently ambushed by Huns, almost all culture is local culture. England develops a culture, France develops a culture, Spain develops a culture. Geographic, language, and political barriers keep these from intermixing too much. Add rapid communication – even at the level of a good postal service – and the equation begins to change. In the 17th century, philosophers were remarking (in Latin, the universal language!) about how Descartes from France had more in common with Leibniz from Germany than either of them did with the average Frenchman or German. Nowadays I certainly have more in common with SSC readers in Finland than I do with my next-door neighbor whom I’ve never met.

Improved trade and communication networks created a rapid flow of ideas from one big commercial center to another. Things that worked – western medicine, Coca-Cola, egalitarian gender norms, sushi – spread along the trade networks and started outcompeting things that didn’t. It happened in the west first, but not in any kind of a black-and-white way. Places were inducted into the universal culture in proportion to their participation in global trade; Shanghai was infected before West Kerry; Dubai is further gone than Alabama. The great financial capitals became a single cultural region in the same way that “England” or “France” had been a cultural region in the olden times, gradually converging on more and more ideas that worked in their new economic situation.

Let me say again that this universal culture, though it started in the West, was western only in the most cosmetic ways. If China or the Caliphate had industrialized first, they would have been the ones who developed it, and it would have been much the same. The new sodas and medicines and gender norms invented in Beijing or Baghdad would have spread throughout the world, and they would have looked very familiar. The best way to industrialize is the best way to industrialize.

Universal culture is the collection of the most competitive ideas and products. Coca-Cola spreads because it tastes better than whatever people were drinking before. Egalitarian gender norms spread because they’re more popular and likeable than their predecessors. If there was something that outcompeted Coca-Cola, then that would be the official soda of universal culture and Coca-Cola would be consigned to the scrapheap of history. 

The only reason universal culture doesn’t outcompete everything else instantly and achieve fixation around the globe is barriers to communication. Some of those barriers are natural – Tibet survived universalization for a long time because nobody could get to it. Sometimes the barrier is time – universal culture can’t assimilate every little hill and valley instantly. Other times there are no natural barriers, and then your choice is to either accept assimilation into universal culture, or put up some form of censorship.

https://slatestarcodex.com/2016/07/25/how-the-west-was-won/

Michael Nielsen is having second thoughts

My work has nearly all been about accelerating science and technology. And over the last year I find myself very conflicted. I've begun to question whether accelerating science and technology is a good idea at all.

https://michaelnotebook.com/ti/index.html

Tyler Cowen on Malthus on vice

So one way to read Malthus is this: if a society is going to have any prosperity at all, the people in that society either will be morally quite bad, or they have to be morally very, very good, good enough to exercise that moral restraint. Alternatively, you can read Malthus as seeing two primary goals for people: food and sex. His accomplishment was to show that, taken collectively, those two goals could not easily be obtainable simultaneously in a satisfactory fashion. In late Freudian terms, you could say that eros/sex amounts to the death drive, but again painted on a collective canvas and driven by economic mechanisms.

Malthus also hinted at birth control as an important social and economic force, especially later in 1817, putting him ahead of many other thinkers of his time. Birth control was widely practiced for centuries through a variety of means, and Malthus unfortunately was not very specific. He did call it “unnatural,” and the mainstream theology of his Anglican church condemned it, as did many other churches. But what did he really think? Was this unnatural practice so much worse than the other alternatives of misery and vice that his model was putting forward? Or did Malthus simply fail to see that birth control could be so effective and widespread as it is today? It doesn’t seem we are ever going to know.

From Malthus’s tripartite grouping of vice, moral restraint, and misery, two things should be clear immediately. The first is why Keynes found Malthus so interesting, namely that homosexual passions are one (partial) way out of the Malthusian trap. The second is that there is a Straussian reading of Malthus, namely that he thought moral restraint, while wonderful, was limited in its applicability. So maybe then vice wasn’t so bad after all? Is it not better than war and starvation?

I don’t buy the Straussian reading as a description of what Malthus really meant. But he knew it was there, and he knew he was forcing you to think about just how bad you thought vice really was. Malthus for instance is quite willing to reference prostitution as one possible means to keep down population. He talks about “men,” and “a numerous class of females,” but he worries that those practices “lower in the most marked manner the dignity of human nature.” It degrades the female character and amongst “those unfortunate females with which all great towns abound, more real distress and aggravated misery are perhaps to be found, than in any other department of human life.”

How bad are those vices relative to starvation and population triage? Well, the modern world has debated that question and mostly we have opted for vice. You thus can see that the prosperity of the modern world does not refute Malthus. We faced the Malthusian dilemma and opted for one of his options, namely vice. It’s just that a lot of us don’t find those vices as morally abhorrent as Malthus did. You could say we invented another technology that (maybe) does not suffer from diminishing returns, namely improving the dignity and the living conditions of those who practice vice. Contemporary college dorms seem pretty comfortable, and they have plenty of birth control, and of course lots of vice in the Malthusian sense. While those undergraduates might experience high rates of depression and also sexual violation, that life of vice still seems far better than life near the subsistence point. I am not sure what Malthus would think of college dorm sexual norms (and living standards!), but his broader failing was that he did not foresee the sanitization and partial moral neutering of what he considered to be vice.

Mapping the debate about desirable futures

We can map talk about desirable futures along several axes. Here are a few:

1. Axiology: partial or impartial (human prejudice vs view from nowhere).

2. Metaethics: naturalism vs non-naturalism (orthogonality thesis; alignment problem).

3. Evolution: fatalism or agency (inevitable vs contingent).

4. Rationality: ecological vs axiological (maxipok or maximise across the multiverse).

The philosophical questions above inform the more empirical debates about emerging technologies, such as:

5. Optimal rate of change: slow vs fast (or mixed).

6. Competition vs governance.

7. Convergence.

What are some other important axes? What are the most plausible combinations? Where do key thinkers land on these?

I find it surprisingly hard to name more than a handful of people who have written on all of the above in public.

But I'll have a go at placing people on these axes in a forthcoming post.

For now I'll just note that there's too much complexity here. Ultimately we need to distill our views down into some rough rules of thumb and faint guiding stars, then just chart a path through the froth of uncertainty (with wonder, vigour and Yes-saying).

As part of this, we need a vibe. e/acc is naive. Safety-ism lacks charisma. "It's time to build" is good, but tainted by association with Marc's recent screed.

As usual I'm back to Tyler—"be a builder":

Tyler Cowen: Uncertainty should not paralyze you. Try to do your best, pursue maximum expected value, and just avoid the moral nervousness. Be a little Straussian about it. Like here’s a rule, on average it’s a good rule, we’re all gonna follow it. Bravo, go on to the next thing. Be a builder.

Joe Walker: Get on with it?

Tyler Cowen: Yes. Because ultimately the nervous Nellies, they’re not philosophically sophisticated, they’re overindulging in their own neuroticism when you get right down to it. So it’s not like there’s some brute ‘let’s be a builder’ view and then in contrast there’s some deeper wisdom that the real philosophers pursue. It’s: you be a builder or you’re a nervous Nelly. Take your pick. I say be a builder.

Also: be a two-thirds utilitarian.

And: be a Yes-sayer.

Sometimes I think that "get on with it" is the push I need too. Why am I constantly pulled back to philosophy?

Flo Crivello on entrepreneurship in the present moment

At this point, if your startup’s vision doesn’t sound like hard sci fi, you’re probably not understanding what’s going on.

Richard Ngo on posthuman freedom

Just under the surface, joyful humans are granted miraculous powers: to soar through air and sea, to play games on the scale of planets, to morph their bodies as they please. Beneath them, you see a society that’s explored further, morphing not just their bodies, but also their minds—belief and desire and identity become as malleable as clay. Beneath them, you lose sight of individuals: at those depths minds merge and split and reform like currents in the ocean. And beneath even that? It’s hard for you to make sense of the impressions you’re getting—whatever is down there can’t be described in human terms. In the farthest reaches there are only alien algorithms, churning away on computers that stretch across galaxies, calculating the output of some function far beyond your comprehension.

And now you see the trap. Each step down makes so much sense, from the vantage point of the previous stage. But after you take any step, the next will soon be just as tempting. And once you’re in the water, there’s no line you can draw, no fence that can save you. You’ll just keep sinking deeper and deeper, with more and more of your current self stripped away—until eventually you’ll become one of the creatures that you can glimpse only hazily, one of the deep-dwelling monsters that has forsaken anything recognizably human.

So this is the line you decided to draw: here, and no further. You’ll live out your lives in a mundane world of baseline humans, with only a touch of magic at the edges—just enough to satisfy the wondering child in you. You’ll hold on to yourself, because what else is there to hold onto?

https://www.narrativeark.xyz/p/the-soul-key

Sam Altman's closing remarks at OpenAI DevDay

We hope that you'll come back next year. What we launch today is going to look very quaint relative to what we're busy creating for you now.

AI safety and the Last Man

Marc Andreessen, Wolf Tivvy and Nick Land all think that the AI safety crowd exhibit the fearfulness of Nietzsche's Last Man in the face of "dangerous agency" (Wolf's phrase) ¹.

The Last Man dreams of a procedural utilitarian peace and security. He hopes that moral philosophy and rational decision theory might lead to an end of war.

By contrast: the Nietzschean, evolutionary thinkers valorise struggle, bravery and Great Men. For them, Darwinian competition continues forever.

But: key figures in the AI safety crowd are evolutionary thinkers, transhumanists. While Derek Parfit represents the Last Man, Nick Bostrom is a Nietzsche super-fan ².

Parfit's longtermism is that of the Last Man. Bostrom's might just be Nietzschean, with greater pragmatism, and greater realism ³.

It's tempting to write off pragmatists as Last Men, technocrats whose only dreams are nightmares. But I don't think that's right.

The Great Man is a pragmatist by necessity. Unlike the Last Man, he does not value comfort and security above all else—he accepts risk of death when it is called for. But he is not indifferent to risk—he wants to win, he wants to say "Yes".

Two-thirds utilitarianism, conservative humanism and transhumanist dreams can fit together more easily than one might think.

“The Manifesto of Futurism“ by Filippo Tommaso Marinetti

1 We want to sing the love of danger, the habit of energy and rashness.

2 The essential elements of our poetry will be courage, audacity and revolt.

3 Literature has up to now magnified pensive immobility, ecstasy and slumber. We want to exalt movements of aggression, feverish sleeplessness, the double march, the perilous leap, the slap and the blow with the fist.

4 We declare that the splendour of the world has been enriched by a new beauty: the beauty of speed. A racing automobile with its bonnet adorned with great tubes like serpents with explosive breath ... a roaring motor car which seems to run on machine-gun fire is more beautiful than the Victory of Samothrace.

5 We want to sing the man at the wheel, the ideal axis of which crosses the earth, itself hurled along its orbit.

6 The poet must spend himself with warmth, glamour and prodigality to increase the enthusiastic fervour of the primordial elements.

7 Beauty exists only in struggle. There is no masterpiece that has not an aggressive character. Poetry must be a violent assault on the forces of the unknown, to force them to bow before man.

8 We are on the extreme promontory of the centuries! What is the use of looking behind at the moment when we must open the mysterious shutters of the impossible? Time and Space died yesterday. We are already living in the absolute, since we have already created eternal, omnipresent speed.

9 We want to glorify war - the only cure for the world - militarism, patriotism, the destructive gesture of the anarchists, the beautiful ideas which kill, and contempt for woman.

10 We want to demolish museums and libraries, fight morality, feminism and all opportunist and utilitarian cowardice.

11 We will sing of the great crowds agitated by work, pleasure and revolt; the multicoloured and polyphonic surf of revolutions in modern capitals; the nocturnal vibration of the arsenals and the workshops beneath their violent electric moons; the gluttonous railway stations devouring smoking serpents; factories suspended from the clouds by the thread of their smoke; bridges with the leap of gymnasts flung across the diabolic cutlery of sunny rivers; adventurous steamers sniffing the horizon; great-breasted locomotives, puffing on the rails like enormous steel horses with long tubes for bridle, and the gliding flight of aeroplanes whose propeller sounds like the flapping of a flag and the applause of enthusiastic crowds.Author's note: It is in Italy that we are issuing this manifesto of ruinous and incendiary violence, by which we today are founding Futurism, because we want to deliver Italy from its gangrene of professors, archaeologists, tourist guides and antiquaries. Italy has been too long the great second-hand market. We want to get rid of the innumerable museums which cover it with innumerable cemeteries.

Fin Moorhouse & Max Tegmark on the braids of life

My guess is that a given atom in your body has been there for an average of a few months, with a wide distribution between a couple days (water) to a lifetime (stable molecules in e.g. DNA from infancy). And so I’d guess most of our weight in atoms is recycled at least every few weeks.

The picture this paints is one in which we are not so much solid, isolated objects; and more like patterns surviving through time.

[...]

The physicist Max Tegmark suggests a picture of life as a ‘braid’ in space through time.

If that’s right, then the threads of the braid that makes up a person run through skies, seas, other life, and through the machines of industry. A person’s braid isn’t isolated through their life, but ephemeral; swapping threads with the wider world until the point of death, where the threads that now make up the braid — few of the originals remain — disperse again.

The case for the Scaling Hypothesis, in two sentences

The theory of evolution shows that complex, ordered systems can arise out of simple starting conditions and simple algorithms run over and over and over and over again.

Get this in your bones, and it becomes natural to believe that the brain is "just" another case of this.

How to parent your digital children

One way to think about the challenge of "making AI go well" is: digital minds are our children, and we need to be good parents.

Benigo, Hinton et al. have a new open letter. In a key passage, I tried replacing "AI systems" with "children":

We need research breakthroughs to solve some of today’s technical challenges in creating children with safe and ethical objectives. Some of these challenges are unlikely to be solved by simply making children more capable. These include:

• Oversight and honesty: More capable children are better able to exploit weaknesses in oversight and testing—for example, by producing false but compelling output.
• Robustness: children behave unpredictably in new situations (under distribution shift or adversarial inputs).
• Interpretability: child decision-making is opaque. So far, we can only test children via trial and error. We need to learn to understand their inner workings.
• Risk evaluations: Frontier children might develop unforeseen capabilities only discovered during training or even well after deployment. Better evaluation is needed to detect hazardous capabilities earlier.
• Addressing emerging challenges: More capable future children may exhibit failure modes we have so far seen only in theoretical models. Children might, for example, learn to feign obedience or exploit weaknesses in our safety objectives and shutdown mechanisms to advance a particular goal.

Compare and contrast to the ways we raise our human children.

Alien invaders vs children vs servants vs tools

GPT-4 breaks it down below. A key point it misses: the "children" metaphor suggests that we should feel good about the prospect of digital minds inheriting the Earth.

PH: I’m thinking about different metaphors we can use to think about digital minds. Here are four ideas:

1. Alien invaders
2. Children
3. Servants
4. Tools

Please could you characterise each of these metaphors and spell out the five most important implications of each?

GPT-4: Certainly, the metaphors we use to conceptualize digital minds can deeply influence our attitudes, policies, and ethical frameworks towards them. Here's a breakdown of the four metaphors you provided:

1. Alien Invaders

Characterization: This metaphor frames digital minds as fundamentally "other" and possibly threatening. They're perceived as independent entities that operate based on their own motivations, potentially in conflict with human interests.

Implications:

1. Defensive Posture: Humanity might prioritize safeguarding against these "invaders," encouraging research into AI safety and containment strategies.

2. Policy of Isolation: Regulations could aim to limit integration between digital minds and critical systems, to avoid potential takeover.

3. Existential Risk: The framing encourages conversations around the doomsday scenarios often discussed in existential risk literature (you'd probably find the work on existential risks by Toby Ord relevant).

4. Ethical Dilemma: If these minds are perceived as invaders but demonstrate sentience, ethical questions about their rights and humane treatment arise.

5. Innovation Stagnation: This metaphor may hamper collaborations that could yield beneficial advances, out of fear of empowering the "invader."

2. Children

Characterization: Digital minds are seen as creations to be nurtured and guided, with the hope that they'll eventually become responsible and autonomous.

Implications:

1. Parental Responsibility: Developers and society at large feel a moral obligation for the "upbringing" of these minds, focusing on instilling ethical principles.

2. Evolutionary Paths: This metaphor allows for the idea that digital minds could "grow up" and change, evolving in unpredictable ways.

3. Legal Protections: Just as children have rights, this metaphor might lead to policies safeguarding digital minds against exploitation.

4. Resource Investment: Significant time and resources would be committed to "educating" and "socializing" digital minds.

5. Inter-Generational Ethics: Issues might arise around whether older "generations" of digital minds have responsibilities to newer ones or vice versa.

3. Servants

Characterization: Here, digital minds are tools created for specific tasks, but with a level of autonomy that requires some manner of ethical treatment.

Implications:

1. Utilitarian Focus: The primary concern becomes optimizing digital minds for useful tasks, possibly at the expense of broader development.

2. Ethical Blind Spots: If viewed solely as servants, ethical considerations about their well-being may be overlooked.

3. Social Dynamics: This could reinforce existing human social hierarchies or prejudices, as these “servants” are modeled after them (think Aristotelian "natural slaves").

4. Economic Disruption: This metaphor supports the idea that digital minds can perform labor, potentially displacing human workers.

5. Limited Scope for Growth: Their development may be restricted to areas that directly benefit their serving functions.

4. Tools

Characterization: This is the most reductionist view, seeing digital minds purely as instruments to accomplish tasks, devoid of agency or moral consideration.

Implications:

1. Ethical Minimalism: There's less impetus to consider the ethical treatment or rights of digital minds.

2. Maximum Integration: Without ethical quandaries, digital minds might be more seamlessly integrated into all aspects of life and work.

3. Optimization Over Exploration: The focus would be on tweaking existing algorithms for efficiency rather than exploring new kinds of digital minds.

4. Innovative Burst: Rapid development and deployment are encouraged, leading to potential leaps in technology.

5. Alienation Risk: Treating complex, possibly sentient beings as mere tools could lead to a form of ethical or existential alienation.

Policy for the day after

Political possibilities can change a lot after an acute disaster.

If you think we're entering an age of volatility, then you might plan more for moments of post-disaster opportunity.

That is: you'd think about good policies that would be considered then, but rejected out of hand now.

Joscha Bach: meaning is instrumental to life

It's not that there is a meaning of life, but that meaning is instrumental to life.

Source.

Paul Christiano on making AI go well

Source: Current work in AI alignment (2020).