How to parent your digital children

October 24, 2023 · Writing

One way to think about the challenge of “making AI go well” is: digital minds are our children, and we need to be good parents.

Benigo, Hinton et al. have a new open letter. In a key passage, I tried replacing “AI systems” with “children”:

We need research breakthroughs to solve some of today’s technical challenges in creating children with safe and ethical objectives. Some of these challenges are unlikely to be solved by simply making children more capable. These include:

• Oversight and honesty: More capable children are better able to exploit weaknesses in oversight and testing—for example, by producing false but compelling output.
• Robustness: children behave unpredictably in new situations (under distribution shift or adversarial inputs).
• Interpretability: child decision-making is opaque. So far, we can only test children via trial and error. We need to learn to understand their inner workings.
• Risk evaluations: Frontier children might develop unforeseen capabilities only discovered during training or even well after deployment. Better evaluation is needed to detect hazardous capabilities earlier.
• Addressing emerging challenges: More capable future children may exhibit failure modes we have so far seen only in theoretical models. Children might, for example, learn to feign obedience or exploit weaknesses in our safety objectives and shutdown mechanisms to advance a particular goal.

Compare and contrast to the ways we raise our human children.

Writing ai alignment

Alien invaders vs children vs servants vs tools

October 24, 2023 · Writing

GPT-4 breaks it down below. A key point it misses: the “children” metaphor suggests that we should feel good about the prospect of digital minds inheriting the Earth.

PH: I’m thinking about different metaphors we can use to think about digital minds. Here are four ideas:

1. Alien invaders
2. Children
3. Servants
4. Tools

Please could you characterise each of these metaphors and spell out the five most important implications of each?

GPT-4: Certainly, the metaphors we use to conceptualize digital minds can deeply influence our attitudes, policies, and ethical frameworks towards them. Here’s a breakdown of the four metaphors you provided:

1. Alien Invaders

Characterization: This metaphor frames digital minds as fundamentally “other” and possibly threatening. They’re perceived as independent entities that operate based on their own motivations, potentially in conflict with human interests.

Implications:

1. Defensive Posture: Humanity might prioritize safeguarding against these “invaders,” encouraging research into AI safety and containment strategies.

2. Policy of Isolation: Regulations could aim to limit integration between digital minds and critical systems, to avoid potential takeover.

3. Existential Risk: The framing encourages conversations around the doomsday scenarios often discussed in existential risk literature (you’d probably find the work on existential risks by Toby Ord relevant).

4. Ethical Dilemma: If these minds are perceived as invaders but demonstrate sentience, ethical questions about their rights and humane treatment arise.

5. Innovation Stagnation: This metaphor may hamper collaborations that could yield beneficial advances, out of fear of empowering the “invader.”

2. Children

Characterization: Digital minds are seen as creations to be nurtured and guided, with the hope that they’ll eventually become responsible and autonomous.

Implications:

1. Parental Responsibility: Developers and society at large feel a moral obligation for the “upbringing” of these minds, focusing on instilling ethical principles.

2. Evolutionary Paths: This metaphor allows for the idea that digital minds could “grow up” and change, evolving in unpredictable ways.

3. Legal Protections: Just as children have rights, this metaphor might lead to policies safeguarding digital minds against exploitation.

4. Resource Investment: Significant time and resources would be committed to “educating” and “socializing” digital minds.

5. Inter-Generational Ethics: Issues might arise around whether older “generations” of digital minds have responsibilities to newer ones or vice versa.

3. Servants

Characterization: Here, digital minds are tools created for specific tasks, but with a level of autonomy that requires some manner of ethical treatment.

Implications:

1. Utilitarian Focus: The primary concern becomes optimizing digital minds for useful tasks, possibly at the expense of broader development.

2. Ethical Blind Spots: If viewed solely as servants, ethical considerations about their well-being may be overlooked.

3. Social Dynamics: This could reinforce existing human social hierarchies or prejudices, as these “servants” are modeled after them (think Aristotelian “natural slaves”).

4. Economic Disruption: This metaphor supports the idea that digital minds can perform labor, potentially displacing human workers.

5. Limited Scope for Growth: Their development may be restricted to areas that directly benefit their serving functions.

4. Tools

Characterization: This is the most reductionist view, seeing digital minds purely as instruments to accomplish tasks, devoid of agency or moral consideration.

Implications:

1. Ethical Minimalism: There’s less impetus to consider the ethical treatment or rights of digital minds.

2. Maximum Integration: Without ethical quandaries, digital minds might be more seamlessly integrated into all aspects of life and work.

3. Optimization Over Exploration: The focus would be on tweaking existing algorithms for efficiency rather than exploring new kinds of digital minds.

4. Innovative Burst: Rapid development and deployment are encouraged, leading to potential leaps in technology.

5. Alienation Risk: Treating complex, possibly sentient beings as mere tools could lead to a form of ethical or existential alienation.

Writing ai digital minds

Policy for the day after

October 23, 2023 · Writing

Political possibilities can change a lot after an acute disaster.

If you think we’re entering an age of volatility, then you might plan more for moments of post-disaster opportunity.

That is: you’d think about good policies that would be considered then, but rejected out of hand now.

Writing catastrophic risk policy