Peter Thiel on the Darwinian view of alignment
The core axiom of the Darwinian view of the world: there is no such thing as a selfless being. Therefore the alignment problem is fundamentally difficult.
Asmir Gračanin et al. on the function of tears
In the inter-personal domain, tears can be considered as a signal that conveys information about the helplessness of the crying individual resulting in increased motivation of observers to react with pro-social behaviour. However, the inter-personal effects of crying are also not always consistent. Although evidence suggests that the perception of tears generally results in helping behaviours, strengthening of social bonds and a reduction of aggression, there is convincing anecdotal evidence that (particularly acoustical) crying may also sometimes evoke irritation and even aggression and violence. The precise determinants of the reactions of others to crying still wait to be disclosed. Nevertheless, it seems obvious that the personality of the observer, the specific antecedent and the perceived appropriateness of crying, as well as well as the relationship between crier and observer may all play a role.
https://www.tandfonline.com/doi/full/10.1080/02699931.2016.1151402
Joe Carlsmith on the utilitarian dream
I think there’s a certain type of person who comes to philosophy and encounters ideas — a certain set of kind of simple and kind of otherwise elegant or theoretically attractive ideas. The ideas I most think of as in this cluster — and this is all separable — is total utilitarianism, Bayesianism, expected utility reasoning.
I remember at one point I was talking with a friend of mine, who used to be a utilitarian, about one of his views. And I started to offer a counterexample to his views, and he just cut me off and he was like, “Joe, I bite all the bullets.” I was like, “You don’t even need to hear the bullets?” He’s like, “Yeah. It’s like, whatever. Does it fall out of my view? I bite it.”
So I think that a certain kind of person in this mindset can feel like, “Sure, there are bullets I need to bite for this view. I need to push fat men off of bridges; I need to create repugnant conclusions, even with a bunch of hells involved.” All this sort of stuff. But they feel like, “I am hardcore, I am rigorous, I have theorems to back me up. My thing is simple; these other people’s theories, they’re janky and incomplete and kind of made up.”
It just has this flavor of you’re kind of unwilling to look the truth in the face. Like, make the lizards… Sorry, the lizards: One conclusion that falls out of total utilitarianism is the idea that for any utopia, there’s a better world with kind of a sufficient number of barely happy lizards plus arbitrary hells.
Infinite ethics just breaks this narrative. And that’s part of why I wanted to work on this topic: I felt like I saw around me some people who were too enamored of this utilitarian dream, who thought it was on better theoretical foundations than I think it is, who felt like it was more of a default, and more of a kind of simple, natural foundation than I think it is.
You’re going to have to start giving stuff up. You’re going to be incomplete. You’re going to start playing a game that looks more similar to the game you didn’t want to play before, and more similar to the game that everyone else was playing. You’re not going to be able to say, “I’m just going to bite whatever bullets my theory says to bite.” You’re running out of track, or to the extent you’re on a crazy train, the crazy train just runs out of track. There are just horrific bullets if you want to bite them, but it’s just a very different story and you’re much more lost.
I think that’s an important source of humility for people who are drawn to this perspective. I think they should be more lost than they were if they were really jazzed by like, “I know it’s total utilitarianism. I’m so hardcore. No one else is willing to be hardcore.” I’m like, I think you should spend some time with infinite ethics and adjust your confidence in the position accordingly.
https://80000hours.org/podcast/episodes/joe-carlsmith-navigating-serious-philosophical-confusion/
Quote joe carlsmith metaethics metaphilosophy utilitarianism
Joe Carlsmith against “non-naturalistic realism or bust”
Are you so sure you even know what this debate is about — what it means for something to matter, or to be “natural,” or to be “mind-dependent”— let alone what the answer is? So sure that if it’s only raw nature — only joy, love, friendship, pain, grief, and so on, with — then there’s nothing to fight for, or against? I, for one, am not.
So “you should have higher credence on naturalism-but-things-matter” is the immediate objection here. Indeed, I think this objection cautions wariness about the un-Bayesian-ness of much philosophical discourse. Some meta-ethicist might well declare confidently “if naturalism is true, then nothing matters!” But they are rarely thinking in terms of quantitative credences on “ok but actually maybe if naturalism is true some things matter after all,” or about the odds at which they’re willing to bet.
[…]
Remember, nihilism — the view that there are no normative facts – is distinct from what we might call “indifference-ism” — that is, the view that there are normative facts, and they actively say that you should be indifferent to everything. On nihilism, indifference is no more normatively required, as a response to the possibility of innocent children being burned alive, than is intense concern (or embarrassment, or desire to do a prime number of jumping jacks). Conditional on nihilism, nothing is telling you not to care about yourself, or your family, or those children: you’re absolutely free to do so. And plausibly — at least, if your psychology is similar to many people who claim to accept something like nihilism — you still will care.
[…]
My own sense is that most familiar, gloomy connotations of nihilism aren’t centrally about meta-ethics at all. Rather, they are associated more closely with a cluster of psychological and motivational issues related to depression, hopelessness, and loss of connection with a sense of care and purpose. Sometimes, these issues are bound up with someone’s views about the metaphysics of normative properties and the semantics of normative discourse (and sometimes, we grope for this sort of abstract language in order to frame some harder-to-articulate disorientation). But often, when such issues crop up, meta-ethics isn’t actually the core explanation. After all, the most meta-ethically inflationary realists, theists, and so on can see their worlds drain of color and their motivations go flat; and conversely, the most metaphysically reductionist subjectivists, anti-realists, nihilists and so on can fight just as hard as others to save their friends and families from a fire; to build flourishing lives and communities; to love their neighbors as themselves. Indeed, often (and even setting aside basic stuff about mental health, getting enough sleep/exercise, etc), stuff like despair, depression, and so on is often prompted most directly by the world not being a way you want it to be — e.g., not finding the sort of love, joy, status, accomplishment and so forth that you’re looking for; feeling stuck and bored by your life; feeling overwhelmed by the suffering in the world and/or unable to make a difference; etc — rather than by actually not wanting anything, or by a concern that something you want deeply isn’t worth wanting at the end of the day. Meta-ethics can matter in all this, yes, but we should be careful not to mistake psychological issues for philosophical ones — even when the lines get blurry.
https://joecarlsmith.com/2022/10/09/against-the-normative-realists-wager
See also: Simon Blackburn on people who think that anti-realism entails nihilism
200 IQ and made of meat
Embryo selection probably works, and may arrive soon. So we can imagine a world where we have 10,000 adults who are roughly twice as intelligent, happy and healthy (and so on) compared to the most fortunate humans today.
What would happen in that world?
(1) Evolutionary perspective: these (trans)humans would attain most positions of power. They would form fertile factions. Within a few generations, the “old model” humans would be a minority of the population.
(2) Religious perspective: these beings would be so wise and benevolent that they would moderate their own power. They would make decisions in the interests of the social welfare function, taking care to minimise their violations of deontological side-constraints.
We should expect something closer to (1) than (2). Such is the way of things.
How do we feel about the prospect of (1)? It’ll depend on the details, but broadly I think people today are happy with the idea of happier healthier and more intelligent humans inheriting the earth. We send our children to school.
Now: replace these transhumans with digital minds, which are made of sand instead of meat. Feels different, but do you endorse that feeling?
Atoms are atoms.1
The phrase “made of meat” is inspired by Terry Bison.↩︎