Despair is a sin and misery is a choice.

200 IQ and made of meat

Embryo selection probably works, and may arrive soon. So we can imagine a world where we have 10,000 adults who are roughly twice as intelligent, happy and healthy (and so on) compared to the most fortunate humans today.

What would happen in that world?

(1) Evolutionary perspective: these (trans)humans would attain most positions of power. They would form fertile factions. Within a few generations, the "old model" humans would be a minority of the population.

(2) Religious perspective: these beings would be so wise and benevolent that they would moderate their own power. They would make decisions in the interests of the social welfare function, taking care to minimise their violations of deontological side-constraints.

We should expect something closer to (1) than (2). Such is the way of things.

How do we feel about the prospect of (1)? It'll depend on the details, but broadly I think people today are happy with the idea of happier healthier and more intelligent humans inheriting the earth. We send our children to school.

Now: replace these transhumans with digital minds, which are made of sand instead of meat. Feels different, but do you endorse that feeling?

Atoms are atoms.¹

https://www.mit.edu/people/dpolicar/writing/prose/text/thinkingMeat.html).

What is AI alignment?

"AI alignment" means different things to different people.

My preferred definition, following many others, is:

An AI system is aligned if it does what its operator intends.

Why not just say that an AI system works if it does what its operator intends?

The reason is that AI systems are more agentic. To understand their behaviour, we adopt the intentional stance; we explain their behaviour partly by ascribing them values.

For an AI system to do what its operator intends, it must share values with its operator to a sufficient degree. Values do not need to perfectly shared in order for the system to do what the operator intends.

The same holds for a manager-employee relationship. In such a relationship, values are not perfectly shared, but they're shared enough that the employee does what the manager intends.

How closely do values need to be shared, for an AI system do to what its operator intends? And how easily can we build such systems? That's mostly an empirical question, and it depends on the use case.

The case of GPT-4 is somewhat encouraging: the system is remarkably good at understanding user intent; better, in fact, than many human colleagues.

Tyler Cowen's core view on AI

My core view: if we humans can get more intelligence into our hands, if we can't turn that into to a positive... what is it we're hoping for? This is our big chance! Let's not blow it.

Episode 41: Is This the AI Renaissance?

Digital Minds or Butlerian Accord? (Part 1)

Two scenarios for 2100 that might be choiceworthy:

1. Digital Minds: digital minds rule the Earth. There was a peaceful transition of power; humans think the 21st century went well and are optimistic about the future.
2. Butlerian Accord: biological humans rule the Earth. They have agreed not to develop AI systems that replace them, and to regulate other technologies that greatly increase extinction risk.

Which would be preferable? Would either be acceptable?

---

Let's imagine each these scenarios have come to pass (bracketing the question of how it actually happened).

If you like the Digital Minds scenario, you may think some or all of:

1. Digital minds will live far better lives than biological humans; there will also be far more of them.
2. Biological humans are overall better off in this scenario.
3. Digital minds will have values somewhat (or very) different from our own, and that's fine.
4. We should think of digital minds as our descendants, not as an alien species.
5. It would be good if our descendants become grabby and occupy a large local chunk of the Universe, rather than letting other civilisations take our place. "Digital Minds" soon increases the chance of this.

If you dislike the Digital Minds scenario, you may think some or all of:

1. Digital Minds will not be conscious, or otherwise capable of wellbeing.
2. Digital Minds will have values somewhat (or very) different from our own, and those values will be worse, objectively speaking.
3. Digital Minds will have values somewhat (or very) different from our own, and I don't like that.
4. We should think of digital minds as an "other" that we dislike.
5. Digital Minds will experience extraordinary suffering, or other kinds of disvalue.

---

If you like Butlerian Accord, you may think some or all of:

1. The human condition is already pretty good at 21st century levels of technology. Same goes for non-human animals.
2. A human civilisation based on a Butlerian Accord can be stable, and flourish for millennia.
3. We should shift to "Digital Minds" eventually, but not before we've thought long and hard about how to maximise the chance that shift goes well.

If you dislike Butlerian Accord, you may think some or all of:

1. We currently live in a Darwinian hellscape; superintelligent digital minds are the only way out of that.
2. A Butlerian Accord would not last.
3. ...

---

What other big reasons are there to like or dislike each scenario?

Who will rule the Earth in 2100?

Scenarios for 2100, with my probability guesstimates:

(1) (30%) Biological humans rule the Earth¹.

(2) (50%) Digital minds rule the Earth^{2 3}.

(3) (20%) Something very different to the above .

The shift from (1) to (2) or (3) might happen—or at least get baked in—within 20 years. I guess 10-30% chance of that.

The short story for (2) goes: the capabilities of AI systems surpass those of humans; AI systems pursue their own values.

The main ways that (1) might continue:

(a) (20%) A global agreement prevents the development of the most dangerous AI systems. ⁴

(b) (10%) A global disaster sets back the development of AI systems (along with much else) by decades. E.g. a major nuclear war.

(c) (<10%) A technological barrier prevents superhuman AI before 2100.

How might we get (3)?

(a) Something causes human extinction before digital minds can sustain themselves without us

(b) Aliens.

(c) Simulation ends.

(d) ???

As Holden says, we should not expect business as usual.

When you're writing to learn, very short posts are fine

When I draft a post for this journal, I often write several good paragraphs and then run into a lot of questions. I start to think about how I can make my writing more understandable. An hour or two later, I'm out of steam, and it's time to move on with my day.

Why? This is a study journal. I'm mostly writing for myself, and, on occasion, a small audience of high-context peers.

The journal format is partly about "getting thoughts out of my head" so that they are easier to interrogate. It's also about providing a partial record of thoughts I'd like to reconnect with later.

I'm going to try going hard on this style for a few weeks.

I'll write the journal I want to have.

@mfweof and @RokoMijic on naturalistic metaethics

Anything moral is only moral because it won, and it won for some reason other than being moral.

—@mfweof

The past is not more ethical than the present. The past is (relative to current standards) less ethical, but so is the future!

The maintenance of present day human values are bound to present day technology in the same way thay the maintenance of medieval hierarchies was bound to the function of knights on the battlefield. Technology is not value neutral, so if you want to upgrade your technology without destroying your values you need to do some kind of alignment work. You need to constrain that technology so that it doesn't do the thing it wants to do in the most efficient way possible: you need to make a gun that only knights can use.

—@RokoMijic

See also: Robin Hanson, Most AI fear is Future Fear.

Eliezer Yudkowsky (1999) on a world with nanotechnology

Unless you've heard of nanotechnology, it's hard to appreciate the magnitude of the changes we're talking about.  Total control of the material world at the molecular level is what the conservatives in the futurism business are predicting.

Taking 10^17 ops/sec as the figure for the computing power used by a human brain, and using optimized atomic-scale hardware, we could run the entire human race on one gram of matter, running at a rate of one million subjective years every second.

Frequently Asked Questions about the Meaning of Life

Kendall Roy on Logan Roy

He was comfortable with this world. And he knew it. And he liked it.

And I say, amen to that.

Tom Holland on Paul’s Letter to the Galatians

COWEN: Which Gospel do you view as most foundational for Western liberalism and why?

HOLLAND: I think that that is a treacherous question to ask because it implies that there would be a coherent line of descent from any one text that can be traced like that. I think that the line of descent that leads from the Gospels and from the New Testament and from the Bible and, indeed, from the entire corpus of early Christian texts to modern liberalism is too confused, too much of a swirl of influences for us to trace it back to a particular text.

If I had to choose any one book from the Bible, it wouldn’t be a Gospel. It would probably be Paul’s Letter to the Galatians because Paul’s Letter to the Galatians contains the famous verse that there is no Jew or Greek, there is no slave or free, there is no man or woman in Christ. In a way, that text — even if you bracket out and remove the “in Christ” from it — that idea that, properly, there should be no discrimination between people of different cultural and ethnic backgrounds, based on gender, based on class, remains pretty foundational for liberalism to this day.

I think that liberalism, in so many ways, is a secularized rendering of that extraordinary verse. But I think it’s almost impossible to avoid metaphor when thinking about what the relationship is of these biblical texts, these biblical verses to the present day. I variously compared Paul, in particular in his letters and his writings, rather unoriginally, to an acorn from which a mighty oak grows.

But I think actually, more appropriately, of a depth charge released beneath the vast fabric of classical civilization. And the ripples, the reverberations of it are faint to begin with, and they become louder and louder and more and more disruptive. Those echoes from that depth charge continue to reverberate to this day.

Carl Schmitt on liberalism

Otto Petras on the conditions for religious awe

A religion that one understands is, for he who understands, no longer a religion. For by comprehending it, he stands above it; he surveys its conditions and possibilities, and to the extent that he does so he no longer feels like the unconditional object of religious demands. One can be possessed and awe-struck only as long as one does not understand how and why that occurs.

Chris Cullen: what do you want to naturalise?

We become our habits. So it's crucial to ask: what do you want to naturalise? What do you want to come easily?

Whatever the heart frequently dwells upon becomes the shape of the heart.

Richard Meadows & Nassim Taleb on FU money

Humphrey Bogart used to keep a $100 bill in his dresser drawer at all times—a decent chunk of change in the 1920s. He referred to it as his ‘fuck-you money’, because it meant he’d never be forced to take a crappy part. According to Bogie, the only good reason for making money was “so you can tell any son-of-a-bitch in the world to go to hell”.

— Richard Meadows

---

A sum large enough to get most, if not all, of the advantages of wealth (the most important one being independence and the ability to only occupy your mind with matters that interest you) but not its side effects, such as having to attend a black-tie charity event and being forced to listen to a polite exposition of the details of a marble-rich house renovation.

Money buys freedom: intellectual freedom, freedom to choose who you vote for, to choose what you want to do professionally. But having what I call “fuck you” money requires a huge amount of discipline. The minute you go a penny over, then you lose your freedom again.

— Nassim Taleb

The last homily of Pope Pius XII

Our mouths are filled with the word "love".

"But I, before anyone else, didn't know how to define it.

Our mouths are filled with the word "beauty".

But I, before anyone else, didn't know how to receive it.

For this, I ask you forgiveness.

Please, forgive me.

At times we confound love with madness.

Beauty with ecstasy.

History has repeated itself.

Madness and ecstasy have once again proven to be irresistible temptations, but they always end the way they did on Ventotene.

With unjust death.

In this case, of a good and innocent priest.

There is a life of happiness to be found in the sphere of gentleness, kindness, mildness, lovingness.

We must learn to be in the world.

And the Church must contemplate the idea of opening up to the love that is possible, in order to fight against the love that is aberrant.

All this, John Paul III, with great humility, calls "the middle way."

In the past few days I have understood.

It's not the middle way.

It is the way.

Ever since I came back, you've been asking yourselves all sorts of questions.

Is he the father or the son?

Is he God or the Holy Spirit?

Is he man or is he Jesus?

Did he wake up or did he rise from the dead?

Is he a saint or is he an imposter?

Is he Christ or is he the Antichrist?

Is he alive or is he dead?

It doesn't matter.

You know what is so beautiful about questions?

It's that we don't have the answers.

In the end, only God has the answers.

They are his secret.

God's secret, which only He knows.

That is the mystery in which we believe.

And that is the mystery which guides our conscience.

And now I would like to come down among you, and do what I have wanted to do since the first moment: embrace you, one by one.

Marc Andreessen on the heros we're allowed to have

The anti-hero is the portrait of the Nietzschean superhero that we are allowed to have. Tony Soprano, Walter White, Don Draper. We can have someone who does Great Things, so long as that person is fundamentally bad by the standards of modern morality.

We are not allowed to have the full version of the Nietzschean superman doing something outstanding. We're not allowed Napoleon figures, the building of the pyramids, Beethoven, or even the person who built the transcontinental railroad, the car industry, that sort of thing.

The full Nietzschean superman is the person who says 'I really am going to rule the world, and rule it much better'. Those narratives are gone. They're too scary. They're absolutely frightening, because if we rediscover that kind of morality it would upend our entire order.

Email to Tyler on Bernard Williams & effective altruism

In your MacAskill interview, and again in the St Andrews talk, I heard you channeling Bernard Williams on Philosophy as a Humanistic Discipline and especially "The Human Prejudice".

I agree that Williams on philosophy and impartiality is an important message for EA. I pushed this line in conversations with Will MacAskill and others in 2015, and with several other Oxford figures since then. I'm surely not the ideal advocate, but in the replies I mostly heard a lot of "ugh, Bernard" followed by weak arguments against superficial misreadings of his work. People seemed very much in the mode of "devalue and dismiss".

My best EA Forum post is also my least popular:

https://forum.effectivealtruism.org/posts/G6EWTrArPDf74sr3S/bernard-williams-ethics-and-the-limits-of-impartiality

Williams' low status within EA is surprising given how seriously Derek Parfit took him as a peer. I understand that Williams was often seen as using non-kosher methods and unkind remarks in his philosophical writing and conversation, and was intensely disliked by some of his peers. So I suspect that much of his neglect is driven by residual animosity in the Oxford crowd. But they ignore this kind of thing and just "take the ideas seriously"... right...?

There are some notable exceptions. For example, Thomas Moynihan is somewhat associated with the Oxford EA scene, and appropriately rates Bernard Williams. Unsurprisingly, Tom has a background in "continental" philosophy.

You've not blogged much about Williams. How about it? E.g.

(1) Was Williams a pragmatist in denial, per Rorty's review of Truth and Truthfulness? Why did he resist Rorty?

(2) What prioritisation errors are made by those who go too far with impartiality? 

On (2): if EAs stopped going "too far" with impartiality, I think we'd see the EA portfolio shift a bit towards catastrophic risk and away from existential risk. The current strong focus on x-risk can be seen as another form of the 51:49 bet.

A couple years ago one of the more influential EAs told me that rejecting the 51:49 bet is a form of egoism. We should not care about our personal chances of survival: we should just follow the rule that maximises EV across all possible worlds. I replied that ecological rationality beats axiomatic rationality in the world I care about. But if you think impartial reasons are the only reasons that count, you can't justify your "arbitrary" care for this particular world over others.

And with that—and your remarks on the useful generativity of a mistake taken seriously—we're back to Nietzsche's remarks on Plato:

It seems that in order to inscribe themselves upon the heart of humanity with everlasting claims, all great things have first to wander about the earth as enormous and awe- inspiring caricatures: dogmatic philosophy has been a caricature of this kind--for instance, the Vedanta doctrine in Asia, and Platonism in Europe. Let us not be ungrateful to it, although it must certainly be confessed that the worst, the most tiresome, and the most dangerous of errors hitherto has been a dogmatist error--namely, Plato's invention of Pure Spirit and the Good in Itself. But now when it has been surmounted, when Europe, rid of this nightmare, can again draw breath freely and at least enjoy a healthier--sleep, we, WHOSE DUTY IS WAKEFULNESS ITSELF, are the heirs of all the strength which the struggle against this error has fostered. It amounted to the very inversion of truth, and the denial of the PERSPECTIVE--the fundamental condition--of life, to speak of Spirit and the Good as Plato spoke of them; indeed one might ask, as a physician: "How did such a malady attack that finest product of antiquity, Plato? Had the wicked Socrates really corrupted him? Was Socrates after all a corrupter of youths, and deserved his hemlock?" But the struggle against Plato, or--to speak plainer, and for the "people"--the struggle against the ecclesiastical oppression of millenniums of Christianity (FOR CHRISITIANITY IS PLATONISM FOR THE "PEOPLE"), produced in Europe a magnificent tension of soul, such as had not existed anywhere previously; with such a tensely strained bow one can now aim at the furthest goals. 

Peter

P.S. Nietzsche's thoughts on effective altruism, according to ChatGPT.

Jonathan Bi on how to live with a Girardian worldview

I compared Girard to my Virgil in the sense that he was able to rescue me through Hell. He was able to show me how to purge more milder forms of perversion.

But, just as Virgil couldn't take Dante all the way to heaven, neither could Girard. Girard kind of just retreats.

What I'm about to share with you is mostly my own creative interpretations on top of Girard.

I think there's in general two solutions, once you've identified there's a metaphysical and there's a physical desire. One wing, and I think this is what Girard leans to, is to say this metaphysical--this is the Buddhist as well as the Girardian way--is to say this metaphysical desire, this desire for being, it's completely perverse. It's _always_perverse, whether from Girard's perspective, because it's essentially a desire to be God. This is why it's satanic. You're desiring persistence; you're desiring power; you're desiring reality. If you push those far enough, those are the metaphysical qualities of the Judeo-Christian God. So, Girard actually sees metaphysical desire as the original sin, as the satanic drive to rival God in his metaphysical splendor.

And the Buddhists--right--we don't have to go into that, but long story short, these metaphysical qualities are not possible in the world. Emptiness is what permeates the world. So, this is a fundamentally wrong sort of desire.

So, for the Christians and Buddhists, the way to good health is to completely get rid of metaphysical desire, to be only concerned by the object physical desire.

There's another, however, strand of thinking, and probably most popular amongst the Germans, in Hegel, is to say there is actually a healthy way--the Germans, and Plato actually, which we'll talk about--there actually is a healthy way to exist in society. And the way, long story short, to do so is for your metaphysical and your physical desires to align.

That is to say: if you really like to do philosophy, don't hang out with a bunch of people who are industrialists. Hang out with a bunch of philosophers, so that the somewhat partial spectator, as we've discussed, will naturally _align_with your normative values, with your physical desires, and thus you'll receive recognition and a form of reality.

https://www.econtalk.org/johnathan-bi-on-mimesis-and-rene-girard/

Nick Bostrom on differential technological development

The Principle of Differential Technological Development

Retard the development of dangerous and harmful technologies, especially ones that raise the level of existential risk; and accelerate the development of beneficial technologies, especially those that reduce the existential risks posed by nature or by other technologies Bostrom, 2002).

The principle of differential technological development is compatible with plausible forms of technological determinism. For example, even if it were ordained that all technologies that can be developed will be developed, it can still matter when they are developed. The order in which they arrive can make an important difference – ideally, protective technologies should come before the destructive technologies against which they protect; or, if that is not possible, then it is desirable that the gap be minimized so that other countermeasures (or luck) may tide us over until robust protection become available. The timing of an invention also influences what sociopolitical context the technology is born into. For example, if we believe that there is a secular trend toward civilization becoming more capable of handling black balls, then we may want to delay the most risky technological developments, or at least abstain from accelerating them.