Joseph Heath on Kantian evolutionary naturalism (rationality, pragmatism and deontic constraints)
One way to approach the puzzle of deontic constraint is to ask whether rational action necessarily has a consequentialist structure, or whether it can incorporate nonconsequential considerations.
[…]
Unfortunately, many theorists (philosophers and social scientists) have been misled into believing that the technical apparatus of rational choice theory, introduced in order to handle the complications of probabilistic reasoning, is also one that prohibits the introduction of nonconsequential considerations into the agent’s practical deliberations. In other words, it is sometimes thought that decision theorists are necessarily committed to consequentialism, or that consequentialism is simply the expression of Bayesian reasoning, when applied to practical affairs. Deontic constraint, or rule-following behavior, according to this view, is either not mathematically tractable, or else violates some elementary canon of logical consistency.
There is absolutely no reason that a rational choice theorist cannot incorporate deontic constraints—or any other type of rule-following behavior—into a formal model of rational action as utility-maximization (although, in so doing, it would perhaps be prudent to shift away from the vocabulary of utility-maximization toward that of value-maximization, given the close connection in many people’s minds between utility theory and consequentialism). The commitment to consequentialism on the part of many rational choice theorists is the result of a straightforward oversight that arose in the transition from decision theory (which deals with rational choice in nonsocial contexts) to game theory (which deals with social interaction). Early decision theorists adopted a consequentialist vocabulary, but did so in a way that made consequentialism trivially true, and thus theoretically innocuous.
Since I am inclined to put rules on the “preference” rather than the “belief” side of the preference-belief distinction, what really needs to be shown is that the preference through which an agent’s commitment to a rule is expressed may also be rational. In order to do so, it is necessary to challenge the prevailing noncognitivism about preferences, or the view that desires are somewhat less susceptible to rational reevaluation than beliefs.
[…]
My goal is to take what I consider to be some of the best thinking done in the past couple of decades in epistemology and philosophy of language, and show how it “fits” with some of the most important work being done in evolutionary theory, in order to reveal the deep internal connection between rationality and rule-following. One of the major forces aiding and abetting the noncognitive conception of preference, for well over three centuries, has been a commitment to representationalism in the philosophy of mind (i.e., the view that “representation” constitutes a central explanatory concept when it comes to understanding the contentfulness of our mental states).
The alternative strategy, which has recently been developed with considerable sophistication by pragmatist theorists like Robert Brandom, is to start with a set of concepts that are tailor-made for the explanation of human action, and then extend these to explain belief and representation. This is based on the plausible intuition that human action in the world is more fundamental than human thought about the world.
[…]
This analysis serves as the basis for my defense of what I call “the transcendental necessity of morality.”
Reading the philosophical literature, it has come to my attention that “Kantian evolutionary naturalism” is not a particularly well-represented position in the debates over the foundations of human morality. This is a deficiency I hope to remedy. The basic Kantian claim, with respect to moral motivation, is that there is an internal connection between following the rules of morality and being a rational agent.
[…]
I would like to defend the rationality of deontic constraints at the level of action, but am not committed to defending “deontology” as a theory of justification.
[…]
There is also an inclination among moral philosophers to draw a sharp distinction between “moral” and what are called “conventional” obligations, such as rules of etiquette, or “social norms” more generally. I reject this distinction, not because I think morality is conventional, but rather because I follow Emile Durkheim in thinking that all social norms (or “conventions” in this way of speaking) have an implicitly moral dimension.
Sam Altman on The Merge
I think a merge is probably our best-case scenario. If two different species both want the same thing and only one can have it—in this case, to be the dominant species on the planet and beyond—they are going to have conflict. We should all want one team where all members care about the well-being of everyone else.
Although the merge has already begun, it’s going to get a lot weirder. We will be the first species ever to design our own descendants. My guess is that we can either be the biological bootloader for digital intelligence and then fade into an evolutionary tree branch, or we can figure out what a successful merge looks like.
Vitalik Buterin on superintelligence: merge or die
Across the board, I see far too many plans to save the world that involve giving a small group of people extreme and opaque power and hoping that they use it wisely. And so I find myself drawn to a different philosophy, one that has detailed ideas for how to deal with risks, but which seeks to create and maintain a more democratic world and tries to avoid centralization as the go-to solution to our problems.
[…]
Unless we create a world government powerful enough to detect and stop every small group of people hacking on individual GPUs with laptops, someone is going to create a superintelligent AI eventually - one that can think a thousand times faster than we can - and no combination of humans using tools with their hands is going to be able to hold its own against that. And so we need to take this idea of human-computer cooperation much deeper and further.
A first natural step is brain-computer interfaces. Brain-computer interfaces can give humans much more direct access to more-and-more powerful forms of computation and cognition, reducing the two-way communication loop between man and machine from seconds to milliseconds. This would also greatly reduce the “mental effort” cost to getting a computer to help you gather facts, give suggestions or execute on a plan.
Later stages of such a roadmap admittedly get weird. In addition to brain-computer interfaces, there are various paths to improving our brains directly through innovations in biology. An eventual further step, which merges both paths, may involve uploading our minds to run on computers directly.
[…]
If we want a future that is both superintelligent and “human”, one where human beings are not just pets, but actually retain meaningful agency over the world, then it feels like something like this is the most natural option. There are also good arguments why this could be a safer AI alignment path: by involving human feedback at each step of decision-making, we reduce the incentive to offload high-level planning responsibility to the AI itself, and thereby reduce the chance that the AI does something totally unaligned with humanity’s values on its own.
https://vitalik.eth.limo/general/2023/11/27/techno_optimism.html
Scott Alexander on conservatism self-preservation-against-optimality urge
I think that it might be reasonable to have continuation of your own culture as a terminal goal, even if you know your culture is “worse” in some way than what would replace it. There’s a transhumanist joke — “Instead of protecting human values, why not reprogram humans to like hydrogen? After all, there’s a lot of hydrogen.” There’s way more hydrogen than beautiful art, or star-crossed romances, or exciting adventures. A human who likes beautiful art, star-crossed romances, and exciting adventures is in some sense “worse” than a human who likes hydrogen, since it would be much harder for her to achieve her goals and she would probably be much less happy. But knowing this does not make me any happier about the idea of being reprogrammed in favor of hydrogen-related goals. My own value system might not be objectively the best, or even very good, but it’s my value system and I want to keep it and you can’t take it away from me. I am an individualist and I think of this on an individual level, but I could also see having this self-preservation-against-optimality urge for my community and its values.
(I’ve sometimes heard this called Lovecraftian parochialism, based on H.P. Lovecraft’s philosophy that the universe is vast and incomprehensible and anti-human, and you’ve got to draw the line between Self and Other somewhere, so you might as well draw the line at 1920s Providence, Rhode Island, and call everywhere else from Boston all the way to the unspeakable abyss-city of Y’ha-nthlei just different degrees of horribleness.)
https://slatestarcodex.com/2016/07/25/how-the-west-was-won/
Quote conservatism transhumanism moral philosophy scott alexander
Scott Alexander on liberalism and universal culture
On liberalism
Liberalism is a technology for preventing civil war. It was forged in the fires of Hell — the horrors of the endless seventeenth century religious wars. For a hundred years, Europe tore itself apart in some of the most brutal ways imaginable — until finally, from the burning wreckage, we drew forth this amazing piece of alien machinery. A machine that, when tuned just right, let people live together peacefully without doing the “kill people for being Protestant” thing. Popular historical strategies for dealing with differences have included: brutally enforced conformity, brutally efficient genocide, and making sure to keep the alien machine tuned really really carefully.
https://slatestarcodex.com/2017/06/21/against-murderism/
On “Western” culture vs universal culture
I am pretty sure there was, at one point, such a thing as western civilization. I think it included things like dancing around maypoles and copying Latin manuscripts. At some point Thor might have been involved. That civilization is dead. It summoned an alien entity from beyond the void which devoured its summoner and is proceeding to eat the rest of the world.
[…]
“Western medicine” is just medicine that works. It happens to be western because the West had a technological head start, and so discovered most of the medicine that works first. But there’s nothing culturally western about it; there’s nothing Christian or Greco-Roman about using penicillin to deal with a bacterial infection. Indeed, “western medicine” replaced the traditional medicine of Europe — Hippocrates’ four humors — before it started threatening the traditional medicines of China or India. So-called “western medicine” is an inhuman perfect construct from beyond the void, summoned by Westerners, which ate traditional Western medicine first and is now proceeding to eat the rest of the world.
[…]
If western medicine is just medicine that works, soda pop is just refreshment that works.
[…]
Sushi has spread almost as rapidly as Coke. But in what sense has sushi been “westernized”? Yes, Europe has adopted sushi. But so have China, India, and Africa. Sushi is [just] another refreshment that works.
Here’s what I think is going on. Maybe every culture is the gradual accumulation of useful environmental adaptations combined with random memetic drift. But this is usually a slow process with plenty of room for everybody to adjust and local peculiarities to seep in. The Industrial Revolution caused such rapid change that the process become qualitatively different, a frantic search for better adaptations to an environment that was itself changing almost as fast as people could understand it.
The Industrial Revolution also changed the way culture was spatially distributed. When the fastest mode of transportation is the horse, and the postal system is frequently ambushed by Huns, almost all culture is local culture. England develops a culture, France develops a culture, Spain develops a culture. Geographic, language, and political barriers keep these from intermixing too much. Add rapid communication — even at the level of a good postal service — and the equation begins to change. In the 17th century, philosophers were remarking (in Latin, the universal language!) about how Descartes from France had more in common with Leibniz from Germany than either of them did with the average Frenchman or German. Nowadays I certainly have more in common with SSC readers in Finland than I do with my next-door neighbor whom I’ve never met.
Improved trade and communication networks created a rapid flow of ideas from one big commercial center to another. Things that worked — western medicine, Coca-Cola, egalitarian gender norms, sushi — spread along the trade networks and started outcompeting things that didn’t. It happened in the west first, but not in any kind of a black-and-white way. Places were inducted into the universal culture in proportion to their participation in global trade; Shanghai was infected before West Kerry; Dubai is further gone than Alabama. The great financial capitals became a single cultural region in the same way that “England” or “France” had been a cultural region in the olden times, gradually converging on more and more ideas that worked in their new economic situation.
Let me say again that this universal culture, though it started in the West, was western only in the most cosmetic ways. If China or the Caliphate had industrialized first, they would have been the ones who developed it, and it would have been much the same. The new sodas and medicines and gender norms invented in Beijing or Baghdad would have spread throughout the world, and they would have looked very familiar. The best way to industrialize is the best way to industrialize.
Universal culture is the collection of the most competitive ideas and products. Coca-Cola spreads because it tastes better than whatever people were drinking before. Egalitarian gender norms spread because they’re more popular and likeable than their predecessors. If there was something that outcompeted Coca-Cola, then that would be the official soda of universal culture and Coca-Cola would be consigned to the scrapheap of history.
The only reason universal culture doesn’t outcompete everything else instantly and achieve fixation around the globe is barriers to communication. Some of those barriers are natural — Tibet survived universalization for a long time because nobody could get to it. Sometimes the barrier is time — universal culture can’t assimilate every little hill and valley instantly. Other times there are no natural barriers, and then your choice is to either accept assimilation into universal culture, or put up some form of censorship.
Michael Nielsen is having second thoughts
My work has nearly all been about accelerating science and technology. And over the last year I find myself very conflicted. I’ve begun to question whether accelerating science and technology is a good idea at all.
https://michaelnotebook.com/ti/index.html
Quote michael nielsen catastrophic risk vulnerable world hypothesis nick bostrom