Riva Tez on soap
People no longer understand ambiance or aesthetics because they’re too distracted by the macro. We have lost the ability to notice details. Once you become a walker, you realize how much people are missing out on life. On walks, you see how perfect light can be. You see how many shades of blue exist between different flowers and skies. You notice the Grandmother who tends to her flowers. You feel her love as she caresses them. You realize that we can’t automate everything.
A friend complained to me that his girlfriend had dragged him to a Parisian perfume store to pick out hand soap, as if it were a frivolous purchase. I thought she sounded marvelous. These are your hands! I told him. Your hands are your primary tools! They are what hold your pen and your sword! You wash your hands every day with soap. It should be the soap of kings, made of the finest ingredients, naturally fragrant with a scent that means something to you. Every time you wash your hands, take a moment to notice and feel the experience on your skin. People often overlook the significance of these small details. Joy starts with noticing details. Joy begins with realizing that even the most seemingly mundane and repetitive actions can be pleasurable. Once you’ve realized that, you can never be depressed.
It’s happening (extreme risks from AI)
Jack Clark’s newsletter (my emphasis):
The things people worry about keep on happening: A few years ago lots of people working in AI safety had abstract concerns that one day sufficiently advanced systems might start to become pathologically sycophantic, or might ‘fake alignment’ to preserve themselves into the future, or might hack their environments to get greater amounts of reward, or might develop persuasive capabilities in excess of humans. All of these once academic concerns have materialized in production systems in the last couple of years.
And:
“As far as we know this is the first time AI models have been observed preventing themselves from being shut down despite explicit instructions to the contrary,” Palisade writes. “While experiments like ours have begun to show empirical evidence for AI models resisting shutdown, researchers have long predicted that AIs would learn to prevent themselves from being shut down to achieve their goal.”
As with the persuasion example, the story of contemporary AI research is that risks once deemed theoretical - ability to contribute to terrorism, skill at persuasion, faking of alignment, and so on - are showing up in the real systems being deployed into the economy.
Human takeover might be worse than AI takeover
In Tom Davidson’s words:
In expectation, future AI systems will better live up to human moral standards than a randomly selected human. Because:
- Humans fall far short of our moral standards.
- Current models are much more nice, patient, honest and selfless than humans.
- Humans are “rewarded” for immoral behaviour more than AIs will be.
- Humans evolved under conditions where selfishness and cruelty often paid high dividends, so evolution often “rewarded” such behaviour. And similarly, during lifetime learning humans often benefit from immoral behaviour.
- But we’ll craft the training data for AIs to avoid this, and can much more easily monitor their actions and even their thinking. Of course, this may be hard to do for superhuman AI, but bootstrapping might work.
https://forethoughtnewsletter.substack.com/p/human-takeover-might-be-worse-than
Automation of most white-collar work—already baked in?
Some senior figures at frontier labs think that even today’s machine learning algorithms are already sufficient to automate most white-collar work (without the need for major algorithmic breakthroughs or much larger models).
On their perspective, the current bottlenecks are simply:
- Frontier labs (and independent startups) having enough staff to create training datasets, and perform fine-tuning.
- Inference compute.
Currently, these constraints are acute.
But these lab figures think the capabilities are there, and the economics viable, even with current technology. We just need to roll it out.
I’m not close enough, or technical enough, to have a confident take on this. But, it certainly strikes me as plausible, especially after a few weeks with o3. Inside view: >1/3.
How well will it generalise?
We now know that “LLMs + simple RL” is working really well.
So, we’re well on track for superhuman performance in domains where we have many examples of good reasoning and good outcomes to train on.
We have or can get such datasets for surprisingly many domains. We may also lack them in surprisingly many. How well can our models generalise to “fill in the gaps”?
It’s an empirical question, not yet settled. But Dario and Sam clearly think “very well”.
Dario, in particular, is saying:
I don’t think it will be a whole bunch longer than [2027] when AI systems are better than humans at almost everything, and then eventually better than all humans at everything.
And:
“I am more confident than I have ever been at any previous time that we are very close to powerful capabilities. […] I think until about 3 to 6 months ago I had substantial uncertainty about it. I still do now, but that uncertainty is greatly reduced. I think that over the next 2 or 3 years I am relatively confident that we are indeed going to see models that … gradually get better than us at almost everything.
Gwern doubts Dario’s view, but he’s not inside the labs, so he can’t see what they see.
