Robin Hanson on value drift

Value drift is just a generic problem for humans, ems or AI. It’s just what’s always happened so far. It’s the default for what will happen in the future. If you hate it you’re in trouble because it’s really really likely. Some people for some reason think that value drift in humans is bounded well, while value drift in machines is not only unbounded but happens quickly and I don’t really see the grounds for that. […] The main thing is that in the past when value drift happened change was so slow that you didn’t see it in your lifetime so you didn’t worry very much about it. As change gets faster and your lifetimes get longer your life will encompass more value drift. And then whether it’s humans or machines or whatever you will see it, and if you don’t like it then you will see something you don’t like.

Book talk at The Foresight Institute.

quote robin hanson futurism metaethics