Trial and error is great until it kills you
Preserving the institutions that correct errors is more important than getting it right first time.
—David Deutsch
Aus der Kriegsschule des Lebens—was mich nicht umbringt, macht mich stärker.
—Friedrich Nietzsche
David Deutsch, channelling Popper, is right to stress the importance of error correction. But I really hope it is not the only way we can learn. Because sometimes we face “one shot” problems, where we need to get it right first time.
As individuals, if we make a fatal mistake, it’s bad for us, but it’s not the end of humanity. As culture and as species, we “learn” from the mistakes of individuals; our norms and genomes evolve. And by this mechanism, our descendants become less likely—individually—to make fatal mistakes when faced with “one shot” problems. Centrally: individuals develop an ability to detect and avoid situations that involve risk of ruin.
On the most disturbing read, the Vulnerable World Hypothesis involves a claim that we are approaching one or more “one shot” problems, at the species level. If we err, we wipe ourselves out—we don’t get a chance to try again.
If we are on track to develop technologies that generate “one shot” extinction risks, it seems clear that “trial and error” isn’t a sustainable strategy. We probably need to develop our ability—as a species—to detect and steer away from situations that involve risk of ruin. And it’d be nice to do this by design—not by species-level selection.
writing david deutsch epistemology existential risk