If ASI Suddenly Appears, Everyone Dies

IABIED contains a gripping horror story that will make a riveting disaster movie (if only via GenAI).  But as a plea to halt development of AGI, it has serious shortcomings. Its primary weakness is that it's uninformed by any traces of economics, such as growth theory, development economics, institutional analysis, organization theory, progress studies, or information economics. The authors have engaged those topics elsewhere, but apparently considered them a distraction for the audience of this book.

Even so, IABIED commits the traditional sin of economists epitomized by the joke "First, assume a can opener." The book should have been titled "If ASI Suddenly Arrives, Everyone Dies".  IABIED assumes that ASI will arrive suddenly and takes zero notice of arguments to the contrary. It makes no mention of

• bottlenecks
• diminishing returns
• local knowledge problems
• Brooks's Law
• scaling limits of global compute/power infrastructure
• data walls
• limits of self-play in non-deterministically-scored problem domains
• limits of synthetic data

IABIED probably did not have space to dive into all of the above considerations, but it didn't even mention them or give them a QR code. Below is a screenshot of all the text I could find in IABIED that describes how or why ASI might suddenly arrive. It's weak soup. In the authors' defense, I'm in the choir when they're preaching about misalignment. (I think alignment is impossible under fast take-off, and probably not needed under slow take-off.) Perhaps they think misalignment is the crux for most people.

IABIED was an entertaining read, with educational discussions about Chernobyl, Mars probes, and Thomas Midgley Jr.  (Besides leaded gas and CFCs, they didn't further dunk on the poor guy by noting that he was strangled by another of his inventions: a harness to lift him out of his polio bed.)

IABIED is a skillfully written, passionate argument for the doomer position. Its purpose clearly isn't to be an unrebuttable doomer manifesto or even a comprehensive overview of the argument space. Be skeptical of anyone who treats it as such.