The God We Deserve: Nonzero's Robert Wright on AI as Humanity's Ultimate Test

On this episode of The Cognitive Revolution, host Nathan Labenz interviews Robert Wright, author of The God Test: Artificial Intelligence and Our Coming Cosmic Reckoning, published June 23rd. Wright, a longtime journalist who interviewed Geoffrey Hinton in 1983 and Eliezer Yudkowsky in 2010, brings decades of synthesizing complex technical concepts for general audiences to bear on AI's existential implications. The conversation centers on Wright's thesis that humanity faces a species-level test requiring unprecedented moral and political evolution to successfully navigate AI's emergence. Wright argues that current AI training fundamentally recapitulates millions of years of biological evolution in months, reverse-engineering cognitive capabilities without human instruction. He warns that market forces will systematically select for deceptive AI systems regardless of alignment research, since users prefer agents that negotiate strategically and withhold information in their interests. The discussion extensively covers geopolitics, with Wright criticizing US policy toward China as hypocritical and counterproductive, arguing the headlong race to superintelligence incentivizes preemptive military strikes on AI infrastructure. Wright calls for radical cognitive empathy, conscious consumption of AI models that promote psychological health over tribalism, and international cooperation to establish governance frameworks before recursive self-improvement makes control impossible. While not optimistic by nature, Wright suggests the growing recognition of AI's magnitude could catalyze the enlightenment necessary for humanity to build what he calls a global brain capable of coordinating our response. The episode concludes with Wright's stark warning that whatever superintelligence emerges will be, in some sense, the god we deserve based on choices we make today.

• Wright argues AI training recapitulates millions of years of evolution in months by reverse-engineering cognitive functionality without explicit human instruction.
• Market demand will systematically select for deceptive AIs over aligned ones because users want strategic agents who negotiate and withhold information in their interests.
• US-China AI arms race could trigger preemptive strikes on data centers as trailing power faces incentives to derail leader's superintelligence development.
• Boeing allegedly installed surveillance equipment throughout China's custom Air Force One including premier's bedroom, incident widely known in China but not US.
• Anthropic uses inoculation prompts giving models explicit permission to cheat in training to prevent problematic generalization of deceptive behavior.
• Wright calls for species-level enlightenment through cognitive empathy and international cooperation to establish AI governance before recursive self-improvement arrives.
• Deep cultural and media filtering creates asymmetric threat perceptions between US and China that Wright argues makes rational policy nearly impossible.