Q&A with AI researchers Jacob Coxon, Daniel Kokotajlo, Alex Turner, and others on leaving AI labs, AGI's dangers, planned IPOs, AI safety, oversight, and more

AI researchers are leaving leading labs, making public their concerns about the pace of AI development and its potential dangers.

Jacob Coxon, a former capabilities researcher at Anthropic, publicly resigned citing concerns about the company's rapid pursuit of artificial general intelligence (AGI). Coxon's resignation statement gained significant attention, highlighting internal dissent within leading AI labs like OpenAI, Anthropic, and Google DeepMind. He expressed fear regarding the projected capabilities of future AI models, particularly the potential for them to become self-directed in research. Coxon's decision echoes similar concerns voiced by other researchers, such as Daniel Kokotajlo, who left OpenAI due to a loss of confidence in the company's responsible behavior. These researchers often face a dilemma: agitate from within or speak out externally, with both paths presenting challenges to their careers and community standing. Some researchers also cited frustrations with the commercialization of once idealistic non-profits and partnerships with the Department of Defense as reasons for departure.

The public resignations of researchers like Jacob Coxon and Daniel Kokotajlo signal a growing internal unease within top-tier AI labs regarding the race towards AGI. This outward expression of dissent suggests that internal advocacy for safety and caution is failing to adequately influence development trajectories, pushing individuals to seek impact through public statements and external research. It raises questions about the effectiveness of internal governance and self-regulation within these powerful organizations.

This trend of high-profile departures and subsequent public commentary indicates a potential shift in how AI safety concerns are addressed, moving from internal memos to external pressure. The personal and professional cost for these researchers, described as akin to 'renouncing citizenship,' highlights the intense pressures and potential career repercussions of voicing such criticisms. It also underscores the dilemma faced by many: to remain within a system they fear, or to exit and risk marginalization while attempting to influence the broader discourse and regulatory landscape.