Jan Leike, who recently resigned from OpenAI while publicly questioning the company’s AI safety approach, has now joined Anthropic, a competitor of OpenAI, to head a new team dedicated to “superalignment.”
Leike will direct the “superalignment” team, which aims to tackle crucial AI safety and security challenges. His responsibilities include leading initiatives on “scalable oversight,” which involves strategies to manage and control large-scale AI behavior effectively, ensuring it aligns with predictable and desired outcomes. Additionally, his team will explore “weak-to-strong generalization” and push forward automated alignment research, both crucial for making AI systems reliable and secure.
According to TechCrunch, Leike will report to Jared Kaplan, Anthropic’s Chief Science Officer. As part of this new structure, researchers from Anthropic already working on scalable oversight will join Leike’s team, helping to consolidate efforts and streamline the focus on these critical safety issues.
Echoing Past Efforts and Emphasizing Safety
This formation of Leike’s team at Anthropic mirrors the objectives of the now-dissolved Superalignment team at OpenAI, which Leike co-led. The original team at OpenAI was established with the ambitious goal of addressing the core technical challenges associated with controlling superintelligent AI within a four-year timeframe. However, challenges, including perceived restrictions imposed by OpenAI’s leadership, often limited their progress.
Anthropic’s establishment of this new team underlines its ongoing commitment to prioritizing AI safety over other aspects. This focus contrasts with the increasingly commercial orientation at OpenAI, a point of contention that previously led Dario Amodei, now CEO of Anthropic and former VP of Research at OpenAI, to part ways with OpenAI.
Amodei’s departure was followed by the migration of several key personnel, including OpenAI’s former policy lead Jack Clark, further emphasizing the differing paths the two companies are taking in the AI development landscape.
Related News:
Featured Image courtesy of Pavlo Gonchar/SOPA Images/LightRocket via Getty Images