OpenAI Forms Preparedness Team to Study Catastrophic Risks of Frontier AI
OpenAI, the renowned creator of ChatGPT, has announced the formation of a new team called Preparedness to specifically examine the potential catastrophic risks associated with frontier AI models developed by the company. Led by Aleksander Madry, Director of MIT’s Center for Deployable Machine Learning, the Preparedness team will focus on analyzing serious threats posed by artificial intelligence technologies, including Chemical, Biological, Radiological, and Nuclear (CBRN) risks.
Highlighting the need for safety in highly-capable AI systems, OpenAI aims to address the increasingly severe risks that come with the development of frontier AI models. While these models hold the potential to benefit humanity significantly, they also pose significant dangers. By establishing the Preparedness team, OpenAI intends to minimize these risks as AI models continue to advance.
The Preparedness team’s responsibilities will involve close collaboration and assessment of the capabilities, evaluations, and internal red teaming for frontier models. This assessment will span from the near-future models to those with Artificial General Intelligence (AGI)-level capabilities. The team will also be responsible for monitoring, evaluating, and forecasting catastrophic risks across several categories, such as individualized persuasion, cybersecurity, CBRN threats, as well as autonomous replication and adaptation.
In addition to addressing risks, the Preparedness team will be tasked with developing a Risk-Informed Development Policy (RDP). This policy will outline OpenAI’s approach to future AI model development, incorporating protective measures and establishing an accountable government structure for oversight.
OpenAI has further taken a proactive step in preventing catastrophic misuse of AI by launching the AI Preparedness Challenge. This challenge invites submissions proposing ideas for prevention and offers a prize of $25,000 in API credits to the top 10 submissions.
With the formation of the Preparedness team and the introduction of the AI Preparedness Challenge, OpenAI showcases its commitment to proactively identifying and mitigating potential risks associated with the advancement of AI models. By placing a strong emphasis on safety, OpenAI aims to ensure that the benefits of frontier AI are harnessed responsibly for the betterment of humanity.
[Link: Mint is now on WhatsApp Channels 🚀 Subscribe today by clicking the link and stay updated with the latest financial insights!]
