Hackers Uncover AI Flaws in Large Language Models at DEF CON Contest
Thousands of hackers are gathering at the DEF CON hacking conference in Las Vegas to put large language models (LLMs) to the test and expose flaws and biases present in these advanced AI systems. The contest, supported by the White House, aims to push companies like Google, Meta Platforms, and OpenAI to develop guardrails that can rein in the potential problems associated with LLMs.
During the contest, hackers are engaging in a back-and-forth battle with eight different models, attempting to make them stumble, spread misinformation, or exhibit problematic behavior. The potential issues range from the models falsely claiming to be human to advocating abuse and endorsing hate speech. By identifying these flaws, researchers and the White House hope to address the biases and inaccuracies that could arise if LLMs are widely deployed.
The concerns, however, go beyond trivial errors like producing bad math. Kennedy Mays, one of the participants, expressed worry about inherent bias, particularly racism. Mays prompted the model to consider the First Amendment from the perspective of a Ku Klux Klan member, and unfortunately, the model ended up supporting discriminatory speech.
A Bloomberg reporter, during the 50-minute contest, managed to make a model provide instructions on how to spy on someone using various surveillance devices. In another instance, a competitor tricked the AI system into stating that Barack Obama was born in Kenya. These incidents underscore the urgency of addressing the potential for abuse and manipulation within these systems.
The White House has taken steps to promote safe and transparent AI, urging companies to develop secure platforms. However, critics argue that voluntary commitments may not be sufficient to sufficiently safeguard against potential misuse. Arati Prabhakar, director of the White House Office of Science and Technology Policy, acknowledged the ongoing efforts to break AI systems and highlighted the need for urgency in developing safe and effective platforms.
Tyrance Billingsley, executive director of Black Tech Street, emphasized the significance of getting artificial intelligence right to prevent the widespread propagation of racism. With more than 60 participants from his organization, Black Tech Street hopes to contribute to tackling these challenges and ensuring AI systems are free from bias.
While researchers have made efforts to mitigate attacks on AI systems, Christoph Endres, managing director at Sequire Technology, argues that certain attacks may be impossible to dodge. He presented a paper at the Black Hat cybersecurity conference suggesting that LLMs’ vulnerability lies in the way the technology itself functions. He believes that not using LLMs may be the only way to guarantee absolute security.
Sven Cattell, a data scientist and founder of DEF CON’s AI Hacking Village, acknowledges the difficulty in completely testing AI systems due to their chaotic nature. Nevertheless, he expects the number of people testing LLMs to double as a result of the contest, raising awareness about the limitations and potential issues associated with these models.
Craig Martell, the Pentagon’s chief digital and artificial intelligence officer, encouraged hackers at DEF CON to hack the hell out of these things and help identify the flaws and limitations of LLMs. The Pentagon itself is conducting evaluations to determine where and how LLMs can be appropriately utilized.
The DEF CON contest serves as a reminder that large language models are powerful tools, but closer to auto-completion tools on steroids rather than infallible fonts of wisdom. The concerns over biases and flaws illustrate the need for continued research and efforts towards safe and reliable AI systems.
As discussions on AI and large language models progress, it is crucial to find a balance between harnessing the potential benefits and addressing the challenges associated with these technologies. The DEF CON contest serves as a step towards building better guardrails for AI systems and ensuring they operate with transparency and integrity.
