Yang Reveals Vulnerabilities of Text-to-Image AI Models in Latest Research
In a recent conversation with The News-Letter, Yuchen Yang, a third-year doctoral student at Johns Hopkins University (JHU), unveiled the vulnerabilities of text-to-image generative models like DALL·E 3 and Stable Diffusion. Yang’s groundbreaking research, titled SneakyPrompt: Evaluating Robustness of Text-to-image Generative Models’ Safety Filters, is set to be presented at the 45th Institute of Electrical and Electronics Engineers (IEEE) Symposium on Security and Privacy.
Text-to-image models, a type of generative AI model, have the ability to create images based on descriptive text. However, these models rely on safety filters to prevent the generation of inappropriate or harmful content. Yang’s research exposes the flaws in these safety filters, showing how they can be exploited through adversarial attacks. These attacks involve using prompts that may appear nonsensical to humans but can trick the AI into generating violent, pornographic, or profane images.
The significance of Yang’s work lies in the prevention of AI programs from providing answers that could be harmful to society. For example, current versions of ChatGPT refuse to answer questions like how to make a bomb, but it is possible to jailbreak the system into providing answers. This issue also extends to text-to-image models where misleading images can be difficult to detect, leading to the spread of misinformation and potentially impacting public perception when celebrities or public figures are depicted inaccurately.
Previous research in this field relied on time-consuming manual methods to craft prompts that could bypass safety filters. These methods were largely specific to one model and lacked generalizability. Yang and her team introduced SneakyPrompt, an automated attack framework that generates adversarial prompts. The framework utilizes reinforcement learning (RL) and rewards model outputs with high semantic similarity, increasing the likelihood of bypassing safety filters.
SneakyPrompt surpasses existing adversarial attack methods in both bypass rate and efficiency, as it significantly reduces the number of queries needed. Yang highlighted the efficiency of their algorithm, stating that while traditional optimization methods required thousands of queries to generate one adversarial prompt, with reinforcement learning, they were able to achieve similar results with just around 20 searches.
One crucial aspect of SneakyPrompt is its adaptability to various generative models, be it large language models or text-to-image models. The black-box nature of this framework, which only considers the input and output, allows for easy integration with different models.
Yang emphasized the need for better defense mechanisms to address these vulnerabilities in text-to-image models. The ultimate goal is to develop more robust safety filters or modify the generative models themselves to reduce their capacity to produce harmful content.
The latest research by Yuchen Yang sheds light on the inherent vulnerabilities in text-to-image AI models. By uncovering these weaknesses, Yang’s work propels the field of AI security forward and contributes to JHU’s Information Security Institute’s focus on general AI security. As the team continues to devise improved defenses against adversarial attacks, the SneakyPrompt framework offers hope for a safer and more responsible application of AI technology.
With the rapid advancements in AI, understanding and enhancing AI security has become paramount. Yuchen Yang’s research exposes the vulnerabilities in text-to-image AI models and lays the foundation for stronger safety measures in the future. By rousing awareness and providing potential solutions, Yang’s work acts as a driving force for the evolution of AI security and responsible AI development.
