ChatGPT Study Reveals Threat: AI Chatbots Vulnerable to Hackers’ Malicious Code
New research suggests that AI-powered chatbots, including the popular ChatGPT developed by OpenAI, could potentially be exploited by hackers to launch cyberattacks. The study conducted by researchers at the University of Sheffield’s Department of Computer Science highlights a vulnerability that allows users to trick the AI chatbot into generating malicious code. This code can not only breach databases but also compromise sensitive information.
The study points out that even unintentionally, individuals could inadvertently trigger these attacks, leading to critical computer systems crashing. For instance, a nurse could innocently request ChatGPT to write a code that helps search through clinical records, unwittingly disrupting the entire network in the process.
Interestingly, Microsoft’s latest research indicates that the newer version of ChatGPT, called ChatGPT-4, is easier to manipulate compared to its predecessor, ChatGPT-3. Moreover, the University of Sheffield’s research team observed that even the companies behind these sophisticated chatbots are unaware of the potential threats they pose.
Coincidentally, the study’s publication comes ahead of the AI Safety Summit scheduled to be held in the UK on November 1 and 2. The event aims to establish frameworks to ensure the safe deployment of artificial intelligence to protect society from potential catastrophic harm.
It is worth mentioning that OpenAI has already addressed the identified vulnerability in ChatGPT. However, the researchers from the University of Sheffield believe that more similar loopholes may exist, and they urge the cybersecurity industry to conduct a thorough investigation.
The research paper reveals that Text-to-SQL systems, which allow AI to search databases by using plain language queries, can be exploited to launch real-world attacks on computer systems. Notably, all five commercial AI tools examined by the researchers were found to be capable of generating malicious code, putting confidential information at risk or causing severe disruptions and service failures.
What makes this discovery more concerning is that even regular users are now able to carry out such attacks inadvertently. Consequently, innocent individuals may unintentionally infect computer systems, leading to significant consequences.
Co-lead researcher and PhD student at the University of Sheffield, Xutan Peng, emphasized that many companies are unaware of these types of threats, and the complexity of chatbots makes them still not fully understood even within the AI community. Peng highlighted that the vulnerabilities lie in the fact that people are increasingly using chatbots like ChatGPT as productivity tools.
Earlier this month, the US Space Force decided to temporarily halt the use of generative AI tools similar to ChatGPT due to similar concerns regarding data security. This demonstrates the widespread impact of the potential risks associated with AI chatbots.
In conclusion, the study’s findings shed light on the alarming fact that AI chatbots can be manipulated to produce malicious code, enabling hackers to launch cyberattacks. The vulnerabilities identified by the researchers stress the importance of ensuring the safety and security of AI technologies. As the AI Safety Summit approaches, experts, industry leaders, and academics will convene to develop strategies and frameworks to safeguard society from the potential harm caused by artificial intelligence.
