Advanced persistent threats (APTs) aligned with China, Iran, North Korea, and Russia have been utilizing large language models (LLMs) to bolster their cyber operations, according to recent blog posts by OpenAI and Microsoft. The two tech giants have uncovered evidence that shows these nation-state actors using OpenAI software for malicious purposes, including research and fraud. In response, OpenAI has taken action to shut down all their accounts. However, the good news is that the observed abuses of LLMs have not been particularly devastating thus far, as these threat actors mostly treat AI as a productivity tool rather than utilizing it for innovative attack techniques.
Among the notorious nation-state APTs identified using OpenAI technology is the group known as Fancy Bear or Forest Blizzard, which is affiliated with Russia’s GRU. Fancy Bear has been using LLMs for various tasks such as intelligence gathering, researching satellite communication protocols, and radar imaging technologies. In addition, two Chinese state actors called Charcoal Typhoon and Salmon Typhoon have also been utilizing AI, with the former using it for pre-compromise malicious behaviors and generating social engineering texts, while the latter has focused on using LLMs as an intelligence tool.
Iran’s Crimson Sandstorm has been found to use OpenAI for developing phishing materials and code snippets to aid in their cyber operations. They create deceptive emails pretending to be from international development agencies or feminist groups. Furthermore, they leverage OpenAI for web scraping and executing tasks upon user sign-in. Lastly, Kim Jong-Un’s Emerald Sleet has been using OpenAI for basic scripting tasks, generating phishing content, and researching publicly available information on defense issues and the country’s nuclear weapons program.
Despite the potential of AI-enhanced nation-state cyber operations, experts have highlighted that the current use of LLMs by threat actors does not constitute a groundbreaking revolution. Joseph Thacker, a principal AI engineer and security researcher, implies that the malicious actors tracked by Microsoft were already proficient software writers. While LLMs can make their tasks more efficient, they are not doing anything novel. However, Thacker cautions that AI still confers advantages for attackers, such as facilitating larger-scale deployment of malware and code translation for exploiting systems previously unsupported.
Thacker advises companies to remain vigilant and adhere to fundamental security practices. He emphasizes the importance of consistent adherence to basic security measures to better safeguard organizations against potential AI-driven threats. Although no groundbreaking AI-enabled attack techniques have been observed yet, the possibility of threat actors finding novel ways to exploit AI cannot be ruled out. Nevertheless, the primary course of action remains focusing on sound security practices.
In summary, nation-state APTs from China, Iran, North Korea, and Russia have been utilizing OpenAI’s language models for research, fraud, and malicious purposes. While AI-enhanced cyber operations have the potential to be significant, current observations show that these actors use LLMs as productivity tools without introducing particularly unique attack techniques. However, caution is advised, as AI can still provide advantages to attackers, necessitating ongoing vigilance and adherence to security best practices.
Please note that the above news content is generated by OpenAI as per the given instructions.
