South Korean cybersecurity researchers reported that a North Korea-backed hacking group had used ChatGPT to generate a forged military identification card. The attack has been attributed to Kimsuky, a long-standing APT unit, which deployed the fake ID as part of phishing campaigns to gain access to South Korean targets’ systems. The forged document was supplemented with an AI-generated image, produced through prompt engineering techniques that bypassed ChatGPT’s built-in safety restrictions.
The findings were disclosed by South Korean cybersecurity firm Genians. According to their report, the phishing email was disguised as an official military communication requesting the “review of a draft military ID design”. The attached forged ID redirected victims to a malicious ZIP file, which installed malware once opened. This approach demonstrated how AI-generated content was deliberately used to increase the psychological credibility of the cyberattack, making it more difficult for victims to detect the deception.
The incident highlights how vulnerabilities in generative AI systems can provide strategic advantages to state-sponsored hacking groups. While OpenAI’s safeguards prohibit the creation of forged documents, the attackers successfully circumvented them and weaponised AI for malicious use. The September 2025 case thus serves as further evidence that artificial intelligence constitutes a global security risk, requiring urgent responses from both governments and technology providers.
Sources:
1.
Jane Lanhee Lee,Bloomberg
2.
Jane Lanhee Lee
3.
TOI Tech Desk
