“Offensive AI” spans adversaries using mainstream LLMs for productivity (reconnaissance, phishing text, translation, scripting, vulnerability research), purpose-built malicious LLMs, generative deepfakes for fraud and impersonation, and the newer frontier of AI embedded in malware or orchestrating agentic intrusions. The consistent finding across credible threat intelligence is that AI is currently a force multiplier — making existing attacks faster, cheaper and more scalable — rather than a source of fundamentally novel capability, with early 2025–2026 signals that this is beginning to shift.
What Is Actually Happening
Google’s Threat Intelligence Group, in “Adversarial Misuse of Generative AI” (January 2025), assessed that APT and information-operations actors gained productivity but no novel capabilities, with heaviest usage from Iran and China (also DPRK, and less so Russia). By its November 2025 AI Threat Tracker, the picture was shifting: GTIG reported adversaries deploying novel AI-enabled malware — named families PROMPTFLUX and PROMPTSTEAL — that generate or obfuscate code at runtime, plus a maturing criminal marketplace. Anthropic’s August 2025 threat reporting documented “vibe hacking”: an actor using an AI coding tool to automate reconnaissance, credential harvesting and intrusion across multiple organisations — one operator doing a team’s work.
Hype Versus Real
- Real: AI-assisted phishing and social engineering at scale and in better language; AI used across the attack lifecycle for productivity; early, nascent AI-enabled malware; and a genuine shift toward smaller teams operating at larger scale.
- Overstated: the original 2023 WormGPT/FraudGPT panic (weak tools, some scams); and sweeping claims of autonomous AI attacks. Even capable 2025 malicious LLMs often produce detectable, known-pattern code (Unit 42). Specific viral loss figures should be treated cautiously.
The Frameworks That Structure the Defence
| Framework | Scope |
|---|---|
| MITRE ATLAS | ATT&CK-style knowledge base of attacks on AI systems (tactics, techniques, real case studies) |
| OWASP Top 10 for LLM Applications | Application-security risks of LLM deployments (LLM01 Prompt Injection, LLM06 Excessive Agency, …) |
| NIST AI RMF (AI 100-1) + AI 100-2e2025 | Governance/risk framework (Govern/Map/Measure/Manage) plus the adversarial-ML taxonomy |
| CISA / NCSC Guidelines for Secure AI System Development | Secure-by-design guidance (Nov 2023) co-signed by 20+ agencies, across the AI lifecycle |
| MITRE ATT&CK | Conventional adversary TTPs — AI mostly accelerates these rather than adding new ones |
Keep these distinct: ATT&CK describes attacks carried out with AI into normal infrastructure; ATLAS describes attacks against AI systems; OWASP covers LLM application risks.
What Defenders Should Do
- Adopt the NIST AI RMF and the CISA/NCSC secure-AI lifecycle for any AI you build or deploy; inventory AI systems and their data and tool access.
- Harden against prompt injection and excessive agency — the dominant application-layer risk for copilots and agents.
- Assume AI-accelerated phishing and deepfakes: strengthen identity verification for high-value actions (out-of-band callbacks; never approve payments on voice or video alone).
- Keep conventional detection strong — AI-generated malware is often still known-pattern and detectable, so EDR, egress control and behavioural analytics remain effective.
- Threat-model with ATLAS, red-team with tools like garak, and follow vendor threat intelligence (GTIG, Anthropic, Microsoft) to track the real capability curve rather than headlines.
Build an AI-Aware Security Programme
Mjolnir Security helps organisations govern, threat-model and defend against offensive AI.
- AI Security Advisory Applying NIST AI RMF and CISA/NCSC secure-AI guidance to your estate.
- Managed Detection and Response Detection that holds up against AI-accelerated attacks.
- 24/7 Incident Response Containment when AI-enabled attacks are suspected. Call +1 833 403 5875.
Stay ahead of emerging threats. Get notified when we publish new intelligence reports and advisories.
Subscribe to Alerts