Generative AI Risks: A Practitioner's Guide to What Matters

Generative AI risks are materializing faster than enterprise security programs can adapt. Unlike the incremental threat surface expansion that came with cloud or mobile adoption, generative AI introduces entirely new attack classes (prompt injection, model confabulation, supply chain poisoning, autonomous agent exploitation) while simultaneously amplifying existing threats like phishing, social engineering, and malicious code generation.

The scale of the problem is measurable. According to Cisco’s State of AI Security 2026 report, 83 percent of organizations planned to deploy agentic AI capabilities, but only 29 percent felt they were truly ready to do so securely. Nation-state actors including Russia, China, Iran, and North Korea have more than doubled their use of AI in offensive operations, using it to translate phishing emails, generate deepfake content, and build malware that adapts in real time to evade detection. NIST data shows a greater than 2,000% increase in AI-specific CVEs since 2022, and industry surveys put 68% of organizations as having already experienced data leaks tied to AI tool usage.

This is not a forward-looking risk assessment. These threats are active. Here is what they are and what defenses apply, with the running incident coverage collected in the AI security threat intelligence hub.

How NIST AI 600-1 Maps the Generative AI Risk Landscape

In July 2024, NIST released AI 600-1, a companion profile to the AI Risk Management Framework specifically targeting generative AI. The document, developed with input from more than 2,500 members of a public working group, identifies 13 distinct risk categories and more than 400 recommended actions across the AI lifecycle.

Confabulation is listed first and for good reason. In enterprise deployments where GenAI summarizes legal documents, drafts contracts, or assists with clinical guidance, a hallucination can be indistinguishable from accurate output without human review. NIST distinguishes between intrinsic confabulation, where the model generates errors independently, and extrinsic confabulation, where errors are introduced by retrieval failures in RAG pipelines.

Data privacy risks stem from models trained on sensitive information or fine-tuned on organizational data containing PII, health records, or trade secrets. Privacy attacks including membership inference and model inversion can extract this information from deployed models without access to the original training data.

Harmful bias and homogenization reflect that large language models encode statistical patterns from their training corpora. When those corpora reflect historical biases, the model reproduces them at scale. Homogenization introduces a different but related risk: multiple organizations running similar base models means a single adversarial input strategy can affect many deployments simultaneously.

Cybersecurity misuse is where generative AI intersects directly with offensive security operations. Models can generate functional exploit code, draft spear-phishing emails that mimic specific writing styles, and automate reconnaissance at speeds manual operations cannot match. This is no longer a theoretical concern. Cisco’s report documents active exploitation of AI-assisted attack tooling in 2025 and into 2026.

Additional NIST risk categories include intellectual property and copyright exposure, value chain and component-level supply chain risks, data quality degradation, and the organizational accountability gaps that emerge when responsibility for model behavior in production is undefined.

Prompt Injection: The Unsolved Number One Vulnerability

While NIST provides a framework-level view, OWASP’s Top 10 for Large Language Model Applications offers a technical attack-surface taxonomy that maps directly to development and security review workflows. Each of its ten categories, and the controls that reduce exposure to them, is broken down in the LLM security risks guide. Prompt injection (LLM01) has held the top position since the list’s first release, and it remains the most operationally significant risk in production deployments.

The attack is straightforward: an adversary crafts input that causes the model to deviate from its intended behavior, executing attacker-supplied instructions instead of operator-defined ones. OWASP distinguishes two variants. Direct injection arrives through the user prompt itself. Indirect injection is embedded in external content the model retrieves and processes, such as a web page, a document, or an email, without the user or operator seeing it. The attacker does not need access to the system; they only need the model to fetch content they control. An agent browsing the web or reading a calendar invite can encounter injected instructions that redirect its actions without any visible user interaction.

The consequences range from information disclosure to full command execution in agentic pipelines where the LLM can invoke tools, APIs, or external services. OpenAI acknowledged in a late-2025 security update for ChatGPT’s Atlas browser that prompt injection “is unlikely to ever be fully solved,” comparing it structurally to social engineering on the web.

Mitigations reduce exposure without eliminating it: constrain model behavior through explicit system prompts, validate and sanitize all external content before it reaches the model context window, enforce least-privilege on every tool the model can invoke, and require human approval before the model executes high-impact actions. Detailed technical coverage of prompt injection variants and documented real-world exploits is tracked at aisec.blog.

Data Leakage: The Shadow IT Problem, Amplified

Data leakage through generative AI tools is not hypothetical. Eleven percent of data employees paste into ChatGPT is confidential, according to research cited in the Practical DevSecOps 2026 AI Security Statistics report. Samsung banned ChatGPT enterprise-wide in 2023 after engineers pasted proprietary source code into the platform. The pattern repeats across industries because employees reach for AI tools faster than IT can provision governed alternatives.

The exposure categories are consistent across incidents: source code, internal documents, personally identifiable information, and credentials. The receiving systems are largely cloud-based SaaS LLM platforms that retain input data for model improvement under default settings, meaning a confidential document pasted into a chat session may enter a training dataset.

The governance gap is stark. Only 23% of organizations have a formal AI security policy despite 68% having already experienced AI-related data leaks. Cisco’s 2026 report reaches the same conclusion from a different direction: the pace of AI adoption consistently outpaces security team involvement, creating an environment where risk acceptance is implicit rather than deliberate. Employees using unapproved generative AI tools that process organizational data are frequently invisible to DLP controls that predate LLM adoption.

Practical controls: deploy a data loss prevention layer that inspects traffic to AI endpoints, maintain an approved tool list rather than relying on informal norms, and audit which data categories are permitted in AI workflows. Sensitive data classification that predates the LLM rollout is the foundation, because you cannot protect data categories you have not defined.

Hallucinations and Fabricated Output

Hallucinations are a distinct risk category from adversarial attacks. The model generates plausible but false content without external manipulation. The risk to organizations is reputational, legal, and operational.

In 2025, Deloitte agreed to partially refund the Australian government after a consulting report delivered using AI assistance contained fabricated citations, including a quote attributed to a federal court judgment that did not exist. The pattern is consistent with how LLMs fail: confident presentation of invented facts in contexts where citation accuracy is verifiable and consequential.

For security operations specifically, the failure mode surfaces in AI-assisted threat intelligence, where a model may fabricate CVE details, vendor advisories, or attribution claims that analysts then act on. AI-generated incident reports and runbooks carry the same risk.

Controls: treat all model output as unverified draft material in workflows where accuracy is consequential, require human verification against authoritative sources before output is acted on, and implement output validation pipelines that check claims against trusted data stores. ai-alert.org tracks real AI incidents and ML vulnerability disclosures as a cross-reference resource.

Supply Chain and Poisoning Risk

Supply chain vulnerabilities (LLM03) affect any organization relying on third-party base models, fine-tuning datasets, plugins, or inference infrastructure. A compromised open-source model uploaded to a public hub, or a poisoned dataset used during fine-tuning, can introduce backdoors that activate on specific trigger inputs. Cisco’s 2026 report specifically flags the AI supply chain as a priority attack surface, noting that organizations integrating external AI components at speed may have bypassed traditional security vetting.

Data and model poisoning (LLM04) and sensitive information disclosure (LLM02) complete the most critical enterprise risk set. Poisoning attacks corrupt model behavior at training time, producing no runtime anomalies that monitoring systems can catch. Disclosure risks, where models surface training data, system prompts, or session data from other users, persist throughout the deployment lifecycle and are particularly acute in multi-tenant SaaS environments where data isolation may be incomplete.

Agentic AI: A New Attack Surface Tier

Excessive agency (LLM06) is the defining risk of the agentic AI era. As LLMs are granted access to APIs, databases, code execution environments, and communication channels, the blast radius of a successful compromise expands dramatically. An agent with write access to a file system, email client, and enterprise calendar presents a fundamentally different risk profile than a text-generation chatbot, and the 2026 supply chain and misconfiguration incidents in how AI agents are rewriting the threat model show what that difference costs in practice.

OWASP’s December 2025 release of the Top 10 for Agentic AI Applications identified the key threat classes: agent behavior hijacking via prompt injection into the planning layer, tool misuse where an agent invokes capabilities beyond intended scope, identity and privilege abuse in multi-agent pipelines where one agent delegates to another with elevated permissions, memory poisoning that corrupts the agent’s persistent context, and cascading hallucinations where false output from one step becomes a trusted input for the next.

The attack surface for an agentic system is the sum of every API, database, file system, and external service the agent can touch. Traditional application security gates (authentication, authorization, input validation) still apply at each integration point, but the orchestration layer itself becomes an attack target. An adversary who can influence the agent’s planning process can direct it to use its legitimate access for illegitimate purposes.

Security controls for agentic deployments must include explicit capability declarations scoped to the minimum the agent needs, structured output validation at every step boundary, human-in-the-loop checkpoints before actions with real-world consequences, and logging of every tool invocation. The OWASP 2026 data security guide calls for runtime monitoring of AI systems as a distinct control, separate from the application security controls applied at build time.

Governance Baseline: What Your Security Team Should Have in Place

The OWASP GenAI Data Security Risks & Mitigations 2026 guide and NIST AI 600-1 provide the current framework consensus. Only 24% of enterprises have a dedicated AI security governance team, according to the Practical DevSecOps 2026 report. NIST’s framework treats generative AI risk management as a continuous process rather than a pre-deployment checklist.

The minimum governance baseline for organizations running LLM-based systems:

• Maintain a model inventory covering what models are deployed, which version, and what data they are authorized to process.
• Classify data before it enters any AI pipeline; apply the same handling rules you apply to other sensitive processing contexts.
• Require security review of any application that gives an LLM access to internal tools or data systems, using the OWASP LLM Top 10 as the test criteria.
• Establish an incident response playbook for AI-specific failure modes: model-generated disinformation, data exfiltration through a compromised AI endpoint, and supply-chain compromise of a third-party model or plugin.
• Red-team models before and after deployment, and test agentic workflows under adversarial conditions before production rollout.
• Maintain human-in-the-loop oversight for high-stakes decision contexts, and continuous post-deployment monitoring, because model behavior in production frequently diverges from behavior observed during pre-release evaluation.

Teams operationalizing defensive controls across guardrails, content filters, and runtime monitoring can reference guardml.io’s coverage of AI safety tooling and deployment patterns.

OWASP’s 2025 list makes a point worth underscoring: securing generative AI is not solely about protecting the model or analyzing prompts. It requires treating the full deployment stack (retrieval systems, agent frameworks, plugins, fine-tuning pipelines, and the organizational processes wrapped around all of it) as the actual attack surface.