Generative AI in Cybersecurity: Use Cases and Risks

Generative AI now operates on both sides of every security program. The same model families that triage alerts and draft detection logic also write spear-phishing lures and stand up lookalike sites in hours. AI-automated phishing emails achieve click-through rates of 54%, compared to 12% for standard phishing attempts, a gap that reflects how quickly attacker tooling matures.

Your security team carries a double mandate: capture the technology's defensive gains while defending against attacks built with it, because the distance between those two adoption curves shows up as fraudulent wires and harvested credentials.

This article covers what generative AI means in cybersecurity, the use cases security teams deploy it for, the risks that attackers and ungoverned adoption each create, and what it takes to come out ahead on both sides.

Key Takeaways

• Generative AI is structurally dual-use. The same models that draft detection logic and triage alerts also generate spear-phishing lures, voice clones, and deepfake video, and AI-automated phishing converts at 54% compared to 12% for standard attempts.
• Attackers run generative AI through every stage of the social engineering attack chain: reconnaissance, weaponization, delivery, persuasion, and execution. This compresses campaigns that once took weeks into hours of machine output across email, SMS, voice, social, and messaging apps.
• Defenders capture real value from generative AI in detection of novel lures, alert triage, simulation content, and threat intelligence synthesis, but lose ground when adoption is ungoverned, bolted onto single-channel tools, or slowed by compliance constraints attackers ignore.
• The defensive program that holds the line pairs machine-speed detection across every channel with tiered human oversight on high-stakes decisions and a closed loop that converts live attacker campaigns into role-based employee training within hours.

What Generative AI Means in Cybersecurity

Generative AI in cybersecurity is the application of models that produce novel text, audio, video, and code to security work. Dual use is its defining property. The capabilities defenders adopt are the same capabilities attackers weaponize.

Generative Models Produce New Content From Learned Patterns

Large language models, image generators, and audio synthesizers share a core mechanic: they learn statistical patterns from massive training data and produce new outputs that match those patterns.

In a security context, this means an LLM can draft a phishing email that resembles legitimate corporate correspondence, and an audio model can clone a voice from a short sample.

Every Generative Capability Cuts Both Ways for Security

A model that summarizes threat intelligence feeds for an analyst can summarize an executive's public statements and build a convincing pretext. A voice synthesizer that powers vishing simulations for employee training can impersonate a CFO on a live call.

The technology reflects the intent of the operator and the controls around it.

Agentic AI Turns Generated Output Into Autonomous Action

Agentic systems chain reconnaissance, content generation, delivery, and adaptation into autonomous workflows that pursue a defined goal. A generative model produces one output per prompt, while an agentic system plans, executes, and adapts until it reaches that goal.

Attackers and defenders are both building these workflows, and the side that deploys them faster sets the operational tempo for the other.

How Security Teams Use Generative AI

Security teams apply generative AI where pattern-matching defenses stall. The highest-value use cases concentrate in four areas: reading the intent behind novel lures, compressing alert triage and investigation, converting live attacks into simulation material, and turning threat intelligence into decisions.

Detection Models Read the Intent Behind Novel Lures

Signature-based detection fails against AI-generated lures because attackers construct those lures to be novel by default. LLM-based detection reads semantic intent and recognizes that a message manufactures urgency around a financial transaction even when every surface-level indicator appears legitimate.

Generative AI Compresses Alert Triage and Incident Investigation

Generative AI compresses triage by correlating threat intelligence with related activity that might not trigger a traditional alert on its own. The fastest observed incidents in 2025 achieved lateral movement and data exfiltration within minutes, and a SOC that measures manual triage in hours cannot contain threats inside that window.

LLM-assisted triage cut ticket completion time on live SOC tickets, with the most pronounced reductions on tickets requiring cross-tool correlation.

Live Attack Data Becomes Realistic Phishing Simulation Material

Static, template-based phishing simulations teach employees to spot last quarter's attacks. Generative AI closes that gap by converting live threat intelligence into simulation content. The lure copy and landing page visuals from a real campaign become a defanged employee exercise the same week.

AI-powered attackers generate large volumes of customized phishing variations quickly, and simulation programs that cannot produce variations at a comparable rate train against an incomplete threat model.

Threat Intelligence Synthesis Turns Raw Feeds Into Decisions

LLMs convert raw threat intelligence feeds into prioritized, contextualized briefs. Enterprise security teams subscribe to more concurrent feeds than any analyst team can process by hand, and indicators lose their value while they sit in a queue.

The models extract indicators of compromise from unstructured data and map each one to your organization's specific environment and exposure.

How Attackers Run Generative AI Through the Attack Chain

Attackers apply the same generative capabilities at every stage of the social engineering attack chain: reconnaissance, weaponization, delivery, persuasion, and execution. Campaigns that once took weeks of manual effort now take hours of machine output, and the five-stage chain maps where each capability lands.

Each stage feeds the next, with reconnaissance data shaping the lure and delivery channel before the persuasion play drives execution.

1. Reconnaissance: LLMs Profile Targets and Mine Open Sources at Scale

Attackers use LLMs to profile organizations and find high-value targets at a scale manual research cannot match. Conference recordings and LinkedIn profiles become raw material: a short audio clip becomes a voice clone, and an org chart becomes a pretext.

2. Weaponization: Generative Models Build Deepfakes, Lures, and Lookalike Infrastructure in Hours

That reconnaissance feeds the infrastructure that attackers build next. Underground channels advertise tools such as FraudGPT and WormGPT, and paid services commercialize jailbreak techniques. Attackers use them to produce phishing emails, clone login pages, and generate deepfake media on demand.

3. Delivery: AI Coordinates One Campaign Across Email, SMS, Voice, and Social Channels

A single campaign now runs across email, SMS, voice, social media, and messaging apps simultaneously, with lures tuned per channel and per target. Attackers already operate inside enterprise collaboration workflows. In one documented attempt, they created a fake WhatsApp account and set up a Microsoft Teams meeting using voice cloning and edited YouTube footage of a senior executive.

4. Persuasion: Cloned Voices and Flawless Copy Borrow Authority and Manufacture Urgency

Voice clones generated from short audio samples sound real enough that employees act on direct instructions that appear to come from their own leadership. The technique reaches the highest levels of authority.

Attackers sent text messages and AI-generated voice messages impersonating senior U.S. government officials, as a May 2025 advisory documented.

5. Execution: Fraudulent Transfers and Stolen Credentials Clear Before Verification Catches Up

Once persuasion lands, the payout moves faster than the controls meant to stop it. Deepfake meeting fraud has already put employees on video calls where every other participant appeared to be a known colleague, and one multinational engineering firm approved transfers worth millions before the fraud surfaced.

Attackers who execute within the verification window, or who use stolen session tokens that bypass authentication entirely, face no automated check.

Where Generative AI Adoption Goes Wrong for Defenders

Generative AI carries risk on the defender's side of the ledger as well. Three failure modes erode the advantage the technology promises: ungoverned use, AI features bolted onto single-channel tools, and adoption constraints that attackers ignore.

Ungoverned Use Produces Confident Errors and Exposes Sensitive Data

Ungoverned adoption leaks sensitive data before any attacker gets involved. 63% of breached organizations either lacked an AI governance policy or were still developing one in 2025. Shadow AI persists even when sanctioned alternatives exist, and employees paste proprietary source code and customer data into unauthorized tools that sit outside approved data-handling controls.

Hallucinations compound the problem: AI outputs carry confident errors that survive into response decisions when no one verifies the findings.

AI Features Bolted Onto Single-Channel Tools Inherit the Old Blind Spots

Adding AI capabilities to a domain-only monitor produces a domain-only monitor with AI. Legacy security awareness training tools that add AI-generated phishing simulations while remaining email-centric leave the workforce unprepared for voice cloning and deepfake video impersonation.

The AI accelerates a single-channel capability while attackers operate across voice, SMS, social media, and messaging apps simultaneously.

Defenders Adopt Under Constraints Attackers Ignore

Every defensive deployment carries prerequisites that attacker tooling skips. Organizations subject to GDPR must account for regulatory requirements when they design and operate AI-driven decision systems, and compliance and data-quality obligations apply before any defensive model ships. Attackers face none of this. They deploy immediately, iterate without prerequisites, and use uncensored models with no governance cycle.

The result is a structural asymmetry that favors offense.

What It Takes to Capture Generative AI's Defensive Upside

Your security program earns generative AI's upside by meeting three requirements at once: detection and response that run at machine speed across every channel, human judgment on the decisions that carry risk, and a training loop fed by the attacks AI generates. The program weakens when any one of the three is missing.

Run Detection and Response at Machine Speed Across Every Channel

The opening stages of an attack form outside the infrastructure you monitor. Lookalike domains, fake profiles, and scam ads go live before any message lands in an inbox, so detection must extend beyond email to voice, SMS, social media, messaging apps, paid ads, and domains.

Speed matters as much as coverage, because attacks that achieve lateral movement in minutes demand detection and response on the same timescale.

Keep Human Judgment on the Decisions That Carry Risk

Autonomous triage and containment work for high-volume, well-characterized, reversible actions. Novel threats, high-stakes environments, and irreversible consequences require human analysts who understand business context.

Build tiered autonomy into the program, and match the level of human involvement to the risk and reversibility of each decision.

Feed AI-Generated Attacks Back Into Employee Training

Simulations change behavior when they mirror the campaigns attackers are running right now. Wiring external detection into the training pipeline turns each detected campaign into a role-based exercise, with invoice fraud scenarios going to finance teams and deepfake impersonation scenarios going to executives.

The moment someone makes a mistake is when they are most receptive to learning, and just-in-time intervention delivers the lesson within hours.

How Doppel Puts Generative AI to Work for Social Engineering Defense

Doppel is the AI-native Social Engineering Defense (SED) platform that unifies Digital Risk Protection and Human Risk Management against exactly this dual-use problem. The Doppel Threat Graph continuously ingests signals across domains, social media, paid ads, telco, dark web, and messaging platforms, then correlates them into campaign-level views of attacker infrastructure.

Agentic AI executes takedowns at scale, so analysts focus on the complex escalations that require human judgment.

The closed loop between external detection and internal training gives security teams a structural advantage. When Doppel's DRP pipeline identifies a real campaign, Dynamic Simulation converts that campaign's content into an employee simulation across email, voice, SMS, and other conversational channels.

Coinbase used Doppel to dismantle large volumes of fraudulent social media accounts and domains.

The defensive workflow mirrors the attacker tooling described earlier, stage for stage:

• Recon AI Agents ingest the same public signals attackers mine, including job postings, partnership announcements, earnings calls, and SEC filings, and turn them into organization-specific simulation templates available on day one.
• Vibe Phishing collapses simulation production into a natural-language prompt that generates the message, a brand-cloned landing page, and role-specific coaching content, matching the variation rate AI-powered attackers already achieve.
• One-click threat-to-simulation conversion means a campaign Doppel detects against your brand today can run as a defanged exercise tomorrow, across voice and Microsoft Teams meetings as well as email and SMS.

Reconnaissance machinery, content generation, and campaign conversion all now run on the defender's side.

Make Generative AI a Defensive Advantage

Generative AI keeps compounding on both sides of the fight. The security leaders who pair AI-native defense with governed adoption set the pace that attackers have to answer. The goal is to make every campaign more expensive to run than the return it generates, until targeting your brand stops paying.

Request a demo to see how Doppel puts generative AI to work across detection and dismantlement, with employee training connected to both.