AI Phishing Detection: How It Works and How to Evaluate It

Products sold under the AI phishing detection (opens in new tab) label reach their verdicts on different evidence, and buyers rarely get to inspect how any one of them decided. Meanwhile, AI-generated phishing emails (opens in new tab) shed the surface tells (opens in new tab) older filters were built to catch.

The stakes sit on both sides of the verdict. AI-automated lures reach a 54% click-through rate (opens in new tab), against 12% for standard attempts, and the human element (opens in new tab) still figures in about 60% of breaches (opens in new tab). A message that clears the filter lands in front of a person, often one authorized to move money (opens in new tab). An opaque verdict cuts the other way, burying a real report the team needed to read.

What separates one method from the next is the evidence each one needs before it can decide. This guide walks through the four methods that run under the label, how attackers degrade each, and what a detection rate and a false-positive rate actually measure.

Key takeaways

• Four methods run under one label. Authentication checks, supervised classifiers, language models, and context checks each reach a verdict on a different kind of evidence, so "AI phishing detection" names an input the tool reads, not a single technique.
• Attackers degrade the methods that learn. Rewording a lure or making cheap edits to a phishing page drops detection accuracy unevenly across classifiers and language models, and human reviewers wave through many of the same adversarial pages.
• A detection rate and a false-positive rate measure less than they appear to. A detection rate only describes the corpus it ran against, and a false-positive rate often blends real errors with permitted activity a business has chosen to allow.

Four methods run under the label, and each needs different evidence to decide

Four methods appear under the label: protocol authentication checks, supervised classifiers, language models, and context checks built on accumulated records. Each one requires different evidence to reach a verdict.

An AI email security (opens in new tab) stack layers several of them and has to apply them at production scale, against a phishing volume that rose quarter over quarter in early 2026 (opens in new tab).

1. Authentication checks use message or SMTP metadata and DNS records

SPF, DKIM, and DMARC authenticate domain identifiers (opens in new tab) using message or SMTP metadata plus DNS-published records. Message-content and lookalike-domain analysis require separate controls under DMARC's own specification (opens in new tab).

DMARC excludes these checks from its scope under Section 2.2 of RFC 9989: it "does not address the use of visually similar domain names (opens in new tab) or abuse of the RFC5322.From human-readable display name (opens in new tab)."

Authentication validates the sender and domain identifiers. Message trustworthiness requires separate analysis.

2. Classifiers need a labeled set of examples to compare against

Supervised machine learning (opens in new tab) over extracted features requires a labeled corpus of confirmed phishing and confirmed legitimate samples. These techniques fall into two families. URL-based approaches (opens in new tab) classify on properties such as a URL's structure.

Webpage content-based approaches, whose visual sub-family compares a rendered page against a known brand page, work from the page itself, and hybrids combine features from both. Whichever family a classifier belongs to, its performance depends partly on the diversity and representativeness of its labeled training data (opens in new tab).

Adding varied labeled examples can extend coverage, and classifiers also generalize to some samples they did not see during training.

3. Language models need the text and a stated definition of suspicious

In benchmark testing (opens in new tab), a large language model used the message plus a written description of what counted as suspicious. It performed strongly on a research corpus and returned a written rationale for each verdict.

The message and instructions it receives bound its verdict, and changing those instructions can change what it flags as suspicious.

4. Context checks need a record the message does not carry

Context checks compare the message with records outside it, such as domain age and history (opens in new tab). They can also compare a message with a sender's past sending behavior, which requires enough prior activity to establish a useful baseline.

That leaves a context check with little to work with when a sender or domain has no sending history. A sender writing to this recipient for the first time through a legitimate platform is one example, and the message reads like any other first contact.

Attackers change the evidence a method reads, and the losses are uneven

Tests of research detectors under rewritten lures (opens in new tab) and edited phishing pages show uneven effects on the two methods that learn: classifiers and language models.

Rewriting the lure with a language model costs both learned methods accuracy

A single few-shot rewording pass moves both learned methods at once. Researchers used a language model (opens in new tab) to rewrite the same phishing emails (opens in new tab), and the rewrite reduced accuracy across every model they evaluated, with larger losses among the classical classifiers than the language model.

Cheap page-level changes break some models and leave others intact

Low-cost edits to a phishing page (opens in new tab) split the field. Attackers generated adversarial phishing pages (opens in new tab) that sharply reduced the catch rates of a stacked machine-learning detector and a deep-learning brand-matching detector.

A multimodal language model showed the smallest decline among the detectors evaluated, though its detection still dropped. The unevenness runs in both directions, since cheap page perturbations (opens in new tab) leave some detectors statistically unchanged.

Adversarial logo swaps and similar page-level edits did all of that damage without altering what the page asks a visitor to enter.

Human review recovers part of what the model misses

Students and IT professionals reviewed the same rendered adversarial pages (opens in new tab). Many looked legitimate to the reviewers, so escalation recovered only part of the detector's misses.

What a detection rate and a false-positive rate actually measure

The losses differ by method, so the two figures buyers lean on for comparison are produced under different conditions. Detection and false-positive rates depend on different samples and different counting rules.

A detection rate is a measurement of the corpus it was run against

A detection rate describes its scoring sample set, and that set can reflect conditions production rarely reproduces. Data splits that ignore time can make security benchmarks overstate deployment performance.

In malware classification, cross-validation can place examples from a family in the training data even though a real deployment would treat that family as unknown at training time (opens in new tab). Laboratory-only evaluation (opens in new tab) also remains common in machine-learning security research, which widens the gap between reported and operational performance.

A false-positive rate counts two different things at once

Operational false-positive figures can combine detector errors with "benign triggers" (opens in new tab): true alarms that legitimate activity produces and the business has decided to allow.

In a study of security operations teams (opens in new tab), most of one practitioner's alarm volume fell into that second category. Benign triggers measure permitted business activity, not detector accuracy.

For example, a security team can log an internal tool sending from a previously unseen subdomain as a benign trigger rather than a detector error. A buyer reading a false-positive figure has to know which of the two populations it counted.

How Doppel produces a phishing verdict

Doppel is the AI-native Social Engineering Defense (SED) (opens in new tab) platform that unifies Digital Risk Protection (opens in new tab), Human Risk Management (opens in new tab), and Email Security. Doppel Email Security settles its phishing verdicts on evidence the message does not carry: domain registrations (opens in new tab) and TLS certificates that describe a sender before it has built up any sending history.

Of the four methods above, Doppel's approach draws on three. It reads the message, checks it against records outside the inbox, and applies readable detection policies (opens in new tab) a security team can audit in plain language. Authentication stays a floor for sender identity. Trust in the message itself comes from separate evidence.

A sender, domain, or channel the inbox has no record for is exactly the condition that starves a context check. The record missing inside the mailbox often already exists outside it, dating from the moment someone registered the domain or issued the certificate. The Doppel Threat Graph (opens in new tab) assembles and connects those records, so a first-time sender already resolves to infrastructure on file.

Phishing Triage (opens in new tab) handles the mail employees flag. Its AI agents classify each reported message (opens in new tab) and pull confirmed-malicious mail from the inboxes it reached. When they escalate, they carry the reasoning behind the verdict, so the analyst opens a case instead of a raw report.

And the platform dismantles the sending infrastructure (opens in new tab) behind a confirmed phish at the source.

External evidence changes Doppel's answer

Each method under the AI phishing detection label decides on different evidence, and the research shows how little of it an attacker has to touch: rewritten lures and edited pages dropped the performance of several learned detectors, and human reviewers put many of the same pages in the legitimate column.

Doppel's approach brings external evidence (opens in new tab) into the decision, the record a first-contact message cannot carry on its own. Request a demo (opens in new tab) to see a verdict and the evidence behind it. The stronger test for any phishing defense is real mail flow plus the infrastructure evidence outside it.