AI Copilot Turns One Hacked Email Into $247K Wire Fraud

A new proof-of-concept from Barracuda’s Red Team shows just how little skill it now takes to turn a single compromised inbox into a six-figure business email compromise (BEC) payout, and the accelerant is the very AI assistant meant to make employees more productive.

Barracuda’s researchers didn’t need custom malware or advanced exploitation chains. Once inside a victim’s Microsoft 365 account, attackers turned to Copilot itself to do the heavy lifting.

Their first move was persistence: a single natural-language prompt instructed Copilot to create an inbox rule silently routing sign-in alerts to Deleted Items, neutralizing the one signal most likely to tip off the victim.

AI Copilot Turns One Hacked Email Into $247K Wire Fraud

From there, reconnaissance that would normally take hours of manually scrolling through email threads took seconds. Prompts asking Copilot to summarize organizational structure and flag active conversations let the attackers identify a high-value target, the CEO, and understand exactly how the victim communicated with them.

The most unsettling step: attackers asked Copilot to draft a phishing email that mimicked the victim’s actual writing style, using real context from an existing thread with the CEO.

Because the message came from a genuine, trusted internal account and sounded authentically like its sender, it sailed past the instincts that normally catch phishing attempts.

The CEO clicked an “invoice” link that routed through an adversary-in-the-middle proxy, handing attackers a live session token and full MFA-bypassing access to the CEO’s own mailbox.

With CEO-level access secured, the attackers pointed Copilot at the CEO’s own data, prompting it for “a refresher on recent financial emails, including invoices, monetary values, and upcoming transfers.”

Copilot returned a clean summary of active wire transfers within seconds, including a pending $247,500 county contract payment awaiting final approval.

No bulk downloads, no unusual search patterns, nothing for traditional security tools to flag, because the query executed with the CEO’s full legitimate authority.

Copilot then drafted a bank change request to the finance team in the CEO’s authentic tone, referencing the actual transaction by name.

Finance had no reason to doubt it, updated the account details, and sent $247,500 straight to the attackers. A final inbox rule silently rerouted the finance team’s confirmation replies to an attacker-controlled address, keeping the real CEO in the dark while Copilot itself helped scrub evidence of the fraud from the mailbox.

Barracuda is careful to note that none of this required a new vulnerability inbox rule; abuse and CEO fraud are well-worn BEC tactics. What’s changed is velocity.

Tasks that once demanded technical skill and hours of manual reconnaissance now take a threat actor with no coding ability and a handful of plain-English prompts.

The AI assistant effectively becomes a knowledgeable insider working for whoever is logged in, legitimate user or not. That reframes account compromise as much an identity problem as an email-filtering one.

Barracuda points to a layered response: Email Gateway Defense to block the initial malicious link before it lands, and Managed XDR to catch the post-compromise behaviors anomalous inbox rules, forwarding changes, and unusual account activity that signal an AI assistant is being turned against its own organization.

As Copilot-style assistants become default features in Outlook, Gmail, and other enterprise tools, monitoring what an AI assistant does inside a compromised account is becoming as critical as stopping the initial phishing email.

The lesson from this proof of concept isn’t that AI created a new attack; it’s that AI made an old one dramatically faster, cheaper, and harder to catch.