AI-enabled phishing and deepfake fraud: what it changes for your controls

This guide is for CISOs, security teams, finance directors and board members who want to understand what AI-enabled phishing and deepfake fraud mean for their existing controls. It explains what has changed, what has not, and which controls to strengthen first.

The central message is reassuring. AI makes deception more convincing, but the controls that work best against impersonation fraud never depended on spotting a fake. They depend on verification through trusted channels, strong authentication and processes that hold under pressure.

#How is AI changing phishing and social engineering?

AI is making phishing and social engineering more convincing, more tailored and easier to produce. The NCSC judges that AI provides capability uplift in reconnaissance and social engineering, making both more effective, efficient and harder to detect. Generative AI can write fluent, personalised messages without the spelling and grammar mistakes that often reveal phishing.

The NCSC's assessment of the near-term impact of AI on the cyber threat, published in January 2024, found that generative AI can already enable convincing interaction with victims and can produce lure documents without the translation, spelling and grammatical mistakes that often give phishing away. It warned that this would make it harder for people at any level of skill to identify phishing, spoofing or social engineering attempts.

Its follow-up assessment of the impact of AI on cyber threat from now to 2027, published in May 2025, judges that AI will almost certainly continue to make elements of cyber intrusion more effective and efficient, leading to an increase in the frequency and intensity of threats. It notes that threat actors are already using AI to enhance existing techniques, including reconnaissance and gaining access through social engineering.

#What are deepfakes, and how are they used in fraud?

Deepfakes are AI-generated or AI-altered audio, video or images that imitate a real person. In fraud, they can be used to impersonate a senior leader, colleague or supplier on a phone or video call, adding apparent authority to an urgent payment or data request. They mean a familiar voice or face is no longer reliable proof of identity.

Deepfake fraud is best understood as a more convincing version of familiar social engineering. The goals are the same as in business email compromise: an urgent payment, a change of bank details, a password or MFA reset, or access to sensitive information. What changes is that a voice message, phone call or video call can no longer be treated as proof on its own.

Scenarios worth planning for include:

• A call or voice message, apparently from the chief executive, asking for an urgent and confidential transfer.
• A video meeting in which a familiar colleague appears to approve an unusual payment.
• A caller posing as an employee who asks the IT service desk to reset their password or MFA.
• A phone call from someone posing as a supplier contact, confirming new bank details sent by email.
• A convincing recruitment or supplier approach used to gather information for a later attack.

#Which controls still work against AI-enabled fraud?

Controls that verify the request, rather than judging how genuine someone looks or sounds, still work. Call-back verification using known contact details, dual approval for payments and bank detail changes, phishing-resistant MFA and least-privilege access all hold up however convincing a message, voice or video appears. AI raises the cost of relying on human judgement alone.

The NCSC's phishing guidance already recommends a layered approach that assumes some attacks will get through. It advises that all important email requests are verified using a second type of communication, such as a phone call, and that staff understand normal ways of working, such as how payments are made. The same principles apply to calls, voice messages and video meetings.

#Verification rules to adopt or reconfirm

1. Never approve a payment, bank detail change or credential reset based only on an email, call, voice message or video.
2. Verify using contact details you already hold, never those supplied in the request.
3. Require dual approval for payments and for changes to supplier or payroll bank details.
4. Agree that urgency and seniority never override verification, and have senior leaders say so publicly.
5. Consider pre-agreed verification questions for the highest-risk requests between senior leaders and finance.
6. Give staff explicit permission to pause and check, without fear of blame.

#How should identity controls change?

Identity controls should move towards methods that a convincing message or caller cannot trick. The NCSC describes FIDO2 MFA as offering guessing, phishing and theft resistance, while message-based codes such as SMS are only likely to be appropriate when nothing stronger is possible. Service desk resets also need stronger checks, because callers can be impersonated.

The NCSC's guidance on recommended types of MFA notes that challenge-based authenticator apps offer only partial phishing resistance because they are vulnerable to prompt fatigue attacks, and that app-based and hardware code generators are vulnerable to one-time passcode interception. Prioritise phishing-resistant MFA for administrators, finance users, executives and remote access.

#Service desk checks

• Do not reset passwords or MFA based on a phone call alone.
• Verify identity through a separate trusted route, such as a call back to the number on record or confirmation from the person's line manager.
• Log every reset, and notify the user through a channel you already hold.
• Apply extra checks for privileged and executive accounts.

#Does security awareness training still help?

Yes, but its purpose shifts from spotting fakes to following the process. As AI removes obvious warning signs, training should focus on recognising high-risk requests, applying verification rules every time and reporting concerns quickly. The NCSC advises building a reporting culture without blame, so that people report even when they think they may have been tricked.

• Teach the triggers: payment requests, bank detail changes, credential resets and requests for secrecy.
• Use realistic scenarios that include calls and video, not only email.
• Make reporting easy and visible, and thank people who report.
• Include executives and their assistants, whose names and authority make them attractive to impersonate.
• Measure reporting, not just the absence of clicks.

#What should boards and finance directors ask?

Boards and finance directors should ask whether any high-value payment, bank detail change or account reset could be completed on the strength of a single message, call or video. They should also ask how strong authentication is for senior and finance users, whether staff feel able to challenge senior requests and when the process was last tested.

• Which processes would let one convincing call or message move money or grant access?
• Do those processes require verification through a separate trusted channel and a second approver?
• Are directors, executives and finance users protected by phishing-resistant MFA?
• Would a junior member of staff feel able to delay a payment requested by the chief executive?
• When did we last test these controls with a realistic exercise, and what did we learn?
• Does our incident plan cover impersonation fraud, including who contacts the bank and Report Fraud?

Leaders strengthen these controls by following them visibly. When directors accept call-backs and second approvals for their own requests, staff find it easier to apply the same rules to everyone, whoever appears to be asking.

#What to do next

Next, review your highest-risk processes, such as payments, bank detail changes and credential resets, and confirm that none can be completed on the strength of a message, call or video alone. Strengthen MFA for key users, brief executives and finance teams on deepfake scenarios and test the process with a realistic exercise.

1. List the processes where impersonation could cause the most harm.
2. Confirm each requires out-of-band verification and, for payments, dual approval.
3. Move administrators, finance users and executives to phishing-resistant MFA.
4. Tighten service desk identity checks for password and MFA resets.
5. Run a tabletop exercise using a deepfake call or video scenario.
6. Make sure staff know that if money is lost, they should contact the bank immediately and report to Report Fraud on 0300 123 2040, or Police Scotland on 101.

#Frequently asked questions