Human Defenses and Artificial Intelligence.
An employee in the Hong Kong office of Arup, the global engineering firm, had received a request that appeared to come from the company’s CFO in London. It concerned a confidential transaction. The request was unusual enough to raise suspicion, but not absurd enough to dismiss.
Then came the video meeting that reassured the employee. The CFO was there. Several colleagues were there. They spoke about the transaction and appeared to agree that the money should be sent. What seemed like a suspicious request now felt legitimate and a priority for senior leadership.
The employee dutifully made 15 transfers to five bank accounts. The total was roughly $25 million.
Then the moment of panic. After contacting the company’s headquarters to follow up, the employee learned it was a fraud. Every other person in the video meeting had been fabricated. The CFO was an artificial intelligence deepfake. So were the colleagues who appeared to confirm his instructions. [1]
Now imagine it was your personal wealth that just got wired to a foreign account, with your permission. Enhanced by AI, sophisticated fraud is on the rise, so it’s time to adopt new rules of financial trust.
The vulnerability in the Arup case was not a compromised email account or password. The fraud worked because technology supplied all the signals human beings normally use to decide that a request is genuine. There was a recognizable authority figure. There were familiar coworkers. There was group agreement. There was confidentiality, which discouraged outside consultation, and urgency, which made delay feel irresponsible. Most successful fraud does not persuade people to behave irrationally. It persuades them that cooperation is the responsible thing to do.
No doubt there will be more of this, and we will all be targets. We need updated defense protocols to protect against updated criminal techniques. Some defenses are ironically low-tech, perhaps exposing AI’s Achilles’ heel. A clever Ferrari employee defeated a highly sophisticated criminal scheme with a simple question. More on that in a moment.
The evidence we used to trust
For most of our lives, recognition has functioned as evidence. We know the voices of our spouses and children. We recognize an adviser’s manner of speaking. If a request received by email seemed questionable, calling the person or seeing the person on video felt like confirmation.
That assumption is becoming unsafe. AI can reproduce a person’s voice, animate a face, and generate messages that sound like the person’s ordinary writing. It can also carry on a conversation. A criminal may need only a few pressured minutes to obtain a verification code, change payment instructions, or authorize a transfer.
A 2025 study published in Scientific Reports tested how well people could identify AI-generated voice clones. When listeners heard a real person and an AI clone of that person, they judged the recordings to be the same speaker roughly 80% of the time. [2]
The FBI’s 2025 report found more than $20 billion in total reported losses, including $893 million across complaints containing AI-related information. [3] The Federal Trade Commission found that reports from older adults losing at least $10,000 to business and government impersonators more than quadrupled between 2020 and 2024. Reports of losses exceeding $100,000 increased nearly sevenfold. [4] AI makes it cheaper to personalize, easier to scale, and harder to recognize.
Why intelligence is not enough
The old defense against fraud was partly observational. Look for misspellings. Notice the strange address. Listen for an unfamiliar accent. Be skeptical of an implausible story. Those clues still catch unsophisticated scams, but they are becoming less useful against the best ones.
AI can manufacture the requester, the witnesses, and the consensus. A criminal no longer has to forge a single message. The criminal can construct the setting in which the message appears reasonable. That changes the goal. We should stop expecting ourselves to identify every fabrication. We need procedures that remain safe even when the fabrication is excellent.
The new rules of authentication
First rule: pause to consider the potential risk of any unsolicited request. My son asking me by text for the Hulu login when I know there is a game on, or my daughter asking for the New York Times login, as she often does, may not make me especially vigilant. A text or email from my bank or credit card company asking me to act absolutely does. We don’t want to live in a paranoid state, but we do need to consider the risk of any action. When the risk is high, the burden of proof increases.
Second rule: recognition is not authentication. A familiar voice, face, writing style, or personal detail can support a conversation. None should independently authorize access or move money. Families should agree on a private phrase for unusual requests. Advisers and clients should establish a separate verification phrase or documented procedure for money movement and changes to bank or contact information. The phrase should not be sent by email or text, where it could become part of the information available to an attacker.
Third rule: verification must leave the conversation. An inbound call, email, text, or video meeting is unverified until you return through a path you selected independently. End the call. Open the institution’s official app, call a saved number, or contact the person through an established channel. The initiator of the contact never chooses the return path.
Fourth rule: no single communication moves money. An email should not be confirmed by replying to the email. A phone request should not be authenticated by the voice on the phone. A video meeting should not, by itself, authorize an unexpected transfer. New payment instructions, new recipients, verification codes, and changes to account access deserve a second channel or a second authorized person. One communication may begin the process. It should not complete it.
Fifth rule: unexpected urgency increases the need for verification; it does not create an exception. Some financial deadlines are real. An anticipated capital call, a negotiated closing, or a known tax payment comes with context established before the request arrives. A deadline first introduced by the person asking for money is different. In my two decades of moving money for clients, only a handful of cases have required unusual and immediate action. ‘Act now or lose the opportunity’ is a classic hallmark of bad investments and scams. One fraudulent transfer can create months of hassle and a permanent loss.
Sixth rule: protect the master keys. The highest vigilance is reserved for a handful of accounts that act as master keys. Your newspaper subscription does not deserve the same attention as the accounts that can reset your financial life. Protect the primary email account, mobile phone, wireless-carrier account, Apple, Google, or Microsoft account, password manager, and primary financial accounts. Use a passkey when one is offered. This is different from choosing a longer password. A passkey is cryptographically tied to the legitimate website or app. A convincing imitation site cannot simply collect it, and a persuasive caller cannot ask you to read it aloud. CISA identifies FIDO authentication, the standard behind passkeys, as the only widely available phishing-resistant form of authentication. [5]
Every critical account should also have a recovery path. For an elderly or vulnerable person, add a trusted contact, establish legitimate shared or limited access where the institution permits it, and put a power of attorney in place before it is needed. Strong security that locks out the people legally responsible for helping is incomplete security.
The question the deepfake caller could not answer
In July 2024, a senior Ferrari executive began receiving WhatsApp messages that appeared to come from Ferrari chief executive Benedetto Vigna. They arrived from an unfamiliar number, but the explanation fit the story: the matter was confidential.
The supposed CEO described a possible acquisition with complications involving China. A currency-hedging transaction was supposedly required. Then the executive received a live call. The voice reproduced Vigna’s southern Italian accent convincingly enough to sustain the conversation.
The Ferrari executive did not continue trying to decide whether the voice sounded real. He changed the burden of proof. “Sorry, Benedetto, but I need to identify you,” he said.
He asked for the title of a book Vigna had recommended to him a few days earlier. The caller could not answer. The call ended. [6]
The executive improvised a shared secret. A household or advisory firm can do better by agreeing on one in advance. A private phrase is safer than a remembered conversation because emails, calendars and other personal details may also have leaked.
Ferrari’s executive did not prove that the voice was fake. He required the caller to prove that he was real. That is the habit worth adopting and passing along.
The goal is not to become suspicious of everyone or spend our lives worrying about the next technical threat. We should acknowledge the obvious changes in the environment and adopt a few new rules in response.
Sources and notes
1. Financial Times. Arup lost $25mn in Hong Kong deepfake video conference scam
2. Scientific Reports. People are poorly equipped to detect AI-powered voice clones
3. Federal Bureau of Investigation. 2025 IC3 Annual Report
4. Federal Trade Commission. False alarm, real scam: how scammers are stealing older adults’ life savings
5. Cybersecurity and Infrastructure Security Agency. More than a Password
6. Fortune reporting on the Bloomberg account. Ferrari executive foils deepfake plot by asking a question only the CEO could answer
