This is what a Deepfake Voice Clone Used in a Fraud Attempt Sounds Like
One of the stranger applications of deepfakes — AI technology used to manipulate audiovisual content — is the audio deepfake scam. Hackers use machine learning to clone someone’s voice and then combine that voice clone with social engineering techniques to convince people to move money where it shouldn’t be. Such scams have been successful in the past, but how good are the voice clones being used in these attacks? We’ve never actually heard the audio from a deepfake scam — until now.
Security consulting firm NISOS has released a report analyzing one such attempted fraud, and shared the audio with Motherboard. The clip below is part of a voicemail sent to an employee at an unnamed tech firm, in which a voice that sounds like the company’s CEO asks the employee for “immediate assistance to finalize an urgent business deal.”
The quality is certainly not great. Even under the cover of a bad phone signal, the voice is a little robotic. But it’s passable. And if you were a junior employee, worried after receiving a supposedly urgent message from your boss, you might not be thinking too hard about audio quality. “It definitely sounds human. They checked that box as far as: does it sound more robotic or more human? I would say more human,” Rob Volkert, a researcher at NISOS, told Motherboard. “But it doesn’t sound like the CEO enough.”
THE TARGET “IMMEDIATELY THOUGHT IT SUSPICIOUS”
The attack was ultimately unsuccessful, as the employee who received the voicemail “immediately thought it suspicious” and flagged it to the firm’s legal department. But such attacks will be more common as deepfake tools become increasingly accessible.
All you need to create a voice clone is access to lots of recordings of your target. The more data you have and the better quality the audio, the better the resulting voice clone will be. And for many executives at large firms, such recordings can be easily collected from earnings calls, interviews, and speeches. With enough time and data, the highest-quality audio deepfakes are much more convincing than the example above.
The best known and first reported example of an audio deepfake scam took place in 2019, where the chief executive of a UK energy firm was tricked into sending €220,000 ($240,000) to a Hungarian supplier after receiving a phone call supposedly from the CEO of his company’s parent firm in Germany. The executive was told that the transfer was urgent and the funds had to be sent within the hour. He did so. The attackers were never caught.
Earlier this year, the FTC warned about the rise of such scams, but experts say there’s one easy way to beat them. As Patrick Traynor of the Herbert Wertheim College of Engineering told The Verge in January, all you need to do is hang up the phone and call the person back. In many scams, including the one reported by NISOS, the attackers are using a burner VOIP account to contact their targets.
Spam calls are a plague, and carriers are finally starting to fight back, but there’s been an unexpected casualty: Google Voice’s “get voicemail via message” feature. The company is pulling it because the messages are getting flagged as spam, via Brian Krebs on Twitter.
The feature transcribes voicemails from a Google Voice number and texts them to the recipient. But because so many of these voicemails contained automated transcripts of spammy robocalls, those SMS messages were starting to get flagged as spam. In order to avoid further issues, Google is shutting down the feature, which is expected to be completed by August 9th.
February 13, 2019
Federal Communications Commission Chairman Ajit Pai told major telecommunications providers today that the agency would step in if the companies failed to implement a plan to fight robocalls this year.
In November, Pai sent letters to several companies, asking them to use a caller authentication system to battle call spoofing, a technique robocallers use to imitate other numbers. Pai’s letter said he expected the companies to implement the plan by the end of this year, and also said that the agency would “take action” if carriers didn’t follow the plan…