Researchers have developed a machine unlearning technique that enables AI to forget specific voices, dramatically reducing the risk of audio deepfakes while maintaining overall speech model performance. (Source: Image by RR)

Machine Unlearning Could Make Audio Deepfakes Much Harder to Pull Off

Researchers at Sungkyunkwan University in Korea have developed a novel application of “machine unlearning” aimed at preventing audio deepfakes by teaching AI to forget specific voices. Unlike existing safety techniques such as guardrails, which attempt to restrict access to certain data, machine unlearning deletes that data altogether from the model’s memory. This approach represents one of the first attempts to apply unlearning techniques to speech generation—particularly important in an age where voice replication can be achieved with just a few seconds of recorded audio and is increasingly used in scams and identity fraud.

The research team recreated Meta’s VoiceBox speech model and trained it to “unlearn” voices it was once able to mimic. The system replaced redacted voices with newly generated, randomized alternatives. According to a story in technologyreview.com, results showed a more than 75% drop in the model’s ability to convincingly imitate forgotten voices—while maintaining most of its performance on permitted ones, with only a slight 2.8% degradation. This balance between preserving utility and enforcing voice forgetfulness is key to enabling real-world safety use cases without crippling the AI’s general capabilities.

One of the challenges with voice-based models is their reliance on “zero-shot” learning, which enables them to mimic voices they weren’t even trained on. To counter this, the unlearning model had to not only forget previously learned voices but also develop an instinct to avoid mimicking similar patterns from unfamiliar ones. This complexity, combined with the need for around five minutes of audio per voice to be redacted and a multiday training process, makes the method promising but still limited in scalability and speed.

While this technology is still in its early stages, researchers and industry experts see high potential for broader deployment. Google DeepMind is exploring unlearning methods, and Meta has so far withheld VoiceBox from public release due to fears of abuse. Although trade-offs remain—such as slower processing and slight performance dips—the voice unlearning breakthrough marks a significant stride toward safer, consent-based use of speech AI technologies.

read more at technologyreview.com