Startup Deeptrace is racing to develop automated detection of fake videos and images as U.S. 2020 elections loom
A perfect storm arising from the world of pornography may threaten the U.S. elections in 2020 with disruptive political scandals having nothing to do with actual affairs. Instead, face-swapping “deepfake” technology that first became popular on porn websites could eventually generate convincing fake videos of politicians saying or doing things that never happened in real life—a scenario that could sow widespread chaos if such videos are not flagged and debunked in time.
The thankless task of debunking fake images and videos online has generally fallen upon news reporters, fact-checking websites and some sharp-eyed good Samaritans. But the more recent rise of AI-driven deepfakes that can turn Hollywood celebrities and politicians into digital puppets may require additional fact-checking help from AI-driven detection technologies. An Amsterdam-based startup called Deeptrace aims to become one of the go-to shops for such deepfake detection technologies.
“We see deepfakes and similar technologies as a new wave of cybersecurity threats, with the potential of affecting every digital audiovisual communication channel,” says Giorgio Patrini, CEO and chief scientist at Deeptrace, a startup based out of Amsterdam in the Netherlands. “We are building the antivirus for deepfakes.”
Before he helped found Deeptrace, Patrini was working as a member of a deep learning research group headed by Max Welling at the University of Amsterdam. He and his colleagues dipped their toes into the deepfakes discussion by publishing results from a “simple experimental fake detector” on Patrini’s personal blog—an initial venture that spurred much interest from the broader research community.
By late 2018, Patrini had decided to team up with a longtime hometown friend, Francesco Cavalli, to create a startup focused on developing deepfake detection software that can work unobtrusively in the background, like antivirus software, to scan audiovisual media that people may encounter while browsing social media networks or search engine results.
“In this case, what we are defending is not software that can be infected and repurposed by malware, but human opinions and actions, manipulated by fake videos, impersonation and sophisticated cyberfrauds,” Patrini says.
From Fake Porn to Fake News
The term deepfakes originated with a Reddit online forum that used deep learning algorithms to digitally superimpose the faces of celebrities onto the faces of people in porn videos. Such deepfake technology is based on generative adversarial networks (GANs) trained to replicate certain patterns—such as the face of a celebrity—and gradually strengthen the realism of the synthetically generated face.
When it was first publicized by a Motherboard article in December 2017, the existence of deepfake porn spurred Reddit to shut down the r/deepfakes forum. Other online services such as Discord, Gfycat, Pornhub and Twitter banned obvious keyword searches for deepfakes. Google updated its policy to enable requests for blocking search engines results relating to “involuntary synthetic pornographic imagery.”
But many examples of such videos continue to appear on popular social media services and even adult websites dedicated entirely to deepfake porn. A 2018 report by Deeptrace found more than 8,000 deepfake porn videos on various adult websites, along with hundreds of such videos on YouTube. The challenge of flagging and removing deepfakes will only grow as the tools for developing deepfakes become more common and easier to use.
“One striking element of the current state of deepfake technology compared to two years ago or so is just how little knowledge of machine learning an individual requires to create synthetic media using this technology,” Patrini says.”
Much work on deepfakes has focused on the face-swapping and facial expression alterations. But Deeptrace sees the deepfakes problem as a broader one that includes digital puppetry of human body movements and the synthesizing of fake audio that mimics the voices of real people.
Comedic or illustrative examples of deepfake videos have sometimes featured famous men such as President Trump and former President Barack Obama. But the spread of deepfake porn has overwhelmingly affected women such as Hollywood actress Scarlett Johansson, whose face has been digitally inserted into pornographic videos viewed millions of times. Even women who are not public figures have often become the targets of deepfake porn made at a going rate of about $20 per video, according to The Washington Post.
Sometimes deepfakes are weaponized as a synthetic form of “revenge porn” designed to publicly humiliate people for personal or political reasons. In April 2018, Indian journalist Rana Ayyub became the target of deepfake porn that both used her image and and fake accounts impersonating her on social media.
The rise of deepfakes represents an especially disturbing prospect when fake news events—often taking the form of conspiracy theories or rumor-mongering—have already led to real-world threats of violence and even the deaths of innocent people. Some experts even worry that truly convincing deepfakes could undermine public trust and heighten misinformation—such as during presidential elections—in ways that threaten the foundations of democratic institutions and governance.
Creating Lines of Defense
It can be difficult to develop deepfake detection when there aren’t many examples of deepfakes in the wild beyond those focused on pornography, says Tim Hwang, director of the Ethics and Governance of AI Initiative at the Harvard Berkman-Klein Center and the MIT Media Lab. He suggests focusing on solutions for concrete deepfake issues instead of developing general deepfake detectors for more speculative scenarios.
“If your real concern is about fake revenge porn or your real concern is about the creation of amateur pornography, that’s quite a different situation than a state actor trying to manipulate political discussion,” Hwang says.
Any technological solutions may involve a sort of AI arms race. For example, Deeptrace sees the same adversarial machine learning used to create deepfakes as a primary tool for detecting deepfakes. Deeptrace’s “antivirus software for deepfakes” comes as part of a broad portfolio of solutions, including a database of known and popular attacks based on existing deepfake algorithms, Patrini says.
Giorgio Patrini, CEO and chief scientist at Deeptrace.
To help prepare for future threats, the company is also creating new deepfake examples specifically to train its defensive software. Another project involves building a database with personalized models based on celebrities, politicians and other public figures, which can better train video analysis algorithms to detect deepfake anomalies. Deeptrace is even exploring the possibility of pairing audio and video channels to boost deepfake detection accuracy.
But any possible solution for detecting deepfakes must do more than just work. The decision-making behind such solutions must be transparent, easily explainable for users and easily debugged by engineers. “Opening the black-box of fake detection is as important as building accurate models,” Patrini says
Hwang sees the likeliest solution as a balance between automated detection tools that can screen millions of videos and more sophisticated human-based scrutiny that can focus on trickier cases. For example, journalists, fact-checkers and researchers can collect and consider supporting evidence about the context of what a video supposedly shows in order to corroborate or debunk its contents. That could prove especially helpful in spotting an especially polished deepfake.
“If you have a big state actor that creates a completely custom deepfake video of someone and really tries to disguise it, it might be harder to figure out if it is really is a fake or not,” Hwang says.
Fighting Future Fakery
Last year, Hwang started an informal bet among researchers about whether or not a deepfake viral video of a U.S. politician would emerge and gain more than 2 million views before the end of 2018. Hwang and other experts who took a more skeptical view of deepfakes technology’s progression won the bet when the 2018 U.S. midterm elections came and went without any deepfake video making a significant splash.
But even the skeptics agree that truly sophisticated deepfakes capable of mass social disruption could more likely emerge around the 2020 timeframe when crucial political campaigns will be in full swing. Even less sophisticated deepfakes could wreak havoc in a world where many people regularly fall prey to online conspiracy theories and fake news posts.
The potential abuse of deepfakes could get even worse as the technology evolves. Deeptrace has observed the open-source development of deepfakes that could play out in real-time during a live event hosted on common video-conferencing software. Patrini expects it won’t be much longer before such deepfake technology becomes available through smartphone apps that anyone could use.
Given the potential impact of deepfakes, there is increasing interest in detection tools. The U.S. Defense Advanced Research Projects Agency (DARPA) has led the way by funding research through its Media Forensics project focused on automatically screening for deepfake videos. In 2018, several academic institutions began releasing video training datasets for deepfake detection and building detection methods.
In September 2018, the AI Foundation raised $10 million to build a tool that uses both human moderators and machine learning to identify deceptive malicious content such as deepfakes. And in December, the Symantec Corporation displayed its demo of a deepfake detector during the BlackHat Europe 2018 event in London.
Popular social media and video-sharing platforms could be among the first customers for such solutions. In anticipation, Deeptrace has committed to making its deepfake detection easy to integrate with existing platform user interfaces and data pipelines. But the startup is also in talks with watchdog organizations that have more limited budgets—fact-checkers, human rights charities and independent journalists—about sharing some of the same tools.
After all, many governments and corporations have the resources and expertise to detect and deal with deepfakes targeted specifically at them. But a more insidious threat may come from the increasing prevalence of deepfakes undermining overall public trust in real digital media—perhaps to the point where more people begin dismissing authentic video or audio sources as fake. Without constant vigilance against that possibility, societal bonds could begin to unravel.
“The biggest danger, therefore, is not how deepfakes may impact governments or sophisticated institutions, but how deepfakes may infiltrate spaces such as social media and personal or trusted interactions,” Patrini says.
Editor’s note: An earlier version of this story incorrectly referred to Deeptrace as Deeptrace Labs.
Social Media