Are deepfakes a novel trend or a novel threat?

Speculation over the potential for misuse of AI and machine learning technologies have been bandied about for longer than the practical applications of these technologies have been available. We’ve seen concerns manifest in popular culture as depictions of a dystopian future, and while reality is considerably less dramatic, there have been some developments that are nonetheless worthy of examination.

To be clear, there is still no semblance of real intelligence in AI tech in the way we understand human intelligence. What does exist today is a fairly good approximation of human decision-making that can be applied to perform specific tasks with great efficiency at scale. That said, some advancements in AI tech have raised some eyebrows. One in particular is deepfakes, a rapidly advancing technique for generating very realistic media that has the potential to be very disruptive when misused.

Deepfakes in brief

Deepfakes are a machine-aided synthetic media technology used for the generation or manipulation of text and video that can appear quite realistic to the untrained eye, and they have the potential for considerable implications across culture, geopolitics, and security.

It is important to delineate the three main types of deepfakes so we can understand their implications:

Mimicking fakes: this is a technique where video of one person is superimposed on a target video using AI to enhance and mask the manipulation. Think of this as an advanced green screen process that effectively lets one person do all the talking while it appears to be someone else, right down to movements and gestures.

Generative fakes: this technique also employs AI algorithms, but in this case, they are used to completely synthesise new audio and video from existing materials to produce ultra-realistic content.

Generative text fakes: this more common technology uses AI applications such as the OpenAI GPT-3 to allow computers to generate text content on almost any subject that is incredibly close to actual human writing.

Each of these applications pose a separate and distinct threat on their own if misused, but in combination, have the potential to produce generative personas that in the near future will be extremely difficult to distinguish from the real thing. This is where the security implications come into play.

Malicious deepfake applications

Generative text fakes using GPT-3 level tech have the potential to make phishing attacks far more convincing and much harder to detect in an automated fashion, as the newer iterations of these techniques can generate text in individualised speaking styles that closely resemble specific people.

Right now, advanced generative text technology is largely confined to a defined group of people that OpenAI granted access to. They blocked the last generation of the tech from public release due to fear of abuse. So, for the moment, that channel isn’t a significant threat. That said, the massive knowledge base used in model training means the tech is getting very close to being indistinguishable from actual human authorship, and it’s not clear that a strategy exists to better detect these fakes in the future.

Mimicking fakes with superimposed video are already good enough to fool an unwitting viewer to a reasonable degree. If this technology can be applied in real-time and coupled with advanced audio fakes, there is the potential that you might one day find yourself in a video conference call with someone you think is your CEO but is actually an attacker.

Think of the impact that the abuse of fake social media accounts has had on the success of malicious state-sponsored disinformation and influence campaigns in recent years. Now imagine it’s not just mono-dimensional sock puppet personas inundating media channels, but dynamic manifestations of what appear to be real people saying and doing real things.

Finally, consider the implications a convincing deepfake video of a high-profile figure, going uncontrollably viral on the internet, could have on public safety or geopolitical relations. The potential for real harm is significant.

Identifying deepfakes

Generally speaking, the successful detection of fake generated media will be based on our ability to recognise repeated patterns generated by the algorithms used in their production. In the case of text fakes that leverage GTP-3, the machine learning relies on a massive collection of human generated text, and the same machine learning techniques used to create the fakes would also be used to identify true text vs fake text. This is a case where the better machine learning application has the advantage.

GTP-3 tech is so advanced that it could be abused to create deep-phishing messages that speak in the voices of many and will be impossible for the current generation of anti-phishing systems to identify. Algorithms for both generation and detection of deepfakes rely on the same data pool for training the models, so it’s very hard to distinguish between text generated by the newer algorithms.

As for video deepfakes, at this point they generally have a perceivable artificial quality to them, even really well-executed fakes. The best weapon we humans have for determining if a video is a deepfake for now is for us to learn to recognise and acknowledge that uneasy feeling that what we’re looking at doesn’t feel real and is somewhat alien. This uncanny valley is the key to unravelling fakes, they just feel wrong intuitively.

But as these technologies improve, it’s clear that more than human intuition will be required to make a solid determination on authenticity, and it may be the case that it is simply not going to be possible without the creators of the technologies or the fakes themselves intentionally leaving an indicator of some sort, like a virtual watermark.

Either way, deepfakes and our ability to ensure they cannot be utilised maliciously, are definitely a subject worthy of further discussion before the technology advances to the point where we can’t put the genie back in the bottle.

Key takeaways

• What exists today is a fairly good approximation of human decision-making.
• Deepfakes are a machine-aided synthetic media technology used for the generation or manipulation of text and video.
• The massive knowledge base used in model training means the tech is getting very close to being indistinguishable from actual human authorship.
• You might one day find yourself in a video conference call with someone you think is your CEO but is actually an attacker.
• More than human intuition will be required to make a solid determination on authenticity of texts or videos.