What is Deepfake Technology and how does it function?

In April, during ESPN's hit documentary series 'The Last Dance,' State Farm debuted a TV commercial and it became one of the most widely discussed ads in recent memory. It appeared to show footage from 1998 of an ESPN analyst making shockingly accurate predictions about the year 2020. As it turned out, the clip was not genuine. It was generated using cutting-edge artificial intelligence. The commercial ad surprised, amused and delighted viewers. The State Farm ad was a benign example of a vital and dangerous new phenomenon in artificial intelligence (AI): deepfake technology.

Deepfake technology enables anyone with a computer and an internet connection to create realistic-looking photos and videos of people saying and doing things that they did not say or do.

What is a Deepfake?

Deepfake is a type of artificial intelligence used to create convincing images, audio and video hoaxes. It describes both the technology and the resulting bogus content, is a portmanteau of deep learning and fake. For example, Barack Obama calls Donald Trump a "complete dipshit," or Mark Zuckerberg brags about having "total control of billions of people's stolen data." Have you noticed these? If yes, you've seen a deepfake. The 21st century's answer to Photoshopping, deepfakes uses deeplearning to make images of fake events, therefore the name deepfake.

How Deepfakes Work

Today, open sourced software such as DeepFaceLab and Faceswap make deepfake videos. These programs use an autoencoder to compress a face into a very compact representation and then decode it back to the original.

To create a deepfake, one needs to feed the program two main pieces of data. One is the source: a video that includes the face that one wants to swap in. Second is the destination: a video that adds the face that one wants to swap onto.

The program requires many good, high-quality videos of both the source and destination to create a believable deepfake video to feed the autoencoder. Once one has the source and destination videos, the individual can proceed through the below steps:

Image Extraction

The first thing that the program does is extract images from the source and the destination videos. The program runs through both of the videos and extracts frame-by-frame screenshots of all of the faces. Then the faces must be cleaned up manually. It indicates that someone must scroll through the thousands of extracted face images to remove any photos that are not relevant, like photos of other people or objects. It is a tedious and hours-long process that must be completed for every deepfake video created.

Encoding and Decoding

The next thing that occurs is the encoding and decoding. The final selection of images is run through encoding and decoding to train the encoder. The process of encoding extracts data from the image and compresses it into its smallest form called latent space.

The encoding concentrates on the parts of the face that change as the person changes their facial expressions. For instance, it doesn't focus on features such as eye colour that remain relatively constant. Instead, it focuses on things like eyebrow location that changes drastically as one speaks.

The compact form of the image is then decoded back into the face, and checked against the original. This process is repeated numerous times until the autoencoder is trained on that face. This process of learning through repetition represents the AI part of the deepfake.

Future Prospect

In 2020, Facebook and TikTok banned deepfakes and will now take them down if they are identified. Although Twitter still allows deepfake videos to stay on its site, it applies a manipulated media label to any deepfake videos detected.

Deepfake videos and deepfake identification technology are symbiotically evolving. As the deepfake videos improve, the identification technology also improves, which in return pushes the deepfake videos to keep getting better.

Deepfakes were anticipated to run rampant in the 2020 election; they never materialised, but that doesn't mean that it can't happen in future. As this technology improves and becomes less resource-intensive, researchers expect that deepfake videos will continue to become more prevalent. Therefore, it will be essential for deepfake detection technology to keep up.

The biggest threat deepfake videos are likely to pose is that they add another layer of distrust to legitimate video and news. Knowing that deepfakes exist is somewhat destabilise, and this is evident in the politicians already claiming that authentic videos are deepfakes created to discredit them.

As long as people continue to stay vigilant, as they do with other fake news sources and frauds, the threat of deepfake videos continues to be small. And consequently, our democracy seems to be safely stable for another day.