DeepFake Micheal Scott Digitally Aged up
Clean with No Visual Effects
Why The Office?
If you know me then, you know I am a huge fan of The Office. I always have it on playing in the background and I am pretty confident I’ve seen it more times than anyone else.
So it seemed like the right choice to incorporate it with something new to learn.
What is a deepfake?
Deepfakes are artificial media in which a person in an existing image or video is replaced with someone else’s likeness, in this case, replaced with an older version of the actor that doesn’t actually exist.
While the act of faking content is not new, deepfakes leverage powerful techniques from machine learning and artificial intelligence to manipulate or generate visual and audio content with a high potential to deceive.
Something to think about in this statement is
“powerful techniques from machine learning and artificial intelligence”
REAL Deepfakes are generated with the help of machine learning and artificial intelligence.
Is this a deepfake?
There are three total programs I am using to achieve this effect.
-
Adobe After Effects – No AI
-
EBSynth – No AI
-
Face App – Uses AI
While these programs do not use machine learning to create the imagery, at least one of the apps uses artificial intelligence, stated here on the FaceApp site.
Within the above parameters, I would call this a “poor man’s deepfake”
The process
This project was my first step into creating a Deepfake. Now, this isn’t exactly a deep fake you would see online with Bill Hader and Arnold Schwarzenegger.
It’s all compositing with a little help from two different apps. The first application is a phone app, called Face App.
What it does is almost magic, but that must just be the Artificial Intelligence.
You import a photo that contains a face, and you can manipulate it with the face filters that they provide.
Some of the common ones are, Beauty filters, Young filters, adding facial hair, and even blending two people’s faces to create a new face altogether.
I exported my clip out as a PNG sequence from after effects, this is needed later for EBSynth, then sent the frames I needed to my phone using Dropbox.
I took the first and last frame of each shot that I was aging his face in and ran it through the app.
These are the results from the app.
EBSynth
The next step to this process is using a piece of Beta Software called EBSynth.
It is currently free to download on their website, but they do plan on releasing a full paid subscription version, so I’d recommend getting the beta while you still can if you don’t want to pay for it later on.
It works like an “input-output” system, you input a modified photo still frame from the clip you want to work on, and it looks across all the frames in your clip and tries its best to make the change you made on the one frame, be applied to every frame in your shot.
Watch how it works below.
These steps will get you about 70% of the way there, the next is bringing the frames into After effects and compositing them with the original footage.
Adobe After Effects Compositing
What I ended up having to do was create multiple edited frames of his face at different intervals in each shot, the first and last frame, and blend them altogether through each shot.
In the first shot, I was able to get away with one pass without much blending.
While shots like this last one took a lot more work and blending to get it to look right. Including creating a matte of this face and head that I had to Rotoscope to keep all of the face morphings contained.
EBSynth is rough on the eyes and mouth as they change in each shot, they just get distorted and smoothed out. The fix I use is manually masking out the eyes and mouth to show through the edited older face.
Here is an example of all the edited portions of the face on the last shot, anything transparent is what would be the original footage shot.
Here is an example of how I had to blend multiple passes together overtime to get it to look right .I darkened the original footage to show the brighter edited parts and how they change over time.
Conclusion
I think learning how to do this is super valuable to any editor who is looking to learn more Visual Effects tricks.
As time goes on more and more face replacements and deepfake videos are being created for both viral online videos and feature films, and now it has already slipped into the commercial world.
For me this has been a great workaround for not having to deal with 3d face tracking, EBSynth really takes care of it for you.
This will work for a lot of other things not just faces, I have used to paint out a tattoo on someone’s neck for a commercial.
I hope this breakdown was helpful for you, let me know if you have any questions or would like to see a more in-depth breakdown.
Please leave a comment below about your thoughts on this, What have been some of your favorite deepfake videos you’ve seen online? or tell me what your favorite episode of the office is, that’s just as interesting to me!
