LogInWētā FX mixed machine learning and Nuke.
Marvel Studios
If you have seen Deadpool & Wolverine already, you might be wondering how facial expressions were animated on Deadpool's mask. Wētā FX shared the workflow it used for the movie and explained how machine learning played its part.
For this film, the studio used processes refined in the previous Deadpool episodes, mixing machine learning into the Nuke pipeline, improving its relighting system, and adjusting angles "to make sure that we were able to rebuild and keep the resolution of the plate and the fabric and everything that we were touching no matter what direction that any character moves," Wētā FX visual effects supervisor Daniel Macarin told befores & afters.
To bring Deadpool's mask to life, the team did match moving (inserting computer graphics into footage in the correct place) in Nuke and animated the mask. Macarin said that the facial setup in Nuke is linked to Wētā FX's facial system of around 128 face shapes
"Also, we’ve been making some tech refinements over the last few shows to make sure that the artist isn’t having to clean up a dramatic amount. The only thing that they’re concentrating on is the performance."
The facial performance was done by Ryan Reynolds or a stuntman, who wore "the actual black leather on the mask," a thin fabric mesh as "the stuntie mask," or nothing at all. Then, Wētā FX linked the mouth movement to the rest of the face.
"It’s the same with body movement or position," Macarin explained. "If the face is doing something that the body isn’t doing, you’re going to quickly read that something is off. You may not know exactly what’s wrong, but you’ll see that something is wrong. The facial movement on a plate [footage or image used as a foundation, background] is only difficult for us when someone says, ‘We’d like to change the line, we’d like them to do something else.’ And then you’re trying to readjust the jaw, stabilize everything, get rid of any lip movement or facial movement, bring it back to a neutral set and then animate on top of that.”
Plate (Wētā FX, Marvel Studios, befores & afters)
Final (Wētā FX, Marvel Studios, befores & afters)
To make sure the animation doesn't look cartoonish, the studio tried to retain the plate as much as possible. After, the team used machine learning to estimate what the facial performance should be from the audio, and an animator took over from there to fix it if needed and add details.
"Anytime someone shuts a car door or you hear an explosion in the background, the eye does a little tweak or a twitch. That wasn’t something we would’ve normally hand-animated because it’s just not part of the dialogue. But when we had those machine learning paths, which had thousands of keys, you could scale down what it was doing and you got this really, really nice ambient motion to the face that gave it a much more natural feel and allowed the artist not to have to do thousands of little micro keys to try and get those really subtle movements. It helped give a little more life to the character than just straight facial performance."
With Wolverine's mask, the studio had other challenges. It has unmovable metal pieces and wasn't supposed to be changed, but Macarin said that Hugh Jackman has a particular brow movement that the team wanted to transfer. It's very subtle, but you can see it in the yellow fabric on Jackman's forehead.
Plate (Wētā FX, Marvel Studios, befores & afters)
Final (Wētā FX, Marvel Studios, befores & afters)
"Now, being that you are taking a plate within this mask and if you scrunch it down, there’s nowhere to grab. So, we had to rebuild the outside sections of the eye anytime his eye got smaller. That’s a challenge we didn’t have with Deadpool."
If you'd like to learn more about it and see how Wētā FX reanimated Wolverine’s corpse, read the full article here. Also, join our 80 Level Talent platform and our Telegram channel, follow us on Instagram, Twitter, LinkedIn, TikTok, and Reddit, where we share breakdowns, the latest news, awesome artworks, and more.