Linking the Unlinkable: iClone Unreal Live Link for Animators

Introduction

I belong to animated movie makers. To a very curious type of animators; the ones that are in a constant hunt for new tools and better solutions. At one point I stopped counting on the CGI - related software I had installed thus far. Along the way, I noticed that the more I work, the fewer tools I use. So let’s say that now I’m using only 5 - 6 different programs. Two of them are the base, and the rest of them I use for preparation and post-production. Most animators work the same way. It is impossible to be an advanced user of 30 different software. Especially, if you belong to the real-time animation world; if, at some point in the production, you use a software that was not made primarily for film-making.

Working with the iClone

After preparing and gathering all the assets I need, the so-called melting pot of animation for me is Reallusion’s software, the iClone.  It is more than capable of creating advanced animation of any kind. It’s a mature program, easy to use, yet it’s very powerful. 

I’ve made several movies with it. Quite often I place characters (that were animated in iClone) into scenes and environments that were created with the “Unreal” engine.  And no, using real-time engines does not mean that the author is impatient. We just enjoy the benefit of the engine’s efficiency and the freedom of the uninterrupted flow of creative thoughts. 

In fact, filming with real-time engines often requires more patience than using traditional CGI software.  For example, Unreal is a tool for making games, not movies. You can imagine the amount of import and export, tweaks and trickeries that are involved here.

I use iClone primarily. It’s a software dedicated to animated movie-making.

Everything is there: assets, props, lights, cameras, timelines, curves, facial animation, lipsync. Plus, there are also customizable 3D characters made in another Reallusion software, the Character Creator or native iClone models, or imported ones from some other programs.

iClone has a pretty decent rendering engine as well,  but if I wanted to send animated characters to Unreal to benefit from the glorious possibilities that only Unreal has, I had to go through the complicated procedure of exporting models from iClone to Unreal. I needed to export them as an FBX file; I had to import and customize the textures and materials, the shaders, etc. Well, I had to - up until now.

 Unreal Live Link Plug-In by Reallusion 

The “Unreal Live Link Plug-in” for iClone bridges two software in a way that the user tends to have the feeling that one is the extension of the other. In a nutshell, the simplicity and efficiency of iClone are combined with the Unreal engine’s magnificent rendering power.  It’s as simple as that.

With that plug-in, I can directly transfer iClone characters to Unreal without a tedious method of the FBX import-export. It’s like a data link: a chain between iClone and Unreal where I can simultaneously do animation by using iClone and follow the real-time render in Unreal.  And I am not just talking about the characters, but the lights and cameras as well.

Imagine the possibilities of using the friendly environment of one software and having the outcome by another. That's why I said that it’s like having Unreal as the extension of iClone.

And let me emphasize that we are talking about two major software in their own fields.  What used to be an endless struggling between the two is now a smooth connection. Let me put it like this:  what used to be some occasional dating is now a serious love affair. What more could you possibly want?

Advantages of working on Live Link Plug-in

The most difficult part became the easiest with the help of the “Character Creator & the iClone Auto Setup Plug-in”, which work in conjunction with Live Link Plug-in. It automatically assigns digital human shaders to the linked characters. It does the same with the skeleton system.

No more manual work with the shader assignment. It used to be an enormous task in the Unreal material system. Now, not only that plug-in does all that for us, it even gives us the possibility to adjust and fine-tune the shaders in Unreal, which is especially important for the visually tricky parts, such as the creation of the eyes and the hair.

When Reallusion gave me those plug-ins to test, I thought I was doing something wrong.

After a few obvious clicks, an animated character from the iClone ended up in an Unreal scene fully synchronized and dressed in all the proper materials, with the already adjusted shaders - without any intervention by me. I thought it was a mistake, a glitch; I suspected that it couldn’t be so simple. It seemed almost too simple for the tortured soul of an animator.

When we transfer a character from iClone to Unreal,  the LiveLink plug-in automatically processes skeleton-mapping and bone-renaming into the Unreal standard. This method opens the way for iClone characters to adopt Unreal motions and for Unreal characters to adopt iClone motions.

The Facial Animation and Live Link Plug-In

 All the morph/bone structure-driven animations are linked to Unreal.  All of them. iClone has an unmatched system of facial animation indeed; 60 facial muscles, an adjustable visem strength, expression pre-set, a facial puppet, blend, shape sliders. Everything iClone has for animation is synchronized with the plug-in and can be edited in real-time. The motion capture data we use in iClone? Sure. Live performance capture for body, face, and fingers. Even simultaneous capture of motion data streamed from different hardware.

I keep telling you, this plugin makes the Unreal engine a logical, easy-going, desirable and attractive partner of iClone. This is not the result of a blind date.

The Influence of Live Link Plug-In on the Workflow

Well, it’s a game-changer for me. 

The era of countless hours of exporting files to FBX, importing them to Unreal, then reassigning the skeleton structure, the materials and shaders are over. It wasn’t just a time-consuming process, but – in a way - a creativity killer. Artists’ mind works fast and chaotic. We are in a constant self-investigating mode, in an endless success-versus-mistake loop. 

For instance, sometimes I am working on one motion of a character and, at the same time, I’m thinking about the next one. At times, something in the Unreal environment makes me rethink the previous import from iClone, so I have to repeat the whole process. And by doing that, my plans for the next motion or scene get lost in the meantime.  And so on, and so forth.

With the LiveLink Plug-in, I can immediately change whatever is necessary because I have a constant and ongoing view of the final look. And if I’m satisfied with the outcome of the scene, I simply record the shot in Unreal. That’s all.

It’s important to add that, in addition to controlling the assets that are transferred from iClone, we can take a walk in the opposite direction, too; we can link the Unreal-created camera, lights and characters to iClone. How good is that?! Not to mention that we can link and relink stuff, control a character in Unreal with a different character in iClone. Stuff like that. We can synchronize the iClone cameras with the Unreal Cine cameras, they altogether have adjustable parameters such as near-far blur levels or FOV.

Current Projects that Require Live Link Plugin

I am working on a documentary that is not really a documentary per se. Also, it’s some sort of a thriller that’s not really a thriller, either. Its title is “Something coded this way comes”.

It relies heavily on the LiveLink Plug-in because the story is related to game engines. In the movie, I intend to literally show how some animators’ tools work – among others, this very plugin that we’re talking about.

The movie is 15 minutes “short”, and I’m about to finish it any day now, so the end is near. Speaking of the end, I have an idea. Someone (perhaps, me) should take an imaginary interview (perhaps, with me) about the love connection between two software in the real-time animation industry. If I were to write that, I would close the interview with the phrase “…and they live-linked happily ever after.”

Tom Jantol, Film Director and Animator 

Interview conducted by Reallusion