Tutorial: Using iClone and Faceware facial animation tools

Sep 14, 2018 at 09:00 am by Submitted

The Adventures of Nkoza and Nankya

Solomon W. Jagwe is a Ugandan 3D Artist and animator based in the USA, with over 15 years’ experience creating content for Games, Cinematics, Simulations and Virtual Reality apps. 

He shared this tutorial with iClone.

Portfolio Website | African Children’s Animated Series

Check out current Autodesk Promotions Here!

This tutorial covers my workflow of how I used iClone to create both Facial and Body animations for the characters in my Ugandan/African animated children’s TV Series, “The Adventures of Nkoza and Nankya.”

I have had this dream since I was a kid growing up in Uganda, of bringing my stories to life using animation, but I didn’t have the funds and team to pull it off. I tried for several years to bring the idea to life with the help of friends, family, and my two close 3D artist friends Bruno Ssekandi (Uganda) and Nateon Ajello (USA), but the progress was slow going because we all had other responsibilities including family obligations.

It wasn’t until I started using iClone, that I truly felt I had finally found a program that made it possible for me to realize that dream. Being able to bring custom characters into iClone, using 3DXchange, opened the animation possibilities to me. Using the iClone facial animation tools, I was able to create convincing dialogue for my series.

In this tutorial, I am using the little girl called Nankya, as the case study. The workflow is applicable to other characters. I am hoping that this process helps others out there who are looking for an affordable solution, to get their dream off the ground, and take it from paper to the screen.

 

3ds Max - Create high-resolution 3D assets for entertainment and design projects with comprehensive animation, modeling, and rendering tools. Buy now!

 

Step 1 – Prepare your Custom Character and Environment

SS_Nankya_Character_Morphs.png

In order for your custom character to work well in iClone and 3DXchange, it is crucial that you prepare your character model to have all the required blend shapes/morphs for the facial animation to work. Reallusion has done an awesome job of providing detailed steps on how you can prepare your character. Visit their YouTube channel for that insight. I also exported the set where the character was going to be animated, including all the props they would need to interact with, such as the phone for Nankya to call her grandpa.

Step 2 – Import Character into 3DXchange

SS_Nankya_character_import_01.png

Once you have a custom character with all the morphs needed for the facial and body animations, export your character as an FBX file, and import it into 3DXchange. You can use FBX 2011 or 2012. You need to have the 3DXchange Pipeline Edition to get the best results.

Step 3 –  Inspect and Export Character to iClone

SS_Nankya_character_Inspect_3DX_01.png

Take the time to check that your model’s morphs match the morphs that are listed in 3DXchange under the Expression Editor. Click on each tab for Eye, Jaw, Viseme, and Muscle. If any of them are missing, click on the missing one then toggle the auto key and shape your character to match the 3DXchange shapes.  This step is extremely important because iClone and the Faceware Facial plugin need all those listed morphs in order for the facial animation to work properly.

Step 4 – Organize your Project Folders

SS_Nankya_Character_Organize_Folders

In order to get your character into iClone3DXchange offers 2 main options. You can either send the character directly by clicking on “Send to iClone” or you can use the “Exporting to iClone Content” dialogue option. In both cases, I recommend first creating a Folder Structure broken down into Character, Props and Animation Subfolders under the Content Tab in iClone. This has saved me countless hours of searching for files, and also makes it easier for iClone to load your content faster.

Step 5 – Plan your Dialogue Shots

SS_Nankya_Character_Takes_00.jpg

This is the stage where you need to refer to any storyboard or animatic you have created to gauge what your character needs in terms of Facial animation and Dialogue. I broke down Nankya’s animations into Active Facial animation and Passive Facial Animation. Active is when she asks her grandpa a question and Passive is when she is listening to her grandpa to answer the questions. That approach enabled me to decide which iClone facial animation tool to use.

Step 6 – Passive Animation using Facial Mocap for Faceware

SS_Nankya_character_Passive_00.png

I broke down the passive animation further into Blinks, Smiles, amazement, surprise, quizzical, questioning and affirmation. I Positioned myself in front of the webcam and recorded a long take of those individual animations, making sure I gave each take a starting “no” animation pose and one for the end, with an average of 2 seconds of acting in the middle. The purpose for that is to make sure they can be blended with other each other after I cut them into individual iMotions.

Step 7 – Active Animation using Facial Mocap for Faceware

SS_Nankya_Character_Faceware.png

I categorize Active facial animation as the character Dialogue. When Nankya calls her grandpa, I used the Faceware iClone plugin to capture that dialogue. iClone offers several tools to achieve that including automated audio file and dialogue script processing, but my favorite and preferred approach is with the Facial Mocap for Faceware because of its ability to capture nuances. Make sure to calibrate your face before you start capturing. A camera that supports 60FPS will offer the best result. I use a Logitech HD webcam, but you can use any other type you can get your hands on.

Step 8 – Motion LIVE for Combo Facial and Body

SS_Nankya_character_Motion_LIVE.png

If you have access to a Perception Neuron suit, this is a great way to capture both the Facial animation and the body at the same time. It can be time-consuming to capture the face separately and then try to line up the body mocap later, which in some cases works well, but by using the Motion LIVE approach, you are able to see your character respond to both the mocap suit and the Facial Mocap for Faceware in real-time!

Step 9 – Create Facial Animation Bank

SS_Nankya_character_Facial_Bank.png

iClone offers a neat structure that enables you to reuse the animations you have created for your characters face. I often find myself working towards a set deadline for a shot, for example, if I need Nankya to blink. To save time, I use the Facial Mocap for Faceware to record various blinks of myself playing out the emotion I can reuse later on, and I save it in the Face Folder. I make sure no other animation is on the timeline, that way when I load those saved blinks and smiles, they layer nicely on the timeline.

Boxout 1 – Render Final Animation using iRay

SS_Render_with_Iray_in_Iclone_00.png

One of my favorite features of iClone, is the ability to animate and then render the completed animation right there inside of iClone. I used to export my animations out to render in Maya and 3ds Max using Octane, but now that Reallusion has incorporated iRayiClone has indeed become a one-stop animation shop for me.

Boxout 2 – Export Completed Animation as FBX

SS_Render_in_3DS_Max_with_Octane.png

You can still use other programs to render the completed animation. To do that, export the entire scene with the character or just the character alone, as an FBX file. If you have added physics to say the clothing or hair of your character, you can export the scene as an Alembic file, which retains the simulated animation, and import that into programs like 3ds Max, Maya, and Cinema4D.


Nvidia Store