If you've seen ADAM: The Mirror and ADAM: Episode 3 you probably have a lot of questions. How did they do that? What does it mean? When can I see more? Have a look at the comments on YouTube. People are passionate about these Made with Unity short films.
A big part of the global audience's curiosity is about the incredible fidelity of the ADAM sets and characters, and how they were constructed, integrated and rendered at 30 FPS in a game engine. For an apparently synthetic creation, there's a whole lotta realism going on.
On the clothing front alone, we see delicate, full-length hooded robes, roughly woven headdresses and cowls, schlumpy leggings, tough leather gloves, gauntlets, belts and other accessories - a whole panoply of apparently castoff couture, from primitive to high tech.
ADAM costumes mocked up at Oats Studios
Making it real
Fascinated by this dystopian clothing and how it came alive in ADAM, we interviewed some of the principals responsible for creating and simulating it for The Mirror and Episode 3. Chris Harvey, Oats' visual effects supervisor, told us:
"The whole approach for ADAM - even the physical environments - was to work from a place where Neill [Blomkamp, director] and I were quite familiar, which is working practical and doing visual FX. There's something innately real about using tangible and physical elements rather than something that is purely artistic and tech."
Additionally, Harvey's experience has taught him that when studios do everything digitally, there are lots of details that they typically overlook. "That's why we prefer to build something in real life - something that you can touch and feel and see."
And Blomkamp concurs: "We're building real-life wardrobe and using costume design like traditional film, and we're going to use photogrammetry to turn it into three-dimensional objects to be draped over our 3D characters in the films."
Getting started early: Clothes before casting
Even before the ADAM casting was complete, Oats gave Kristin Thurber, their costume designer/seamstress, the go-ahead to begin designing the clothing for memorable characters like The Mirror, Needalus, and Marion. Without knowing the exact measurements of the actors, they knew they would be able to tune and tweak the clothing digitally, further down the pipeline.
Like many of the characters in ADAM, Thurber scavenged materials to build her wardrobes. "These are all recycled items of pre-made clothing that have been destroyed, painted, melted, then duct-taped back together and layered. I pillaged a lot that was lying around in the studio."
Costume designer Kristin Thurber working on Marion's clothing
Sewing pattern recognition
As she constructed the various costumes, Thurber was working from traditional sewing patterns she developed, which were also an essential ingredient for "rebuilding" the costumes later in CG, as Harvey explains:
"People will probably be surprised to learn that we work from traditional sewing patterns - you know, the kinds we saw our moms using when we were growing up - but importantly, nothing is regular in them, oftentimes they're triangular shapes with weird cuts and grooves and stitches that hold them together, so the way the clothing ends up draping is controlled based on those elemental patterns."
The Mirror's costume being built in Marvelous Designer. Note the sewing pattern
According to Harvey, most people modeling CG clothing just build a tube, stick it on an avatar and then sculpt it virtually, causing it to move very differently, if not unrealistically, from something that is built from more traditional sewing patterns, which are more attuned to actual human shapes and movement.
Two-fold scanning: Digitizing the cloth
Once the costumes were created, they were taken into Oats' photogrammetry room to be photographed in high resolution for later reference by Sean Frandsen, a "digital tailor," as Harvey explains: "Those 'cyber-scanning' sessions were crucial so we could see exactly how Kristin intended the clothing to look and drape, then Sean could look at it on the real actors after the scan, and he could make adjustments to his patterns to make sure the digital version draped and moved realistically."
Frandsen explains: "I worked from Kristin's actual patterns because the pattern stays on the body better, moves how the characters are supposed to move. I try to follow the correct physics of each type of cloth because I have control of all that."
The real costume being scanned (left) and the simulated version with authentic physics and movement
As well as the full 3D cyber-scanned references for volume and colors, Frandsen did a second, closer pass, according to Harvey. "Sean spent hours in a darkened room with this rig we built to take multiple single shots of the thread-weave from 12"-square [30 cm] swatches, giving us 4K resolution. So that step gives us lots of control over the fabric details and colours, which we can use to tile across an entire article of clothing."
Frandsen explains further: "There were a number of textures we wanted to get. For example, The Mirror has holes in part of her robe, so we needed some opacity. To get that we put a light underneath the robe in the box so we could get a perfect opacity map. Then, without moving the cloth, we'd take eight different angles all the way around the cloth, then put all those angles together. We pick up all the little ripples and even finely woven textures, and capture it all as a normal or a bump or a displacement map. That's essentially how we would start, by building all of our own maps and all of our own textures. That gives us two UVs [texture maps] on top of all the cloth."
Detail: The Mirror's costume showing perforations and opacity
It was a Marvelous journey on the way to Unity
After some work in Photoshop and Adobe Lightroom, the swatches were imported into Marvelous Designer. Working from Thurber's sewing patterns, Frandsen commenced building the clothing, sewing pieces together digitally, taking great care to realistically mimic the natural characteristics of the fabric, whether it was wool, cotton, leather, etc.
According to Frandsen, this is a key differentiator from people working solely with synthetic cloth: "They'll bake-in folds that won't move or look right, whereas with all of the folds we do - nothing significant is sculpted or baked-in so that if a character lifts up his arms, we make sure that all the wrinkles and folds appear and behave exactly as they would in real life."
Clothing being animated in Marvelous
Blomkamp concurs: "For really high-resolution cloth movements, we simulated that outside the system, then we brought that into Unity. And so with all of the difficult pre-compositions done, the computer just needs to render the precalculated cloth so it looks like it's flowing. It's still unbelievable that the computer can compute that."
Crossing the Alembic bridge
A key to Oats' high-fidelity simulation hinged on getting all their geometry data into Unity exactly as they wanted it, as Chris Harvey says: "Using Unity's Alembic Importer was a super key bridge for us. We needed it to be able to cache large geometry datasets. It's a way for our artists like Sean to work in a very familiar manner in tools like Marvelous, Maya or 3ds Max and yet also be able to port that stuff very easily into the game engine."
You can read all about Unity's new Alembic support and how it helped Oats achieve their vision in ADAM in Sean Low's blog post.
The most challenging costume
As we know, when you're innovating processes and taking chances on new effects, you can run into any number of issues, so we asked Frandsen which was the toughest character to dress.
"Oddly enough, Marion [in Episode 3] - the character I thought would be the simplest - was actually the hardest due to multi-layer penetrations from all those hoses and stuff she's wearing, as well as the moving cloth. While the simulation itself might have been easy, the team had to do a lot of fixing, especially Chris."
Because of her multiple layers of clothing, Oats decided part way through development not to build everything in Marvelous because there'd be too much data to push through. So some of her under-layers were sculpted more traditionally by senior character artist Ian Spriggs, using a typical cyber-scanning process for the mesh, which they fitted around her, then sculpted it and used it for texture-baking. To finish, Frandsen built the other layers in Marvelous and draped them over top.
Concerned with getting even the smallest details right - tying up loose ends, if you will - Frandsen also took some clothing pieces into ZBrush and did a sculpted pass for stitches and other seam-level details to export as the normal maps.
Old-school advice for next-gen effects
Finally, what advice does Harvey have for anyone trying to mimic the best-looking cloth animation ever achieved in Unity?
"I would recommend that they start with sewing patterns. Use real references, base it in real physicality. Those are always smart things to do."
Learn more about Unity
Find out how Unity 2017 and its features like Timeline, Cinemachine, Post-Processing Stack, and real-time rendering at 30 FPS help teams like Oats Studios change the future of filmmaking.