VFX Team Talks 'Mulan' and How to Create Digital Extras Amid COVID-19 Era

 Source: The Hollywood Reporter

Weta created the Imperial City and populated it with digital citizens.

A key assignment for the visual effects team behind Niki Caro’s Mulan, which premiered last March just prior to the pandemic lockdown and was recently launched on Disney+, was creating its vast world and populating it with digital extras. As Hollywood restarts production, there’s increased interest in these techniques, as visual effects are recommended in order to limit the number of actors required on set.

The estimated 2,000 VFX shots in the production were shared by multiple VFX houses, including lead shop Weta Digital, whose work included the enormous Imperial City and its population that included real and digital inhabitants.

There are numerous techniques that could be used to create digital extras. In the case of Mulan, it started with scanning 70 extras in full costume. “We specifically built a new photogrammetry rig of like 124 cameras that was very flexible. That allowed us to capture actors and props and extras very quickly,” explained Weta VFX supervisor Anders Langsland. “We came up with a system whereby we could very quickly and inexpensively go straight from that 3D scan from the photogrammetry rig into a moving character in our crowd system." This crowd system used to animate the “extras” was the Scientific and Technical Academy Award-winning Massive, which initially became known when an earlier iteration of the software was used to create the digital armies in Peter Jackson’s VFX-Oscar-winning The Lord of the Rings trilogy.

Of the inhabitants of the Imperial city in Mulan, the movie's overall VFX supervisor Sean Faden explains, “All the digital extras are based on the real people that we had on set. So we could basically populate the entire city with all of that detail and all of those beautiful costumes.

“There’s still no substitute for being able to have at least some layer of what we would call our mid-ground or foreground extras," he adds. "That’s pretty hard to make those digital these days still. It seems like production-wise, we have to be smart about spacing those people, maybe doing multiple passes so that people can, in post, look like they’re a little closer together. But then once you get behind that first layer of people, then you’re into greenscreening and or you’re into replication of crowds and then you’re using the techniques that Anders is talking about where you can scan all of your extras, and then use those to make realistic [digital] people.” He emphasizes that for these scans, there's a continued need for costume design and the like.

“You might not have to have as many [extras as Mulan] but it’s really helpful for companies like Weta to have 10 or 20 extras that could be sampled and scanned and utilized to create those crowds.”

Langsland notes that these are conversations that Weta is having with filmmakers looking to return to production amid the pandemic with techniques for adding extras "whether that be by shooting elements spaced out as Sean’s describing or adding things in CG."

Weta is also getting inquiries about location work "because the logistics of moving production en masse and organizing safe accommodations for everybody and moving units around, is much more complex in these times," he continues. "In Mulan we created something that never existed, but creating digital versions of actual locations is something that is becoming increasingly important as it gets harder to actually shoot in locations in a safe fashion.”

Weta’s work on Mulan included an expansive digital Imperial City to augment the live-action sets. Based on the ancient Chinese capital of Chang’an, the digital city was used to give it a scale that Langsland described as the size of Manhattan. He notes that to do this, Weta developed a layout tool that he likened to a Lego set, effectively allowing the artists to generate the city from "building blocks" of homes and other structures.

In addition to Weta, VFX houses including Framestore, Image Engine, Sony Pictures Imageworks and Crafty Apes contributed to the work on Mulan, which also ranged from a climatic avalanche to more subtle de-aging of Mulan's parents. “We had to show the parents at their youngest stage when Mulan is a 9-year-old girl, then we had to show them again when she’s 16 or 17 and she’s about to leave, and then we had the final look where she comes home after the war." Faden explains that to do this, they used compositing techniques (compared with more complex de-aging approaches such as full digital head replacement).