INTERVIEW
with Sony Pictures Imageworks’ Theo Bialek
by Jessica Fernandes, Spark CG Society
© 2019 · Spark CG Society
August 22, 2019
Meet SUE: a Super Uber Elemental
In Spider-Man: Far From Home, our friendly neighbourhood Spider-Man is unwillingly thrust into the spotlight, battling a power-hungry, tech savvy, vengeful adversary and his legion of drones. With a shot count of 320, and crew size of 200, Sony Pictures Imageworks was responsible for the bulk of the impressive third act of the film. Theo Bialek, VFX Supervisor at Sony Imageworks, sat down with us to chat through their work on the project.
Having VFX supervised both Spider-Man: Homecoming, and Far From Home, what struck you as the biggest differences between them?
The scope. Homecoming was a much simpler FX film for us in the sense that it’s really just one character (the Vulture) against Spider-Man. In Far From Home, it’s Spider-Man against Mysterio, Beck, thousands of drones, and the final Elemental creature (which is a complex set of different FX phenomena — water, fire, wind, clouds, lighting, lava). It was a much more varied and complex set of characters and circumstances, seen from a lot of different angles. And unlike on Homecoming, our Far From Home scenes occurred in daylight. There’s less that you can hide when it’s bright, especially when you’re trying to recreate or extend a live action set.
I had been cautioned that with Marvel films especially, as the story is continually refined that changes to the sequences would be inevitable up till the end of post. Having recently finished on Spider-Man: Homecoming, another 3rd act joint Marvel project, I was confident I knew the drill. We took precautions, overbuilding key assets and staffing up for complexity increases we felt were likely to come our way. However, it didn’t take too long before our team realized this time was different and the precautions we undertook were only the start of what would be needed. Keeping our production team flexible was the best way to absorb all of the new ideas and constructs along the way.
Can you run us through some of the elements you worked on?
The Super Uber Elemental:
In the final act, Spider-Man battles and flies into the final Elemental creature (we named her SUE, the Super Uber Elemental). As originally conceived, Spider-Man was to swing along the peripheral of the storm, disabling the SUE illusion after confronting Beck and damaging his suit. Later in the production it was decided a more dynamic demise for SUE was needed. When instead an alternate idea of Spider-Man flying into the interior of SUE to disable hundreds of drones constructing her illusion with an electrified web network was pitched we thought maybe it was a joke. It didn’t take long for the reality to sink in when we realized the scope of the challenge. Rapidly prototyping various styles and methods for the interior shell of SUE we endeavored to figure out what the inside of an illusion could look like. Eventually settling on the construct of malleable voxels of light powered by drone holographic lasers to make up SUE’s interior skin shell.
On the exterior SUE had a collection of different effects that made up her form, but one of the most expensive was her water tentacles, these huge streams of churning water with an internal core of fire that go into the sky as lightning crackles along the perimeter. To generate these effects came at a large computational cost, both in the time to simulate and the space required to store the data needed for rendering. As the number of SUE shots grew in excess of 50 it became apparent that custom tentacles for every shot wasn’t going to be feasible. To help keep our CPU and disk space expenditures to a minimum we decided to cache out several long simulations and share them across the shots as opposed to generating unique effects per shot. A single 400 frame sim from our water tentacle, with its associated spray, internal fire core, and lightning, approached 1000 core hours to simulate and 5TB of data on disk. A large amount of data to keep on disk for the duration of the show, but it still ended up saving considerable time versus creating bespoke FX per shot.
To facilitate the fire effects on SUE and elsewhere in the sequence we created Inferno, a proprietary tool developed in Houdini, that allows us to run distributed fire and smoke FX simulations across multiple machines on a queue (versus having to run sims on a single machine). It also allows the artist to run a lower res setting, get feedback on the results, and then if approved, iterate, refine to a higher res and render. Typically, when you have low res simulations for these types of phenomenon and effects, there is a lot of wasted work to add the higher resolution back. But, in our tool, the rough version was a good indication of what the high res would be. That was a new stage we developed specifically on this show, to help us verify our sims. It’s now being used on multiple projects.
Drone Battle & Set Extensions:
The third act required vast distances to be covered within a shot given the expected amount of drone and Spider-Man combat throughout the environment. Given this, we needed to ensure we had a large enough CG environment to accommodate all the action. How much of the set to build virtually at high detail was made especially challenging given the scope and complexity of the shots remained fluid and ever expanding throughout the production. Whenever possible we would elect to use plates, however we often found the constraints of the shot design and camera moves required a full CG environment. By the end of the production we had created a high detail CG version of the bridge and surrounding area that could be used for either set extension or full CG.
The majority of the plate work was filmed on the backlot of Leavesden Studios outside of London. Partials sets were constructed to recreate portions of the bridge roadway and the elevated walkway above where Beck coordinates his attacking drones.
Shooting on the actual bridge wasn’t an option logistically as the cost and constraints of shutting down a functioning landmark wasn’t practical for the time a full shoot requires. We were allowed limited access though this was only useful for acquisition photography and reference footage. On this morning the city shut down the bridge for 2 minute intervals roughly every 15 minutes over the course of the morning. This allowed us just enough time to clear traffic off the bridge and run out to various locations to shoot unobstructed pano and tiles along the roadway of the bridge.
Plates were also shot on location during normal hours from the towers and sidewalks of the bridge, but this meant hundreds of pedestrians and traffic in frame, mostly rendering the footage as being valuable reference only. As a general rule, whenever possible we would always try to shoot our plates and reference at three times of day: morning, afternoon, and late afternoon. This was done so we could capture different sun directions from each location. Additional footage was shot from a barge on the river, atop a double decker bus driving over the bridge, from a helicopter above, along the shores both on foot and from atop various neighboring buildings including City Hall and The Shard. By the end of the acquisition phase there wasn’t a spot our team hadn’t trekked across multiple times.
I heard that the different Elementals were based on previous Spider-Man villains (Sandman, Hydro Man, Molten Man, Cyclone). What did you use as reference for yours, as it’s an amalgamation of a number of different elements?
They supplied us with an initial piece of artwork that established the groundwork for her design, establishing the number of limbs and relative shapes of her appendages. What it didn’t do was give a detailed look at how all of the parts connected or the differing phenomena interrelated. As SUE was meant to be derived from the other earlier Elementals in the film and those were also being developed in parallel there was often a wait-and-see component to our character: where are the other vendors with their designs, let’s try to borrow ideas from that.
Originally SUE wasn’t intended to be featured so heavily in the film, only expected to be seen from a distance in a handful of shots. As the scope grew, and the design got increasingly more complex the character quickly became our largest challenge on the film. Originally imagined as discrete segments, the body was to be constructed solely of volcanic earth, the tentacle arms as water and a smokey cloud-like head sat atop. Following this design we approached her creation in the typical serial fashion – concept, model, rig, lookdev, FX, composite. When the direction for her design evolved it was all hands on deck and we rebooted the process back to the concept stage. The updated brief was to create a mixture of all the elements that relate more closely to the other elements now further along in development. This required the previous volcanic form to now include regions of clear earth as rock and mud, water, and smoke.
In terms of additional reference, we took our cues from films like Clash of the Titans, looking at other large types of creatures. We tried to learn from that, as well as graft from the other Elementals being created.
Who came up with the name SUE?
It was an Imageworks suggestion. Some of the early concept work was titled “Uber Elemental”, while another had the name “Super Elemental”. Because we didn’t yet have a unifying name, in jest I suggested we call it the “Super Uber Elemental”, combining both names. Realized SUE would be the acronym it just seemed like a natural fit. They liked it, so it stuck.
Was any mocap used for SUE?
Although some of the other Elementals used mocap, because SUE had multiple arms, even using one of her arms more as a leg, it didn’t really make sense for us to go that route and as a general rule we hand animated her performance.
In terms of process, our animators often act out and record their performances using a modified Xbox Kinect system. In the case of SUE, our animators would pretend to be the massive creature moving slowly as they try to mimic the motion a several hundred foot large monster. This motion is recorded both as video and as three dimensional scans. The 3D scan is then loadable into Maya where it is used as a dynamic reference to help match timing and poses to. This worked quite well in regards to SUE as fast method to block out first pass animation.
What can you tell us about the final drone fight in the walkway?
The extended opening shot of the drone fight in the walkway was definitely one of our most difficult shots. Originally envisioned as a reality-only shot, Spider-Man battles the last remaining drones in the confined walkway without the aid of webs. As a single camera take, well over 20 seconds in length, the shot was incredibly complex given the length, dynamic FX and destruction required. Spider-Man’s animation was a combination of several mini snippets of mocap takes and hand animation all spliced together and enhanced with a unifying layer done by hand. Deep into production the requirements changed and the shot design was based on starting in the illusion realm that gradually fades away from a void to reality as the drones are destroyed. This updated twist amplified the already complex shot as FX demands increased to now include the devolving illusion effects and accompanying signature Mysterio green vapor that inhabits the void world. With all of the flashing explosions and blue illusion effects It was a particularly challenging shot to wrangle the visual elements in a manner that didn’t devolve into chaos.
What was the biggest challenge on this show?
The unexciting answer to that is the timeframe and pace we kept. The nature of the project required us to remain extremely flexible and revisit and update shots frequently. With such a large sequence and so many interconnected characters it was logistical puzzle that required large resources to keep under control. On a purely technical side SUE was the biggest challenge for us. It could take six or more FX artists just to generate the elements for one of her shots.
What are you most proud of on this show?
The animation. It’s always a challenge on superhero films to keep the motion both grounded in reality, but still service the ideal that it has to be exciting and out of the ordinary. As Spider-Man was much more active in this film than in Homecoming, we needed to take more risks on how far we could push the performance but still not break the sense of realism. With well over 100 shots needing a full CG Spider-Man, the level and scope of animation was immense.
There are also a large number of shots in the film that required a full CG suit replacement. After the point in the sequence where Spider-Man catches on fire all later shots in our sequence needed to have the suit replaced with a burnt CG suit. As this idea was added late into the production all of the live action footage after this event now needed to have the damaged CG suit. That ended up being an excess of over 70 shots, that all needed rotomation, animation, lighting and comp to swap in a CG version.
Well done, I didn’t notice it.
You wouldn’t be looking for it, so that’s the advantage that we had!
Anything you wish you could have done, if given more time?
There’s a segment in the middle of the battle on the bridge we internally called the car gauntlet that was abandoned. Spider-Man is leaping over and through cars along the roadway as explosions are going off all around. It was a very dynamic and exciting action bit that ended up getting cut due to time constraints. Had we more time I would have loved to finished that bit up.
Thank you so much to Theo for taking the time to chat with us. I was fully engrossed from beginning to end with Far From Home. Well done Sony on wrapping the film up so spectacularly!
Spider-Man: Far From Home – © 2019 CTMG, Inc. All Rights Reserved.