Hit Amazon Prime series shot with virtual production workflow involving Stage Precision

Based on the video game franchise, the hit Amazon Prime series Fallout has captured the imagination of fans across the globe. Its post-apocalyptic storyline required the use of extensive virtual production (VP) technology to bring its environments to life on celluloid film for executive producer Jonathan Nolan. Creative design and VP studio partners, All of it Now (AOIN) and Satore Studios consulted Nolan’s production team on their first time shooting in a large scale 24K resolution in-camera VFX LED volume - and their toolset included deploying Stage Precision’s SP software as a versatile monitor to manage multiple tracked objects and cameras in the middle of the set.

“Our goal was to blend background scenes generated in Unreal Engine and seen in-camera with data from tracking systems attached to those same cameras to create a seamless production pipeline of visuals displayed on the LED screens in the volume,” explains Berto Mora, Head of Creative Technology at AOIN.

One of the main virtual production elements of the show focused on scenes featuring the “Vertibird” helicopter, specifically those where a backdrop of a barren, post-nuclear disaster wasteland was required. To do this, a Vertibird hydraulic-powered prop placed in front of a series of moveable LED screens displaying that background formed the physical basis of the shoot in the volume.

AOIN and Fuse Technical Group customised Stype Follower tracking beacons to fit both the Vertibird prop and the volume’s moveable LED carts in order to send their positional and rotational tracking data within the volume to Unreal Engine and Stage Precision simultaneously. As the Vertibird, film cameras, and moveable LED carts move around the volume, it is essential that the tracking data updates the background LED content to the correct position and perspective in relation to the doors of the helicopter, and the views of the cameras.

Not only did SP provide a valuable tool for data validation and camera tracking, the AOIN crew leveraged SP’s data monitoring features to build an alert system in case of data issues. After importing 3D models of each screen, prop and camera model into SP, they programmed each model to light up green whenever there was good data transmission, and red in case of any errors.

“There is a lot of data to be handled, so it was fantastic to bring all of that data into SP where we can check and monitor the statuses of our tracked systems,” says Neil Carman, Technical Director at AOIN. “Being able to catch any dropouts or errors in data means being able to rectify any issues before the production crew is aware, which is critical when any downtime to troubleshoot on a film set potentially means costly budget overruns. I believe traditional film crews hope that their virtual production counterparts do not delay or intrude too much on their workflow. Stage Precision allows us to work in parallel with them instead of halting the whole production. Our post-production partners can also work faster knowing we have delivered fully informed tracking data recorded within SP instead of data unknowingly strewn with errors.”

Extensive and usually time-consuming camera tracking calibration across different technologies was unified into SP, further saving time. Recalibration throughout the shoot could be run quickly and efficiently from the single source of truth in SP. “SP gave us the agility we needed to adjust on the fly,” continues Mora. “If we lost tracking at any point, we could unhook a camera from the data flow in the SP interface and move it to where it was meant to be in the workflow without needing to manually move cameras around.”

With a unified tracking and data monitoring workflow in place, the AOIN and Satore teams also found SP’s 3D viewport features to be pivotal in providing a visual reference to what was happening in the volume. AOIN built a 3D model of the volume, complete with Vertibird, cameras and moveable LED screens to compose a bird’s eye view of the action taking place within the studio.

“Because both the set and the volume were so large, we didn’t have a direct line of sight to what was going on during filming. SP gave us visibility over exactly what was happening, which is beneficial to us, but also to other crew members who are not so familiar with VP,” remarks Tupac Martir, VP Advisor at Satore. “When the director of photography had questions about the position of the LED screens, we were able to use SP as a previsualization tool to show him how he should block out his movements and lensing.”

The AOIN team also capitalised on the collected data and 3D models to create a virtual tour of the set on a VR headset and deployed AR tests to ensure the 3D space remained aligned with the physical set. Reflecting on the Fallout project, the AOIN team testify to the time-saving power of handling complex VP workflows in SP and its ability to reduce downtime. “There are a lot of people on set, and navigating all of these moving parts can be challenging,” says Carman. “We were fortunate to have SP in our toolkit on this project. Even though this was a new workflow for us we realised we needed it once we were on-site. SP’s user interface made it easy and intuitive to roll out.”

AOIN came to understand that the system they created in SP expedited the smooth operation of the volume and offered an extra layer of visibility that would not have previously been possible on a shoot of this scale. “The whole crew were appreciative of how smoothly the shoot went, and I believe that’s thanks to SP,” says Mora. “SP just keeps the production rolling.