How The Third Floor Helped Director Joss Whedon Plan and Shoot Marvel's Biggest Superhero Epic to Date

Something amazing happens when reality exceeds expectations. For comic-book fans, seeing all six Marvel superheroes on screen fighting evil in a film together was a long-anticipated and often-promised event. And, when writer/director Joss Whedon took it up a notch with a smart script, clever dialog, and deft stage management, the film achieved that rare combination of insane anticipation, critical acclaim, and boffo boxoffice. Marvel Studios' The Avengers, brought to us by Walt Disney Pictures, grossed $178 million in its opening weekend overseas, followed by a record-breaking estimated $200 million weekend in North America. .

There are many reasons why The Avengers works so well – not least, the highly-praised visual effects supervised by Janek Sirrs and created by artists at Industrial Light & Magic, Weta Digital, Pixomondo, Digital Domain, Scanline VFX, Luma Pictures, Hydraulx, Fuel VFX, Evil Eye Pictures, and others. One studio in particular helped Whedon sort out the action: The Third Floor, which provided previs and postvis.

Leading the team of 15 Third Floor artists who worked on the film were Nick Markel, who supervised previs from August 2010 until August 2011, and Gerardo Ramirez, who supervised technical previs — techvis for short — Preand postvis from May 2011 until January 2012.

Previs

“If I had to add [up] the runtime for the previs in my head, I’d saw we did around 45 minutes,” Markel says. The studio previs’d three main sequences: A mountain fight in which Captain America, Thor, and Iron Man face off against each other, later realized by Weta Digital; the helicarrier attack, realized by ILM and Weta Digital, in which Hawkeye causes engine three to explode (Weta) and Hulk and Thor have a standoff (ILM); and the end sequence – the climactic battle – which ILM created, in which the superheroes battle Thor’s evil brother Loki and his army of aliens in New York City.

“When I finished the show, the end sequence was around 26 minutes,” Markel says. “The great thing about this film is that this sequence takes place in New York, so we had plenty of reference and a known location to work from.”

The previs artists started with storyboards, but often the two processes overlapped. “We had a chance to sit in some of the storyboard review meetings to see how Joss [Whedon] reacted to the visuals,” Markel says. “And Bryan Andrews, the lead storyboard artist, would stay involved and be in our meetings as well.”

For the previs, the Third Floor artists working in Maya updated low-res versions of the superheroes they had created for previous Marvel films, modeled characters new to the studio, and built environments and sets. “It usually takes between two weeks and a month for the asset builders to create all the pieces we need to start our previs,” Markel says. “And that work continues for assets that appear later.”

For a first pass, the artists design camera angles, frame the shots, and block in the characters on still frames. “We’re working in three dimensions, creating still frames to generate ideas quickly and get an initial buy-off from the director,” he says. “And it’s a great blueprint for motion capture.”

For motion capture, The Third Floor uses Xsens MVN, a self-contained camera-less system that uses inertial sensors and a biometric model that keeps the data humanly possible. “The great thing is that we can do mocap on the fly,” Markel says. “We put stunt guys in the suits — our own artists get in them. We use it quite heavily on all our shows to create a higher level of previs with more finesse in the animation in the same or less time than keyframing. It makes you feel like you’re almost watching the film in videogame resolution.”

Although motion-capture data worked well for Thor, Captain America, Iron Man, the Black Widow and Hawkeye, animators performed the Hulk with keyframe animation. “He’s unique in his proportions and in the way he moves,” Markel says. “He’s big and strong, but he’s in our world with our physics. We try to make the previs as accurate as we can and keep everything in reality.”

Because the artists typically start working on previs before a director of photography joins the production, they often don’t know what lens packages the cinematographer will select. “If I don’t know what the camera is, I use super 35 and a standard lens package,” Markel says. “As we get more information, we adjust.”

As for the shooting style, the previs team gets notes from the director and then incorporates the DP's vision as soon as he or she is on board. For this film, Markel brought director Joss Whedon and cinematographer Seamus McGarvey into The Third Floor’s offices to scout digital locations. The company has three InterSense IS-900 virtual camera motion tracking systems that they use with Autodesk’s MotionBuilder. Those systems are similar to the ones they used while working with Robert Zemeckis on Beowulf and Mars Needs Moms, according to Markel. “Sometimes we use the systems for shot production,” he says. “But on this show they were purely for digital scouting, to give Joss and Seamus a sense of the locations and identify issues before shooting.”

Markel and the artists met weekly with Whedon for official reviews, but the director often stopped by at other times to look at the latest work. “He’s great,” Markel says. “He had a good sense of what he wanted. And the four of us — Joss [Whedon], Janek [Sirrs], Bryan [Andrews], and I — would collaborate. The way I approach previs is to get the creative part figured out. I don’t want the director to fall in love with something that’s impossible to shoot, but we want to provide an open canvas for creativity. We keep the technical things in the back of our minds, but the hardest thing to achieve is the creative. Once we have an idea cut together as a previs edit, we can go back and see where technical issues might occur.” The crew honed technical issues during techvis, as well.

Techvis

“We would prep techvis throughout principal photography,” Ramirez says. “We’d do technical diagrams and breakdowns to support the second unit or principal unit with the shoot.”

For example, working from footage taken of the helicarrier set – the entire bridge – the techvis artists calculated the camera angle used to film the actors. Next, they calculated camera angles and positions for the aerial photography team shooting background plates. “That way, the production could avoid having to do CG skies,” Ramirez says. “They could film all the angles with the correct perspective, confident the direction they pointed would match the plates back in the office.”

For another sequence, this during the alien battle, they calculated the volume and rate of expansion for an explosion. “The shot was originally previs’d,” Ramirez says. “The special effects team requested the technical breakdown to see whether the height and speed of the smoke would obstruct the camera view.”

The special-effects team filmed the explosion they planned to use and The Third Floor artists built a CG duplicate of the production set in New Mexico. The techvis artists matched the explosion’s shape, volume and speed frame by frame and figured out what the camera would see. Then, they plotted out where the explosions should go. “We gave them a new camera move, new timing for the explosions, and new timing for the characters running through the shot,” Ramirez says. “They rigged the explosions according to the techvis schematic and filmed it, and it looked great.”

Postvis

Once filming started, The Third Floor’s postvis artists applied the previs elements to the plates they received. “One advantage of postvis is that while shooting, we can apply temp visuals to a shot within a day or two, find out how the actual footage works, and if the camera is looking in the right direction,” Ramirez says. “Sometimes we applied the CG characters and set extensions to a series of shots and the director and editorial decided to re-shoot a different version.”

Postvis also gives the director and editors richer scenes for cutting decisions, especially for sequences with partial sets and ones in which CG effects will drive the story. In fact, Ramirez found himself working closely with the editorial teams.

“The end battle was the biggest challenge,” Ramirez says. “It’s packed with action and there are hardly any moments with just dialog. We spent a good portion of postvis doing shots for that battle. It had so many elements and each character had a moment within the battle. In one epic shot, one camera moves to each of the characters. We had six or seven elements shot on greenscreen to track and tie together to incorporate into one massive camera move. Thankfully, because we did the previs we had a huge library of mocap, and we could re-use a lot of elements we created for previs in postvis.”

On location in New Mexico, Ramirez and a group of five artists tracked cameras, did tech vis, animated, and composited shots. After principal photography, the team moved into Marvel’s production offices in Manhattan Beach. “It was a great setup,” Ramirez says. “It was great being next to production throughout previs and postvis,” Ramirez says, “[and] to have one group of artists that know the film doing previs, techvis, and postvis.”

All images copyright Marvel Studios. Previs and postvis imagery courtesy of The Third Floor.