So imagine a room full of cameras able to capture performance from every conceivable angle, then reproduce the images inside a digital 360-degree environment, with VR users able to freely roam around the space and observe the action from a comprehensive set of vantage points. Phew! Now that we’re all on the same page, let’s delve into how volumetric capture plays a critical part in Runnin’—the new interactive VR music video from musician Reggie Watts, director Kiira Benzing and Intel Studios. Donning the VR goggles and clutching onto a controller in each hand, you enter into a neighborhood record store, replete with aisles and aisles of old-school vinyl bins. Once you’ve clicked on a number of visual cues—light beams, records that pop up from the bins, etc.—you’re transported to an interactive dance party in a retro-future world, where you and other pixelated partiers boogie down to Watts’ 2018 single “Runnin’.”
For the second panel of this year’s edition of The Portal, Portal curator and Film Independent Artistic Director Jacqueline Lyanga took a closer look at Runnin’ with the help of Watts, producer Adam Rogers, head developer Tim Lobes and Sarah Vick, Head of Business Development at Intel Studios. Here’s what we learned:
How did Runnin’ get started?
When VR fanatic Watts—who can currently be found as James Corden’s bandleader on The Late Late Show—first learned about large-scale volumetrics, he knew it was the special sauce to punch up the hypnotic techno beat he’d been working on with composing partner John Tejada (the duo band goes by the name Wajatta.) The germ of the project was then presented to Vick at Intel Studios, which was intrigued by the potential of injecting VR into the music space. But it wasn’t exactly Intel’s first rodeo into VR—the tech giant had already invested in two volumetric immersive studios four years prior, installing extensive volumetric capture equipment in a half- dozen NFL stadiums to produce Intel’s True View technology, which gives football fans the 360-degree replays and freeze frames we’ve all grown accustomed to in recent years. Immersive reality developer Lobes also saw the tremendous opportunity presented by this marriage of VR and music: “We wanted to take this completely audio-reactive universe and have a bunch of dancers. Since we were trying to make the Sundance deadline, we put this together pretty quickly in about four months.”
Judging image quality in VR
Lyanga brought up the piece’s stylistic choices, the hyper-reality of Runnin’s interactive record store bookend and the abstracted teleportational dance party, in which the visuals are more 2D graphic novel than hi-def cinema. “For me, it’s about the movement of the body,” said Watts. “What’s great about dance is the focus is on the expression of the body and not so much the intensity of facial expressions. I think it looks great—I love the abstract nature of it,” he said. Producer Rogers used to work in cinema and respects the medium, but working on this project with Watts has given him a newfound perspective. “Fidelity of the final image [in VR] is not going to be perfect looking as in real life. Let’s forget about making it perfect, break it apart and put it back together in a brand new form.” “Photorealism gets mentioned a lot in VR, but what does it really mean to be photorealistic? When you look at black and white films or silent films of the past, that was held up as ‘realistic’ for the time,” Vick said as she started to explain that one of the challenges of virtual reality is that people have grown used to 4K theater-quality and now expect the same standard in VR. “But none of the VR headsets in the market right now is anywhere close to 4K projection quality,” she said—hence the creative decision to render the dance party sequence in a series of colorful, abstract shapes.
What kind of tech was used to capture the dancers?
“When we shot Reggie’s project, we had 76 5K cameras running,” Vick said, adding: “Today we have 100 8K cameras on current projects.” To tackle the sheer volume of raw data captured for this project—which debuted at Sundance’s New Frontier exhibit this January—Intel also made available its proprietary data compression tool to help ease the pain. Volumetric data can be translated into four formats—traditional VR, looking glass display, volumetric display and finally, traditional flat 2D video (the “holographic version”) which further expands its utility and versatility for content creators.
Using Music to Maximize VR
Watts worked closely with director Kiira Benzing to tailor the live capture of the dance routines. “It’s all live capture,” he said. “Kiira worked with the choreographers to modify the routines to the 30-second maximum, because that’s how long we could shoot in one go. She kept track of all that and creatively melded the content once we’re done with live capture.” “I want people to have a reason to check out VR, and not just for the sake of trying the latest form of technology. If it doesn’t translate or if it’s not enjoyable, it’s not enjoyable, and it’s doesn’t matter if it’s live theater or VR. I think a music video is the perfect way to optimize and innovate entertainment and storytelling in VR,” Watts continued. Watts loved the record store in Runnin’—a sort of “book end anchor point” in the experience. “I just wanted to hang out at the record store. It doesn’t take much to make people feel welcome—it just takes the right vision and space.”
Pre-viz and Storyboards
“It’s kind of as you go, because the content that’s been captured is a done deal. But what’s interesting is you capture the content, and then you overlay that content onto an environment,” said Watts, adding: “In a way, it’s somewhat similar to live theater in a physical space.” Instead of worrying about how to do it for VR, Watt’s advice is to just design the piece traditionally as though you have a black box theater—get a lighting designer to do the lighting, get a choreographer to think about the spatial orientation piece, etc. Specifically for this project, they knew it was going to be a nightclub experience. “As we went along, we thought, Oh, I guess we can put people on the ceiling or the walls, and that just comes as you’re developing in real-time,” he said. But why “Runnin’”? The song was a single off of a Wajatta album that had already been released. “Before we released the record, we had this dream of a transmedia campaign for the album, where the single comes out, and a week later the VR experience gets released. Then, after that, a 2D video comes out on Vevo, followed by the holographic display, etc.” Watts laughed.
Early bumps in the VR sandbox
To Watts, where the XR industry is now is just the tip of the iceberg, saying we should be patient with the early bumps of this new medium. “You can forgive some of the imperfections and the early limitations of the technology,” he said. An audience member asked if Watts had any plans to incorporate VR into his comedy work. “There a little bit of that in Runnin’ where myself and [comics] Kate Berlant and Ben Schwartz are just riffing by the record bin aisles. We should be able to do a dialogue-based segment, but the limitation [of volumetric capture] right now is facial expressions.” Since it’s quite a big investment to create VR, another audience member wanted to know what steps are being taken to bring other consumers into this medium and branch out beyond hardcore VR enthusiasts. As Rogers lamented, they’re still figuring out how to deliver VR content at this point: “There’s actually a lot of VR content out there, but it’s just so scattered that it’s sometimes hard to locate the platforms to find what you want.”
Will there be an entire VR album?
“Oh yeah! Definitely—we had talked about that when working on Runnin’,” said Watts. The secret is getting the development period shortened so that the initial idea is still fresh by the time it’s released.” Watts thinks the secret sauce to mass VR adoption is in getting to the point where non-gamers are eager to get the headset: “You gotta generate that excitement, and I think music is the easiest and fastest way to get there.” To view the complete program of work exhibited at The Portal this year, check out our event program and watch highlights form this year’s event below:
A free event produced by Film Independent, and in partnership with Loyola Marymount University, The Portal took place September 20-22 at LMU’s Playa Vista Campus. To learn how to become a Film Independent Member, click here.
Follow Film Independent…
Twitter Instagram Facebook
