Virtual Reality’s Breakthrough Is Coming From Filmmaking, Not Goggles

VR isn't dead, it's just different

For years, I’ve been making a quip to illustrate a principle about the power of storytelling. Recently, I realized that it came true.

The principle is this: Humans are wired for story. We tell each other stories to connect and build relationships. We tell ourselves stories to make sense of the world. Stories help us remember information better than charts or stats. Stories help us care about people and can motivate us to do challenging things.

The quip is this: No matter what newfangled VR-Snap-Robot-Media-Contraption gets invented next, that human power of storytelling will be the thing that makes it successful. Twitter only emerged as a force in the world when it became less about sharing what you had for breakfast and more about sharing stories—or links to stories. Tiktok’s virality comes from the stories built by creators on top of each others’ content. Instagram Stories are literally what’s kept the photo sharing app from going the way of the buffalo.

That brings us to virtual reality tech. Outside of video games, VR has struggled to find mainstream adoption. Like many tech enthusiasts, ever since I first put on a virtual reality headset years ago, I’ve been convinced that VR would change the world one day. But after half a decade since Oculus Rift came out and kids everywhere weren’t switching from living on Instagram to living in VR, pundits have variously declared that we should temper our expectations, or even that “VR is dead.” 

Except, something wonderful has happened.

If you’ve been watching Rings of Power, or House of the Dragon, or the new Star Trek—or if you’re one of the 16 million people who watched all of season 1 of The Mandalorian—you’ve spent hours of your life on a newfangled VR-Snap-Robot-Media-Contraption.

In a terrific example of lateral thinking, VR technology is now powering your favorite movies and TV series. We’re not putting on headsets, but the tech that powers a VR headset is being used to create virtual worlds for actors to perform in. Instead of flying film crews to Tunisia or building expensive physical sets, we can put far-flung locations on LED walls in a film studio anywhere. VR tech is what makes it look real.

“Be it filmmaking, broadcast, cut scenes for video games, these different types of storytelling are all using the same technology now,” says Ray Mosco, Developer Relations Partner Manager at HTC Vive, which makes VR and camera-tracking technology. 

For decades, filmmakers have used “rear projection” to pretend that actors are somewhere they’re not. Usually you see it in car scenes (where the world outside the car windows looks kind of fake). 

A classic example is the taxi cab scene from Pulp Fiction, when Bruce Willis sits in the backseat, talking to the driver, in front of a blurry projection of a road. It’s supposed to show the car in motion, and even though it’s likely by design, it looks so artificial that it distracts the viewer from the dialogue. 

The problem with rear projection is it often looks fake. As soon as the camera moves, the wall appears flat. And a projection on a wall just can’t quite compete with the crispness of the real world.

But now we have affordable LED walls—essentially giant TVs—that we can put behind actors. And to make things look real when the camera moves, clever folks have hacked VR technology to subtly move the LED background in 3D so the human eye believes it’s real. 
Instead of putting a 3D world inside a headset, we can put that world on a film studio wall.
As filmmakers started hacking VR tech to pull off this kind of virtual production, companies like HTC Vive, which originally focused on gaming tech, have stepped in to help.

“We saw the direction that the industry was going,” Mosco says, “and adjusted our existing hardware infrastructure to support that.”

In other words, the VR industry is leaning into filmmaking.

Recently, while several device-makers have launched enterprise tools to help big productions connect cameras to their 3D background walls, HTC Vive released a cost-effective VR product specifically for filmmakers. Vive’s relatively new Mars tracker lets any film crew connect a camera to a 3D world for a few grand—less than the cost of many film cameras themselves.

All of this suggests that mass adoption of VR tech won’t start inside some sweaty goggles; it could actually stem from the ability to help filmmakers big and small tell stories on regular screens. Sure, VR gaming will continue to make leaps. But your grandparents watch Disney+ at night, and your little cousins watch Youtube on their tablets. Those are the places VR tech is making the biggest impact right now.

Simply put, VR’s mainstream success might be as a tool for creating stories for different platforms, rather than VR being the medium itself.

“I don't think virtual production is a silver bullet for every film production,” Mosco says, “but there are definite cost savings, and it's really exciting to be able to make this technology more accessible so that more people can tell stories with it.”