'Volumetric' AR Poised to Ride the 5G Wave
"Help me, Obi-Wan Kenobi. You're my only hope."
We've all watched the scene unfold in Star Wars: A New Hope numerous times -- R2D2 projects a 3D, holographic image of Princess Leia pleading for Kenobi to get off his sorry butt and aid the Rebels in their struggle against the evil Empire.
That bit of fiction was set a long time ago in a galaxy far, far away. But back here, today, in the real world, using smartphones (and, soon, fancy connected headsets) equipped with augmented reality (AR) capabilities to capture and display so-called "volumetric" content that simulates 3D depth is apparently on its way to becoming a more affordable (and portable) reality.
And that reality, particularly when it comes to connected mobile devices, is expected to be driven into the mainstream on the 5G networks of tomorrow.
Taking aim at the emerging market for "volumetric capture" is Jaunt, the well-financed, software-focused production and distribution startup that initially targeted the market for cinematic virtual reality (VR) before branching into AR and mixed reality (MR).
Jaunt, which counts Walt Disney Co. (NYSE: DIS) and Google Ventures among its backers and UK-based Sky as a key deployment partner, is focusing on what it views as the next wave of AR content with a system that can capture and distribute volumetric video, including 3D renderings of people. (See Disney's Ben Havey: Collaboration & Content in a 5G World.)
Jaunt's volumetric capture system aims to inject true scale -- with respect to both economics and reach -- to a segment the market that has been relegated to multi-million-dollar volumetric stages that require hundreds of cameras and massive processing power (and time) to output the content.
The Jaunt VR system optimized for volumetric capture is designed to be portable and sold under the company's software-as-a-service model. It pairs a mix of off-the-shelf hardware -- cameras (an array of six to eight, typically at $250 each), microphone stands and a computer to handle real-time processing -- with Jaunt's specialized software.
George Kliavkoff, Jaunt's president and CEO, stressed that the software is Jaunt's secret sauce, as it features machine learning capabilities to auto-calibrate camera placement and depth-sensing, and meshes that data with the captured video and audio into a smooth volumetric form for transmission. Playback also happens on the Jaunt XR platform to enable its partners to deliver it via their mobile apps, social networks, websites and other compatible platforms.
The latest version of Jaunt's handiwork, Kliavkoff explains, is designed to take advantage of the onboard rendering capabilities of iOS and Android smartphones with an eye toward more advanced AR glasses and headsets. (See Magic Leap Jumps to the 5G Future and AT&T Invests in Magic Leap, Strikes Exclusive Consumer Deal.)
The debut also comes about a year after company founder and CTO Arthur van Hoff created an R&D team focused on developing a scalable AR content platform that was also a step above early, static and flat AR apps and experiences such as Snapchat filters.
Jaunt envisions a variety of immersive, interactive use cases where consumers can, for example, obtain volumetric views of celebrities directly from the red carpet or individual football players during game introductions or during game breaks.
"It creates a new asset type," Kliavkoff said.
Jaunt didn't get into the bit-rate requirements for the transmission and delivery of volumetric content, but Kliavkoff said live playback on WiFi worked "perfectly" and that it's expected that existing mobile networks will be able to support that content by Q4. Further ahead, performance will be even better on tomorrow's 5G networks, he predicts.
"We think [volumetric capture] is an incredible use case for 5G … It's a compelling argument for 5G to have this new media type that can be distributed," Kliavkoff said, adding that volumetric capture is sizing up to be a "mobile-first experience."
Kliavkoff said Jaunt is working with a "very large" US mobile carrier on an effort that will use this advanced form of AR to show off their 5G network. Jaunt is also introducing this new capability ahead of an expected commercial launch in November with an unnamed "global partner," he added.
Jaunt has reformulated the company in the couple of years that Kliavkoff has been at the helm. After focusing on the consumer market with high-quality, cinematic VR, Jaunt has become more B2B focused as it put more emphasis on AR and MR systems for studios and other partners looking to dive more deeply into immersive storytelling.
"We're actually doing more AR than VR these days," Kliavkoff said.
The brief video below illustrates the process behind Jaunt's new volumetric capture system:
— Jeff Baumgartner, Senior Editor, Light Reading