Recently video games have made their way into cars. In December, Tesla got a holiday update that brought Steam to some of its vehicles. Likewise, NVIDIA’s game streaming service is available in certain Polestar and Hyundai models. But what about AR and VR experiences in moving vehicles? That’s what Meta and BMW are working to bring to new BMW models. The companies believe such experiences could lead to more productive and fun travels.
BMW and Meta first began investigating how XR experiences could one day be integrated into smart vehicles in 2021. Virtual Reality headsets use inertial motion sensors and cameras to track players’ movements. However, this becomes trickier if the user is in a car that’s also in motion. When used in a vehicle in motion, the sensors measure the acceleration of the car while the cameras observe the user’s motion relative to the inside of the car. This results in VR headsets being unable to work properly in accelerating vehicles.
VR in Cars: Fixing the Sensor Problem
To allow for the use of VR headsets in travelling vehicles, Meta has collaborated with BMW to integrate IMU data from BMW’s sensor array into Meta’s Project Aria research glasses tracking system in real-time. With this integration, the Meta Quest Pro can ‘accurately anchor virtual objects to a moving car’ through a digital twin.
Meta has been developing VR and AR demos for passengers in moving BMWs using this new tracking system. As a next step, the company plans to add the vehicle’s location relative to the world to enable world-locked rendering. Doing so will allow for the rendering of virtual content outside of the car, such as identifying landmarks.
“If we get it right, this technology could revolutionise travel in cars, trains, planes and beyond, unlocking new forms of hands-free communication, entertainment and utility — giving us far more value than the screens and instruments we’re used to seeing in vehicles today,” states Meta in a recent news post.
Jack Brassell is a freelance journalist and aspiring novelist. Jack is a self-proclaimed nerd with a lifelong passion for storytelling. As an author, Jack writes mostly horror and young adult fantasy. Also an avid gamer, she works as the lead news editor at Hardcore Droid. When she isn't writing or playing games, she can often be found binge-watching Parks & Rec or The Office, proudly considering herself to be a cross between Leslie Knope and Pam Beasley.
