We can say what we want about the metaverse’s perceived lack of traction, but despite the many setbacks it has faced, Meta CEO Mark Zuckerberg has clarified that the company is not giving up on its metaverse ambitions – and after seeing Meta’s new VR avatars, the company’s vision for the metaverse is still very much alive.
In a recent ‘Metaverse interview’ on the Lex Fridman podcast, Zuckerberg unveiled Meta’s latest Codec virtual reality avatars, which utilise scanning technology to create 3D models of the user’s face.
Unlike the company’s initial Horizon Worlds metaverse avatars, which faced ridicule for being legless, having lifeless eyes, and unsettling appearances, these new projections are exceptionally lifelike.
Photorealistic Codec Avatars
When asked how these photorealistic avatars came to life, Zuckerberg told Fridman, “We both did these scans for this research project we have at Meta called Codec Avatars. The idea is that instead of our Avatars being cartoony, instead of actually transmitting a video, what it does is we’ve scanned ourselves and a lot of different expressions and we’ve built a computer model of each of our faces, and bodies, and the different expressions that we make.
“And [we] collapse that into a codec that then when you have the headset on your head, it sees your face, sees your expression and it can basically send an encoded version of what you’re supposed to look like over the wire. So in addition to being photorealistic, it’s also much more bandwidth efficient than transmitting a full video or especially a 3D immersive video of a whole scene like this,” said Zuckerberg.
Codec Avatars are remarkable not only for their striking resemblance to real individuals but also because people can easily understand them by watching a video on YouTube. Zuckerberg proposed that users could modify their virtual look-alikes to appear more expressive, and humorously mentioned that he might use it to enhance his well-known expressionless delivery.
“This gets to the core of what the vision around augmented reality is, of delivering a sense of presence no matter where you are in the world,” said Zuckerberg. “Most of communication is not actually through words, but with expressions – we tried to capture that with the more cartoon-designed avatars that we have, but there’s obviously a certain realism that comes with delivering the photorealistic experience.”
Meta’s vision for the metaverse is still alive but will take time
While Meta’s Codec Avatars aren’t based on an abstract idea like the nascent metaverse, they are a highly practical and straightforward feature with a clear purpose to connect people from all around the world. The responses to the podcast on Twitter and YouTube have been full of enthusiasm, and have reignited belief in the metaverse vision for many.
As remarkable as the avatars are, it will take some time before they become readily accessible to the public. Zuckerberg mentioned that Meta is still in the process of making the scanning process more user-friendly and that the integration of this technology into Meta’s products will occur gradually over the next few years.
The photorealistic projections represent a significant leap forward from the cartoonish avatars employed in Horizon Worlds. Zuckerberg faced harsh ridicule on social media last year when he shared a selfie of his metaverse avatar outside a virtual replica of the Eiffel Tower which the public compared to the graphics of video games from the 1990s.
Zuckerberg acknowledged that the avatars were rather ‘basic’ at that time.
Meta has faced challenges in generating enthusiasm for Horizon Worlds, including among its own staff. Zuckerberg informed Fridman that the introduction of the new photorealistic avatars might attract individuals who were deterred by the metaverse’s somewhat eerie graphics.
“People were blown away by the experience, but we do get some feedback that people are saying that the avatars don’t feel as realistic in this environment,” said Zuckerberg. “I think something like this could make a very big difference,” he added.
