Ray-Ban Stories are the first product to come out of a multi-year partnership between Facebook & EssilorLuxottica. Facebook › $FB › is working on a long-term project — an augmented reality metaverse. The seriousness of its ambitions is reflected in its headcount. Nearly 10,000 people are working in its group developing augmented & virtual reality devices, or nearly one-fifth of Facebook’s total global workforce. Facebook took the first step on this road in 2014 with the acquisition of VR headset maker Oculus, & they are three generations into commercial versions of that. The latest is the Oculus Quest 2, the first mass market VR headset that did not have to be tethered to another device like a PC or gaming console.

VR remains a niche product for gaming, & besides the price, there are other barriers to adoption even in gaming. But that is not Facebook’s endgame. Their endgame is something that Apple › $AAPL › is also working on — a pair of AR glasses, as light & fashionable as Ray-Ban Wayfarers › OTCPK › $ESLOF › that will replace the touchscreen smartphone as people’s main device by the end of the decade.

Facebook is coming at this project from three directions

The Oculus headset, which is big and bulky, & is about as powerful as a top end Android smartphone. The new Facebook/Ray-Ban Stories glasses. These are much more light & fashionable, but have few features, & are a smartphone accessory. On the software side, Facebook has begun previewing their version of the metaverse, first for CBS This Morning in August. The end goal is a device that brings that all together, dispensing with the compromises of the two current devices, creating a new experience, and the first entirely new user interface since the graphical user interface was invented in the 1970s.

VR, or virtual reality, is a system that feeds the user an entirely computer-generated environment via a headset and usually a glove or other handset for control. Everything the user sees and hears is computer-generated. The best current example is Facebook's Oculus Quest 2 headset.

AR, or augmented reality, mixes computer-generated elements with the real world. This can be a heads-up display, or computer-generated objects that are situated in a real world environment and move with it. The example that many will be familiar with is Pokemon Go.

You will also hear of “mixed reality” or, tortuously, “augmented virtuality,” but they are offshoots of these two main categories.

You can think of the metaverse as a massively multiplayer game, except it includes all social activities: meetings, conferences, education, live events, and of course gaming. As such it subsumes gaming, webcasts, video meetings, distance learning, & just about any activity where two or more people are together virtually.

These “smart glasses” are a pricey smartphone accessory, but it is best to think of them as a transitional device. They have camera, speakers and microphone, along with a touch interface on one of the arms for limited control. More control comes via Facebook Assistant voice control, which will be a key part of the eventual interface. Facebook’s aim here is too get people used to the idea of wearing a computer on their faces, & learn about use-cases.

The glasses are primarily Ray-Ban branded. I think this is a recognition of two hurdles here for Facebook.

For a device worn on the face, fashion is crucial, & Ray-Ban is the fashion brand. Facebook has lost a lot of trust regrading privacy, & that is one of the key hurdles to adoption. On the user end of privacy, Facebook is taking a relatively hands-off stance. You need a Facebook account to use them, but it collects a fraction of the data that the Facebook app does, & the data it does collect by default is entirely related to operation of the device. Users can choose to send more data, but again it is not the usual Facebook highly invasive stuff; it is more general data about how customers are using the device. This is a transitional device that is more about Facebook learning about how users want to use a face computer.

Trickier than user privacy is the privacy of everyone around the user. The Facebook page on Stories privacy highlights the challenge:

Are there going to be some ugly headlines coming out of this? Yes there will. Relying on users for 100% compliance is a losing bet. But Facebook has seen years of ugly headlines, including a week of relentlessly promoted “Facebook Files” articles in the Wall Street Journal just recently. They have very thick skins at this point, and they have a very big fish to fry here — the future of computing.

Facebook imagines this will be a much broader thing than just meetings, think free-to-use courses online that teach everything from language to technology, an open-source project needs a contributors, a community charity that needs a volunteer fundraiser, a co-worker who needs coaching on a new presentation, a story that needs to be told. Facebook, the service we now know, will be entirely subsumed by the metaverse.

Facebook realizes that there are a lot of scary dystopian elements to this, and in combination with all that lost trust, this can derail the whole thing. They have recently allocated $50 million to research ways in which to alleviate these issues. Facebook is working with experts in government, industry & academia to think through issues and opportunities in the metaverse.

Here are a few key areas where we’ll work with others to anticipate the risks and get it right:

Economic opportunity: how we can give people more choice, encourage competition and maintain a thriving digital economy

Privacy: how we can minimize the amount of data that’s used, build technology to enable privacy-protective data uses & give people transparency and control over their data

Safety and integrity: how we can keep people safe online & give them tools to take action or get help if they see something they’re not comfortable with

Equity & inclusion: how we can make sure these technologies are designed inclusively and in a way that’s accessible

—Andrew Bosworth, VP, Facebook Reality Labs and Nick Clegg, VP, Global Affairs, September 27, 2021

Facebook and Apple are the two primary companies working on this in my mind. There are others playing in this space for sure, but those two companies have the clearest idea of what the end product looks like, something we will not see in stores for years.

The device is a pair of AR glasses that many believe will eventually replace the touchscreen smartphone as people’s main device.

Each computing platform becomes more ubiquitously accessible and natural for us to interact with. While I expect phones to still be our primary devices through most of this decade, at some point in the 2020s, we will get breakthrough augmented reality glasses that will redefine our relationship with technology...

Even though some of the early devices seem clunky, I think these will be the most human & social technology platforms anyone has built yet.—Mark Zuckerberg, January 9, 2020

The big idea here is that we spend all this time looking at two-dimensional screens as a representation of reality, which dissociates us from the world around us. AR combines the real world with computer-generated elements & brings the virtual world of our phones & the real world together.

This is not just a new device, but a new human interface, the first invented since the graphical user interface was invented in the late 1970s. Multitouch replaced the mouse with a finger, but the 2D representation of a desktop has remained. A key part of the new interface will be using voice control to replace the mouse or finger.

In augmented reality, you're going to really need a pair of glasses that look like normal looking glasses in order for that to hit a mainstream acceptance. & that, I think, is going to be one of the hardest technical challenges of the decade. It's basically fitting a supercomputer in the frame of glasses.—Mark Zuckerberg, April 28, 2021

On the hardware side, Facebook is now coming at it from two directions. The Oculus Quest 2 is a standalone device built around Qualcomm’s Snapdragon platform, as most Android smartphones are. It has limited AR features, more geared towards letting developers play around with it. It’s successor, which we should see within a few months, will have more AR features, now for end users.

The Quest 2 weighs about a pound, & any vigorous use will make your face sweat because of the light seal. They look awkward, & cannot be used for long periods. It is primarily a niche product for gaming.

The other direction is the Ray-Ban Stories glasses, with far less hardware & features. They look very much like a regular pair of classic Wayfarers, except with camera hardware where the hinge rivets usually go. The strategy here is twofold. In the first place, Facebook loves data, & as I alluded to, the important data they are getting off these devices is not personal — it is usage data. They want to learn how people use these devices so they can tailor future development around that. The Quest has an installed base of about two million. This is a relatively small user base, but a very large test group.

Secondly, they want to meet in the middle, dispensing with the problems with both devices. The product they envision will look like Ray-Ban Stories, and also be a far more featured version of the Quest — less a niche VR device and more geared towards being an all day device that displaces the touchscreen smartphone sometime towards the end of the decade.

This is more or less where Facebook & Apple split in their approach. Facebook’s ambitions here are even broader than Apple’s because they are also building a metaverse. Apple is looking to transition the iPhone experience to a new device & interface. This will involve a lot of metaverse-like features, such as the soon-to-be-released SharePlay shared-experience interface. But they are not working on a full-blown virtual world like Facebook and others.

Archive

Tags