Both the real & a virtual Mark Zuckerberg recently appeared on CBS recently to promote Facebook’s first public version of their metaverse. This is Facebook Workplace Horizons, a substitute for video conferencing. 

Facebook imagines this will be a much broader thing than just meetings, think free-to-use courses online that teach everything from language to technology, an open-source project that needs a contributors, a community charity that needs a volunteer fundraiser, a co-worker who needs coaching on a new presentation, a story that needs to be told & Facebook, the service we now know, will be entirely subsumed by the metaverse. The seriousness of its ambitions is reflected in its headcount. Nearly 10,000 people are working in its group developing augmented & virtual reality devices, or nearly one-fifth of Facebook’s total global workforce.

Facebook imagines that your social network & interests on Facebook will be at the center of the metaverse experience. This is essentially what Facebook imagines the metaverse to be, a mixture of your real world setting & the virtual Facebook world with your friends and interests. Facebook has been clearly thinking about this since 2013 & 2014 when they bought Oculus. 

Facebook realizes that there are a lot of scary dystopian elements to this, and in combination with all that lost trust, this can derail the whole thing. They have recently allocated $50 million to research ways in which to alleviate these issues. Facebook is working with experts in government, industry & academia to think through issues and opportunities in the metaverse.

Here are a few key areas where we’ll work with others to anticipate the risks and get it right:

Economic opportunity: how we can give people more choice, encourage competition and maintain a thriving digital economy

Privacy: how we can minimize the amount of data that’s used, build technology to enable privacy-protective data uses & give people transparency and control over their data

Safety and integrity: how we can keep people safe online & give them tools to take action or get help if they see something they’re not comfortable with

Equity & inclusion: how we can make sure these technologies are designed inclusively and in a way that’s accessible

—Andrew Bosworth, VP, Facebook Reality Labs and Nick Clegg, VP, Global Affairs, September 27, 2021

Ray-Ban Stories are the first product to come out of a multi-year partnership between Facebook & EssilorLuxottica. Unlike on the hardware side, where their primary competition is the world’s largest company, in the metaverse, Facebook primarily faces competition from much smaller gaming companies like Roblox › $RBLX & Epic. 

Labor & value & hard technology. Facebook imagines this will be a much broader thing than just meetings.

Facebook › $FB › partnered with European eyewear conglomerate Essilor Luxottica, Ray-Ban's parent company, to launch smart glasses in early 2019. Ray-Ban Stories, the smart glasses developed by Facebook & Ray-Ban, was launched on September 9, which marks a new chapter for Facebook as an eyewear developer that caters to the booming smart glasses market. These glasses allow the consumer to capture photos, record videos from a first-person perspective, listen to music, & take phone calls just like a normal smartphone.

Ray-Ban Stories come with a dual integrated 5 MP camera that allows the user to capture photos & easily record videos as experienced by the naked eye for up to 30 seconds, through a click of a button on the right-hand side of the glasses, or using Facebook assistant voice commands. Streamlined open ear micro speakers are built-in with a three-microphone audio array that delivers voice and sound transmission for calls and videos. Beamforming technology & background noise suppression algorithm provide an enhanced calling experience similar to what a user would expect from dedicated headphones. These features & an optimized Snapdragon processor, a capacitive touchpad, & a battery are fitted into the smallest possible frame one could imagine, which only weighs just five grams more than the original Ray-Ban Wayfarer design.

The glasses have a specially designed portable charging case to charge on the go. The technology of Ray-Ban Stories allows the user to pair up captured memories & stories with Facebook View app to import, edit and share them seamlessly on Facebook, Instagram, WhatsApp, Messenger and also on Twitter, TikTok, and Snapchat while facilitating to save content on the camera roll as well. Facebook View is a standalone app available on iOS and Android that lets the user import photos & videos from glasses and create unique content while having the freedom to control over what and when to share.

Ray-Ban Stories is just the first product of the multi-year partnership between Facebook & EssilorLuxottica. I expect the partnership between these 2 companies to result in more innovative VR products hitting the market in the next few years, but that is not the main reason why I believe investors should take Facebook's Virtual & Augmented Reality targets seriously. There are many reasons for that.

Facebook is taking Virtual Reality technology seriously, which itself is a good enough reason for investors to take things seriously as well. According to data collected by The Information, nearly 10,000 Facebook employees (or one-fifth of the total workforce), are engaged in developing augmented & virtual reality devices & software, which highlights that Facebook is very serious about pursuing the growth opportunities available with the increasing popularity of the VR technology. 

When I recently spoke with a few industry experts regarding the AI technology used by Upstart › $UPST › I made sure to inquire from them how Facebook could possibly integrate the VR & AR technologies to improve the user experience. These experts broadly discussed this topic and claimed that they would not be surprised if Facebook leverages this technology to become a mainstream VR & AR company that caters to the technological needs of multi-billion-dollar industries such as healthcare and education. One thing these experts agreed on is that these technologies will make connecting with people virtually feel much more realistic in the future, and I thought to myself, Facebook is going to be a big winner of  VR ‹› AR technology.

Facebook imagines this will be a much broader thing than just meetings, think free-to-use courses online that teach everything from language to technology, an open-source project needs a contributors, a community charity that needs a volunteer fundraiser, a co-worker who needs coaching on a new presentation, a story that needs to be told. Facebook, the service we now know, will be entirely subsumed by the metaverse. The seriousness of its ambitions is reflected in its headcount.

You can think of the metaverse as a massively multiplayer game, except it includes all social activities: meetings, conferences, education, live events, and of course gaming. As such it subsumes gaming, webcasts, video meetings, distance learning, & just about any activity where two or more people are together virtually.

Early signals of a monumental shift towards AR & VR

Facebook acquires Downpour Interactive, the developers of Onward. Facebook’s Oculus division has snatched up yet another popular virtual reality game developer. Recently, the company announced it has acquired Downpour Interactive, the studio behind the hit multiplayer military simulator Onward.

“We’ve seen great success with Onward on the Oculus platform for several years — first on Rift and more recently on Quest,” Mike Verdu, Facebook’s VP of AR / VR content, said in a blog post. “Becoming part of the Oculus Studios family will give Downpour Interactive the opportunity to cultivate both the Onward community with the full support of Oculus Studios resources, and, in the future, pursue other projects.”

Everyone from Downpour Interactive will move over to the Oculus Studios team “in some capacity,” Verdu said. He also confirmed that Onward will still be supported on all of the platforms where it’s currently available: the Oculus Rift, Oculus Quest, and Steam.

Dante Buckley, founder and CEO of Downpour Interactive, said in an FAQ on the studio’s website that Onward’s feature roadmap “remains unchanged” and that the studio plans to “accelerate the speed of our development” because of the acquisition. Downpour Interactive will also continue to retain “full control of all development,” he said.

The acquisition is Facebook’s latest to beef up Oculus Studios. The company bought Beat Saber developer Beat Games in November 2019, Asgard’s Wrath maker Sanzaru Games in February 2020, and Lone Echo creator Ready at Dawn in June 2020.

The launch of Ray-Ban Stories did not attract a lot of attention from investors, but with a good understanding of Facebook's VR strategy, it is very easy to see why this product launch marks a significant milestone for the company - Facebook wants the mass market (not just gamers) to embrace VR and AR products. With an effective strategy in place to pursue the growth opportunities available in this unique sector, I believe Facebook is poised to become the leader of the AR industry. Facebook stock price could hit $500 in the near future, and I am not making any adjustments to my valuation model at this time although I feel I will be obliged to make some positive adjustments in the coming months.

Facebook & EssilorLuxottica › The Future of AR & VR

Ray-Ban Stories are the first product to come out of a multi-year partnership between Facebook & EssilorLuxottica. Facebook › $FB › is working on a long-term project — an augmented reality metaverse. The seriousness of its ambitions is reflected in its headcount. Nearly 10,000 people are working in its group developing augmented & virtual reality devices, or nearly one-fifth of Facebook’s total global workforce. Facebook took the first step on this road in 2014 with the acquisition of VR headset maker Oculus, & they are three generations into commercial versions of that. The latest is the Oculus Quest 2, the first mass market VR headset that did not have to be tethered to another device like a PC or gaming console.

VR remains a niche product for gaming, & besides the price, there are other barriers to adoption even in gaming. But that is not Facebook’s endgame. Their endgame is something that Apple › $AAPL › is also working on — a pair of AR glasses, as light & fashionable as Ray-Ban Wayfarers › OTCPK › $ESLOF › that will replace the touchscreen smartphone as people’s main device by the end of the decade.

Facebook is coming at this project from three directions

The Oculus headset, which is big and bulky, & is about as powerful as a top end Android smartphone. The new Facebook/Ray-Ban Stories glasses. These are much more light & fashionable, but have few features, & are a smartphone accessory. On the software side, Facebook has begun previewing their version of the metaverse, first for CBS This Morning in August. The end goal is a device that brings that all together, dispensing with the compromises of the two current devices, creating a new experience, and the first entirely new user interface since the graphical user interface was invented in the 1970s.

VR, or virtual reality, is a system that feeds the user an entirely computer-generated environment via a headset and usually a glove or other handset for control. Everything the user sees and hears is computer-generated. The best current example is Facebook's Oculus Quest 2 headset.

AR, or augmented reality, mixes computer-generated elements with the real world. This can be a heads-up display, or computer-generated objects that are situated in a real world environment and move with it. The example that many will be familiar with is Pokemon Go.

You will also hear of “mixed reality” or, tortuously, “augmented virtuality,” but they are offshoots of these two main categories.

These “smart glasses” are a pricey smartphone accessory, but it is best to think of them as a transitional device. They have camera, speakers and microphone, along with a touch interface on one of the arms for limited control. More control comes via Facebook Assistant voice control, which will be a key part of the eventual interface. Facebook’s aim here is too get people used to the idea of wearing a computer on their faces, & learn about use-cases.

The glasses are primarily Ray-Ban branded. I think this is a recognition of two hurdles here for Facebook.

For a device worn on the face, fashion is crucial, & Ray-Ban is the fashion brand. Facebook has lost a lot of trust regrading privacy, & that is one of the key hurdles to adoption. On the user end of privacy, Facebook is taking a relatively hands-off stance. You need a Facebook account to use them, but it collects a fraction of the data that the Facebook app does, & the data it does collect by default is entirely related to operation of the device. Users can choose to send more data, but again it is not the usual Facebook highly invasive stuff; it is more general data about how customers are using the device. This is a transitional device that is more about Facebook learning about how users want to use a face computer.

Trickier than user privacy is the privacy of everyone around the user. The Facebook page on Stories privacy highlights the challenge:

Are there going to be some ugly headlines coming out of this? Yes there will. Relying on users for 100% compliance is a losing bet. But Facebook has seen years of ugly headlines, including a week of relentlessly promoted “Facebook Files” articles in the Wall Street Journal just recently. They have very thick skins at this point, and they have a very big fish to fry here — the future of computing.

Facebook imagines this will be a much broader thing than just meetings, think free-to-use courses online that teach everything from language to technology, an open-source project needs a contributors, a community charity that needs a volunteer fundraiser, a co-worker who needs coaching on a new presentation, a story that needs to be told. Facebook, the service we now know, will be entirely subsumed by the metaverse.

Facebook and Apple are the two primary companies working on this in my mind. There are others playing in this space for sure, but those two companies have the clearest idea of what the end product looks like, something we will not see in stores for years.

The device is a pair of AR glasses that many believe will eventually replace the touchscreen smartphone as people’s main device.

Each computing platform becomes more ubiquitously accessible and natural for us to interact with. While I expect phones to still be our primary devices through most of this decade, at some point in the 2020s, we will get breakthrough augmented reality glasses that will redefine our relationship with technology...

Even though some of the early devices seem clunky, I think these will be the most human & social technology platforms anyone has built yet.—Mark Zuckerberg, January 9, 2020

The big idea here is that we spend all this time looking at two-dimensional screens as a representation of reality, which dissociates us from the world around us. AR combines the real world with computer-generated elements & brings the virtual world of our phones & the real world together.

This is not just a new device, but a new human interface, the first invented since the graphical user interface was invented in the late 1970s. Multitouch replaced the mouse with a finger, but the 2D representation of a desktop has remained. A key part of the new interface will be using voice control to replace the mouse or finger.

In augmented reality, you're going to really need a pair of glasses that look like normal looking glasses in order for that to hit a mainstream acceptance. & that, I think, is going to be one of the hardest technical challenges of the decade. It's basically fitting a supercomputer in the frame of glasses.—Mark Zuckerberg, April 28, 2021

On the hardware side, Facebook is now coming at it from two directions. The Oculus Quest 2 is a standalone device built around Qualcomm’s Snapdragon platform, as most Android smartphones are. It has limited AR features, more geared towards letting developers play around with it. It’s successor, which we should see within a few months, will have more AR features, now for end users.

The Quest 2 weighs about a pound, & any vigorous use will make your face sweat because of the light seal. They look awkward, & cannot be used for long periods. It is primarily a niche product for gaming.

The other direction is the Ray-Ban Stories glasses, with far less hardware & features. They look very much like a regular pair of classic Wayfarers, except with camera hardware where the hinge rivets usually go. The strategy here is twofold. In the first place, Facebook loves data, & as I alluded to, the important data they are getting off these devices is not personal — it is usage data. They want to learn how people use these devices so they can tailor future development around that. The Quest has an installed base of about two million. This is a relatively small user base, but a very large test group.

Secondly, they want to meet in the middle, dispensing with the problems with both devices. The product they envision will look like Ray-Ban Stories, and also be a far more featured version of the Quest — less a niche VR device and more geared towards being an all day device that displaces the touchscreen smartphone sometime towards the end of the decade.

This is more or less where Facebook & Apple split in their approach. Facebook’s ambitions here are even broader than Apple’s because they are also building a metaverse. Apple is looking to transition the iPhone experience to a new device & interface. This will involve a lot of metaverse-like features, such as the soon-to-be-released SharePlay shared-experience interface. But they are not working on a full-blown virtual world like Facebook and others.

Facebook ‹› $FB ‹› is working on a long-term project — an augmented reality metaverse. The seriousness of its ambitions is reflected in its headcount. Early signals of a monumental shift towards AR & VR. Ray-Ban Stories are the first product to come out of a multi-year partnership between Facebook & EssilorLuxottica. Xiaomi has now joined them. 

Chinese smartphone giant Xiaomi Smart Glasses are an engineer’s look into an advanced future. Global technology leader, Xiaomi recently announced Xiaomi Smart Glasses – a brand new smart wearable device capable of combining imaging system and sensors into a subtle, ordinary glasses design through MicroLED optical waveguide technology. Weighing a mere 51g, Xiaomi Smart Glasses is able to display messages and notifications, make calls, navigate, capture photos, and translate text right before your eyes.

Any sufficiently advanced technology is indistinguishable from magic. Xiaomi Smart Glasses use advanced microLED optical waveguide technology to create a future viewpoint. 

To match the form of ordinary glasses, subtle challenges on imaging and internal component arrangements must be overcome. Xiaomi Smart Glasses adopts MicroLED imaging technology to lower the design space needed in structural design, as well as the overall weight of the device.

In order to fit the imaging system seamlessly into the frame, MicroLED undoubtedly is the most optimal choice. Like OLEDs, MicroLEDs pixels are individually lit – its emissive property allows for brighter displays and deeper blacks. But MicroLEDs have a higher pixel density and longer lifespan while having a simpler structure. This allows for a more compact display, as well as easier screen integration.

Xiaomi Smart Glasses has a display chip measuring just 2.4mm x 2.02mm. Under a microscope, the display is roughly the size of a grain of rice, with individual pixels sized at 4μm—enabling the display to be fitted perfectly within the frame of the glasses. To allow sufficient light to pass through complicated optical structures before reaching the eye even in harsh direct sunlight, we opted for an ultra-efficient monochrome display solution, which is capable of reaching a peak brightness of 2 million nits.

By adopting optical waveguide technology which refracts lights at 180°, the MicroLED display accurately transmits light beams to the human eye through the microscopic grating structure of the optical waveguide lens.

The grating structure etched onto the inner surface of the lens allows light to be refracted in a unique way, directing it safely into the human eye. The refraction process involves bouncing light beams countless times, allowing the human eye to see a complete image, and greatly increasing usability while wearing. All this is done inside a single lens, instead of using complicated multiples lens systems, mirrors, or half mirrors as some other products do. Optical waveguide technology significantly reduces device size and weight, and therefore allows smart glasses to approach the form factor of traditional glasses.

Not a “Secondary Smartphone Display” but an Independent Smart Wearable

Xiaomi Smart Glasses integrates a total of 497 components including miniature sensors and communication modules, so it is not just a “second screen” for your smartphone, but instead functions as a new smart terminal with independent operating capability. In addition to basic notification, call display, etc., it can also independently complete functions such as navigation, taking photos, teleprompter, and real-time text and photo translations. Considering usage, key interaction logic has also been implemented to minimize interruptions at inconvenient times, and display important information timely when critical.

We need to build an interaction logic which “does not disturb ordinary use, but displays key information in a timely manner”. After many in-depth discussions, several key scenarios including “key notification”, “phone call”, “navigation”, and “photo translation” were finally confirmed, with XiaoAi AI Assistant as the primary interaction method. In this way, Xiaomi Smart Glasses can “display the most important information at the moment it is needed”.

With “Notifications” as an example, it is obviously inappropriate to display all push notifications from your smartphone in front of your eyes. Therefore, Xiaomi Smart Glasses selects and pushes the most important messages, such as smart home alarms, urgent information from office apps, and messages from important contacts, etc., to avoid excessively disturbing users.

“Phone call” is the most natural use scenario for smart glasses, and you will be able to see the other party’s number when they call you. Using the built-in dual beamforming microphone and speaker, both parties can hear each other clearly.

“Navigation” is very useful in driving and cycling scenarios. Xiaomi Smart Glasses can present roads and maps in front of you in real time so that you can keep your eyes on the road safely and conveniently.

The 5MP camera on the front of the glasses can also take photos and translate text in photos, which is highly convenient when traveling. By utilizing the built-in microphone and proprietary translating algorithm, Xiaomi Smart Glasses is capable of transcribing audio into text with translations in real time. The indicator light next to the camera will illuminate when the camera is in-use to indicate that photos are being taken.

To achieve this high-level of functionality, the glasses need to be equipped with a quad-core ARM processor, battery, touch pad, Wi-Fi/Bluetooth modules, Android operating system, and more. Even with all these features, the device weighs only 51g. All this makes it possible for Xiaomi Smart Glasses to resemble traditional glasses form factor, but surpass other existing smart glasses in function.

Xiaomi Corporation was founded in April 2010 and listed on the Main Board of the Hong Kong Stock Exchange on July 9, 2018 (1810.HK). Xiaomi is a consumer electronics and smart manufacturing company with smartphones and smart hardware connected by an IoT platform at its core.

Embracing our vision of “Make friends with users and be the coolest company in the users’ hearts”, Xiaomi continuously pursues innovations, high-quality user experience and operational efficiency. The company relentlessly builds amazing products with honest prices to let everyone in the world enjoy a better life through innovative technology.

Xiaomi is one of the world’s leading smartphone companies. The company’s market share in terms of smartphone shipments ranked No. 2 globally in the second quarter of 2021. The company has also established the world’s leading consumer AIoT (AI+IoT) platform, with 374.5 million smart devices connected to its platform as of June 30, 2021, excluding smartphones and laptops. Xiaomi products are present in more than 100 countries and regions around the world. In August 2021, the company made the Fortune Global 500 list for the third time, ranking 338th, up 84 places compared to 2020.

Xiaomi is a constituent of the Hang Seng Index, Hang Seng China Enterprises Index, Hang Seng TECH Index and Hang Seng China 50 Index.

Global technology leader, Xiaomi recently announced Xiaomi Smart Glasses

Integrating Intelligent Functions into Eyewear

Recently, Ray-Ban Stories introduced its smart glasses in collaboration with Facebook. Xiaomi has now joined them. Early signals of Facebook's monumental shift towards AR & VR, labor & value & hard technology. Unlike on the hardware side, where their primary competition is the world’s largest company, in the metaverse, Facebook primarily faces competition from much smaller gaming companies like Roblox › $RBLX & Epic.

The Chinese technology giant's smart glasses are called Xiaomi Smart Glasses, & although they have a smart system with a number of sensors, at first glance they look like ordinary glasses. Weighing just 51g, users can view messages & notifications, make phone calls, use navigation, take photos or translate text literally right in front of their eyes & in real time.

All this is possible thanks to the advanced technology of the MicroLED optical waveguide. As with OLED panels, MicroLED pixels are illuminated individually, ensuring higher sharpness & richer blacks. Compared to OLED, however, this technology has a higher pixel density, longer life & at the same time a simpler structure, thanks to which the display is more compact & easier to integrate even into small electronic devices.

Xiaomi Smart Glasses have a display chip measuring only 2.4 mm by 2.02 mm. The display is therefore about the size of a grain of rice & the individual pixels have a size of 4 μm . The visibility of the image even in direct sunlight guarantees a maximum brightness of 2 million rivets.

Thanks to the MicroLED technology, the light is refracted at an angle of 180 ° so the display can transmit light rays to the human eye with absolute accuracy. This process involves countless reflections of light rays, thanks to which the observer sees the complete image. All this takes place inside a single lens instead of the complex systems of multiple lenses, mirrors & the like used in most similar products.

It should also be emphasized that Xiaomi Smart Glasses are not a secondary display of a smartphone, but a smart device that can be controlled & used completely independently. The novelty hides a total of 497 components, including miniature sensors & communication modules. 

Xiaomi also admits that the big challenge was not to overwhelm the user's glasses, but to display only "the most important information exactly when it is needed." It is therefore unnecessary for the user to see all notifications from the phone in front of the eyes. That's why the glasses select & send only the most important messages (eg messages from important contacts, urgent information from office applications or even information from home alarms). They primarily use XiaoAi AI Assistant.

Of course, there is also telephoning (with the owners of glasses before answering the call will see the number & contact of who is calling), navigation using the map (which will be priceless especially when cycling or driving), photography (using the front 5Mpx camera), perform text presentation in real time (thanks to the camera's capture of the surroundings) & can also transcribe audio into text , including real-time translation.

So it's a whole host of advanced features, so it's no wonder that Xiaomi glasses need to have a quad-core ARM processor, a powerful battery, a touch panel, Wi-Fi / Bluetooth modules, an Android operating system & more. However, information on price & availability is still missing. However, given what the glasses offer, we can say with almost certainty that they will not be cheap equipment.

Facebook in recent years has laid out an aggressive strategy of building its own consumer hardware to lessen its dependence on Apple & Google, makers of the two dominant mobile operating systems. Unlike on the hardware side, where their primary competition is the world’s largest company, in the metaverse, Facebook primarily faces competition from much smaller gaming companies like Roblox & Epic.

Meanwhile, one of Facebook’s other large employee groups, engineering, recently promoted David Mortenson as its new leader. He replaces Jay Parikh, who left last month after spending over a decade building the social network’s data centers & technology infrastructure. In another previously unreported move, Facebook last summer hired a former YouTube & Spotify executive, Shiva Rajaraman, to lead its budding commerce efforts—a key growth area for the company as it seeks to become more of a shopping destination. These changes, along with updates to the leadership of Facebook’s apps & the formation of its new payments group.

The seriousness of its ambitions is reflected in its headcount. Nearly 10,000 people are working in its group developing augmented & virtual reality devices, or nearly one-fifth of Facebook’s total global workforce.

The People With Power at Facebook ‹› Hardware & Commerce Ambitions Expand

Archive

Tags