Apple Vision Pro: Everything You Need to Know

Introduction

In this video, the speaker discusses Apple's rumored entry into the virtual reality headset market. The speaker provides insights on what is known about the headset and speculates on its potential impact.

Virtual Reality Market

• Virtual reality headsets have been around for 8 years but have not gained widespread adoption.

• VR headsets are expensive and have a limited software catalog.

• Apple is rumored to be entering the VR headset market in 2023 with a headset that will be many times more expensive than existing headsets.

Design of Apple's Reality Pro Headset

This section focuses on the design of Apple's rumored Reality Pro headset.

Exterior Design

• The headset is expected to resemble modern VR goggles like the Meta Quest Pro.

• It will look similar to AirPods Max with an aluminum, glass, and mesh cushion construction.

• The headband is alleged to house speakers.

• There will be a curved display on the outside of the headset that can show a wearer's eyes to those not using the headset.

Internal Design

• Much of the internal workings are rumored to be constructed of carbon fiber and lightweight polypropylene.

• Built-in eye tracking will allow for foveated rendering which may be necessary due to two Sony OLED MicroDisplays - one per eye - at 4K each.

• The displays use OLED on Silicon technology which allows them to put CMOS circuits right on silicon substrates giving them the ability to drive smaller, brighter pixels.

Display Resolution and Refresh Rate

This section focuses on display resolution and refresh rate for Apple's rumored Reality Pro headset.

Display Resolution

• The Reality Pro's displays are projected to use a pixel size as tiny as 4 microns - 10 times smaller than a high-density smartphone display.

• The headset is targeting 4K-per-eye resolution which is very high but not unprecedented.

Refresh Rate

• A sufficiently high pixels-per-degree value is needed to get rid of the screen door effect - the visible black gaps in between individual pixels.

• PS VR2 uses OLED with a high enough pixel density and response time to best out any LCD-based headset, but retains the fresnel lens technology used in VR headsets before it—something Apple is boldly moving away from.

Conclusion

In this section, the speaker concludes by summarizing what we know about Apple's rumored Reality Pro headset and speculating on its potential impact.

Summary

• Apple's Reality Pro headset will be many times more expensive than existing headsets.

• It will resemble modern VR goggles like the Meta Quest Pro and have a curved display on the outside that can show a wearer's eyes to those not using the headset.

• The internal workings are rumored to be constructed of carbon fiber and lightweight polypropylene with built-in eye tracking for foveated rendering due to two Sony OLED MicroDisplays - one per eye - at 4K each.

• The displays use OLED on Silicon technology which allows them to put CMOS circuits right on silicon substrates giving them the ability to drive smaller, brighter pixels.

Potential Impact

• The speaker speculates that if Apple enters the VR market, it could disrupt it as they have done with other markets in the past.

• However, given that VR has not gained widespread adoption despite being around for 8 years, it remains to be seen if Apple's entry into this market will be successful.

Apple's VR Headset: What We Know So Far

This transcript discusses the features and components of Apple's upcoming VR headset.

Pancake Optics

• The headset will use expensive pancake optics with many folding lenses on top of each other in a curve.

• These optics make the headset much lighter and thinner than conventional VR headsets, but they have their own flaws.

• Pancakes offer a wider sweet spot, no chromatic aberration, and no lens distortion correction processing for your GPU to deal with. However, they don't offer as wide a field-of-view as some other headsets.

Display

• The headset is expected to have a 120-degree field-of-view which is better than the PS VR2 and Meta Quest 2 but not nearly as good as something like the Valve Index or Pimax 5K.

• The pancake lenses require significantly brighter displays to get the same perceived brightness value compared to fresnels.

• It's very possible that the displays in this thing would be pushing $1,000 wholesale.

Components

• The first model will ship with an M2 chip and a secondary ISP to combine camera and sensor data for tracking and also to handle mixed-reality camera distortion.

• An M2 is quite powerful, perhaps too powerful given how warm it can get after just a few minutes of playtime on other headsets like Quest 2.

• Apparently, Apple has moved on from the idea of a self-contained headset to one where at least the battery pack - but perhaps even the SoC - will be external to the device with a cable running from your pocket or belt up to the headset itself. The battery pack is said to be large - about the size of two iPhone 14 Pro Maxes stacked one on top of the other - and even a battery as large as that is only expected to power the headset for about two hours.

Sensor Suite

• Eight camera modules will be used for see-through mixed reality, and six of those modules would be used for "innovative biometrics."

• All body movement will be at least somewhat tracked - legs, arms, and even facial expressions. Everything will be done with hand tracking, and there will not be any physical controllers.

• Two LIDAR modules - one short and one long-range - will help with movement tracking and also room mapping. There were allegedly sensors to track eyebrow and jaw movement but those have been replaced with machine learning. In addition to eye position tracking it has been said that there will also be an iris scanner for biometrics—Face ID.

Mixed Reality Mode

In this section, the speaker discusses Apple's rumored mixed reality mode and how it will work.

Passthrough Cameras

• Users can transition to a mixed reality mode where they can see their surrounding environment using passthrough cameras.

• The software library is expected to make up for the lack of transparency in lenses.

• To operate the device, users will direct their gaze towards an item displayed on the screen and trigger it with a pinch of their thumb and index finger.

Operating System

• The internal name for the operating system is xrOS, which is a fork of iOS.

• It is said to look "nearly identical" to that of an iPhone with a grid of app icons on a springboard.

• Rumors suggest that existing iOS apps can be run in 2D in VR.

FaceTime

• One of the big reveal demos will be FaceTime.

• The M2 will realistically render a user's face and full body in VR, allowing video conferencing with another Reality Pro user in a virtual room.

• Due to immense processing power required, it'll only work initially for one-on-one FaceTime calls.

Video Consumption

In this section, the speaker discusses Apple's focus on video consumption for its upcoming headset.

Content Development

• Apple has worked with Disney, Dolby, and others to develop VR-specific content for the headset.

• It is rumored that Apple will update its own Apple TV+ material to work with the headset.

Gaming

• Gaming is expected to be a big part of this headset's value proposition.

• Cupertino has allegedly created an underlying engine to power VR-specific titles.

External Monitor

• The rumored ability to use the headset as an "external monitor" for a Mac could be a potential game-changer for several fields.

• Being able to see one huge virtual display or a triple monitor setup from your airline seat could be useful if passthrough works well.

Conclusion

In this section, the speaker concludes by summarizing what we know about Apple's upcoming mixed reality headset based on rumors and leaks.

• The device is expected to have passthrough cameras and run on xrOS, which is a fork of iOS.

• FaceTime will be one of the big reveal demos, allowing users to video conference in VR with another Reality Pro user in a virtual room.

• Video consumption and gaming are also expected to be key features of the device.

• The ability to use the headset as an external monitor for a Mac could also be a game-changer in certain fields.

Apple's Reality Pro Headset

This transcript discusses the development and potential release of Apple's Reality Pro headset, a departure from their typical product line.

The Project's Background

• The project has been underway for many years, with leaks and rumors dating back to 2017.

• The headset seems un-Apple-like as it ventures into an underdeveloped market that has consistently failed to capture a wide audience.

• Tim Cook, CEO of Apple, is viewed by Wall Street as one of the most successful CEOs in history. He and Jeff Williams are pushing hard for the release of the Reality Pro this year despite design team concerns that more time is needed.

Potential Impact

• While Cook may not care about creating a legacy like Steve Jobs did, this product could be a huge departure from Apple's typical products.

• Gurman suggests an eye-wateringly expensive $3,000 retail price for the Reality Pro headset. Even at that price point, it will not meet Apple’s typical gross margin expectations and will exceed its unit cost.

• Shipping units now allows programmers, enthusiasts, and others to start developing with this headset that’ll instill desire in regular consumers once Apple can finally produce a lower-end, more cost-effective headset in the future.

Conclusion

• Legacy can’t be built without risk. This is the first time we’ve seen Tim Cook acting like Steve Jobs and taking such risks.