Apple’s Reality At Our Finger Tips
The ‘Reality Pro’ comes more into focus…
Smoke beget fire beget an inferno. Just a few weeks after a couple reports apparently revealed a ton of details about Apple’s forthcoming ‘Reality Pro’ AR/VR/XR headset, now we seemingly have even more, once again, thanks to Mark Gurman. And these new details are worth their own play-by-play.
Here’s how it will work: The headset will have several external cameras that can analyze a user’s hands, as well as sensors within the gadget’s housing to read eyes. That allows the wearer to control the device by looking at an on-screen item — whether it’s a button, app icon or list entry — to select it.
Users will then pinch their thumb and index finger together to activate the task — without the need to hold anything. The approach differs from other headsets, which typically rely on a hand controller.
Much of the focus to date has been on the headset itself. But all of these devices need some sort of input. If it’s truly a mixture of advanced eye-tracking paired with finger gestures, this could be a legitimate breakthrough in the space. It sounds simple and that’s the point. While the Meta Quest 2 controllers are nice, they’re still controllers — something that makes using VR more like a game (even when not playing a game). That’s not likely to fly with “regular” folks. Apple’s entire history is basically creating (or re-purposing) interaction and input paradigms to move computing forward. This could be that again… if it works well, of course.
And this doesn’t even mention the notion that the eye-tracking could very well also be the authentication method for the headset. Which would also be massive, if true. Something only Apple can likely pull off, right now.
The device will have a so-called Digital Crown — like the Apple Watch — that lets users switch between VR and AR. When in VR, the wearer is fully immersed. When AR is enabled, the content fades back and becomes surrounded by the user’s real environment. Apple expects this to be a highlight of the product, according to the people, who asked not to be identified because the project is still under wraps.
You can easily envision how this would work just from Gurman’s description and if you’ve ever used not just the Apple Watch, but more so the AirPods Max, which also have a (truly awesome) digital crown for volume control. It would be like that but to dip in and out of VR, it sounds like. AR would be the “shades up” and VR the “shades down”. Interesting.
The headset’s FaceTime software will realistically render a user’s face and full body in virtual reality. Those avatars will allow two people — each with an Apple headset — to communicate and feel like they’re in the same room. The technology differs from virtual meeting rooms on Meta’s headset, which creates a more cartoon-like avatar of the user.
Because of the immense processing power necessary for the feature, the headset will only support realistic avatars during one-on-one video chats. It will still allow for FaceTime sessions with several people, but additional users will be displayed as an icon or Memoji — Apple’s customized emoji.
This sounds like something Apple could and would nail. And could be a killer feature — something Apple should have long ago done in the living room — with one big caveat: you have to know other people willing to drop $3,000 on a first generation headset. Though I suspect it would work with FaceTime for iPhone/iPad/Mac as well. But for the Reality Pro wearer, it would be like FaceTime on steroids. Perhaps not perfect or convenient for every usage (which is will remain what the iPhone/iPad version is for), but it’s one of those things that could be truly amazing for times you want to be fully immersed with a loved one virtually.
Apple expects to sell about 1 million units of its new headset in its first year. That compares with more than 200 million units for the iPhone, the Cupertino, California-based company’s biggest moneymaker. In a rare move, it’s also not planning to make a profit on the initial version — even at the high price — indicating that the company is taking a long-term view of the platform.
This is perhaps the most surprising element of the report. If Apple has ever shipped a device which has lost money on every unit sold, it has been a long time. Probably before Steve Jobs came back to the company in 1997? Certainly not a device this expensive. This would be a pretty massive break in character for Apple. Perhaps they’re okay with it given the relatively muted volume they’re aiming to sell. Or it’s just a first generation write-off to seed the market, much like the first Xbox? Still, wow (if true).
The device will also have productivity features, including the ability to serve as an external monitor for a Mac. With that feature, users will be able to see their Mac’s display in virtual reality but still control the computer with their trackpad or mouse and physical keyboard.
The headset’s operating system, internally called xrOS, will have many of the same features as an iPhone and iPad but in a 3D environment. That includes the Safari web browser, photos, mail, messages and the calendar app. And it will also have apps for the company’s services, such as the App Store to install third-party software, Apple TV+, music and podcasts. The company is working on health-tracking functions as well.
The experience should feel familiar to Apple users. When they put the headset on, the main interface will be nearly identical to that of the iPhone and iPad, featuring a home screen with a grid of icons that can be reorganized. Users will be able to pin widgets, such as the weather, calendar appointments, email and stock-market performance, among their app icons.
This mostly sounds right. It would be a bit disappointing not to have a new UI paradigm to match a new input paradigm, but perhaps Apple is trying to not overload users with too much on day one. Then again, how would an iOS UI morph into an external monitor for a Mac? Is the Mac UI an “app” on the device? Or does it auto-load if you AirPlay from your Mac? How do you do this with your headset on? Also using a mouse/keyboard while wearing the device may be another bridge to this brave new world but also sounds a bit awkward. All this makes the Reality Pro seem less like a mobile device and more of a desktop/seated device. A lot of questions here.
To that last bit:
When users need to input text, they can use the Siri voice assistant or rely on an iPhone, Mac or iPad keyboard. Unlike with an Apple Watch, though, an iPhone isn’t required for operation. The company is developing technology that will let users type in midair with their hands, but such a feature is unlikely to be ready for the initial launch.
Typing midair obviously sounds more “new school” but presents a ton of other challenges. The most surprising element here may be how little the device seems like it will be tethered to the iPhone. Few expected it would require such tethering, like the Apple Watch did at first, but still, having such a powerful device always in your pocket seems like an obvious way to extend the headsets capabilities — but instead it may rely more on extending a Mac’s capabilities? Again, a more desktop-like experience?
But making the processors powerful enough brought another concern: having the device overheat while it’s on a user’s face. To address that problem, Apple made the decision to offload the battery from inside of the headset to an external pack. It rests in a user’s pocket and connects over a cable. Another tweak is the inclusion of a cooling fan like on high-end Macs.
The headset can last about two hours per battery pack, in line with rival products. The battery, however, is large: roughly the size of two iPhone 14 Pro Maxes stacked on top of each other, or about 6 inches tall and more than half an inch thick. Still, some internal prototypes for software development have a built-in battery and charge over USB-C.
Meta, in contrast, puts its batteries on the back of its headset in a way that helps balance the device on a person’s head. Apple’s approach may ultimately be less comfortable for users, especially if they’re watching a whole movie while wearing the headset. Some testers have complained that the product can be cumbersome, according to the people.
This is starting to get a bit worrisome as it matches the earlier report about the external battery pack. This would obviously keep the headset relatively light — with the added bonus of not having a giant hot battery next to your brain — but this sounds pretty awkward. And especially if the battery life for that trade-off is only going to be two hours?!
Apple has acknowledged those challenges internally, and it’s been trying to set realistic expectations for the product. One benefit of the device, the company believes, is that it could spur customers to visit Apple retail stores — not necessarily to buy the product, but to try it out. They may then purchase another device, such as an iPad or AirPods.
To show off the new headset, Apple is creating a “store within a store” concept — an area within its retail outlets dedicated to demonstrating the product. The company did something similar when it launched the Apple Watch, which is now central to a $41 billion division.
So not only is the product a loss generator, it’s a loss leader?! It makes sense to leverage the Apple Stores as this will be a device people need to see in order to believe (or not), but this is also weird positioning for Apple. It is a bit like the initial Apple Watch, I suppose — I certainly went to a store to see one/try it on/etc. But I would rather it be more like the iPhone, where I walked into an Apple Store on day one of the device being on sale just to check it out but never expecting to buy a $600 phone — and I walked out with an iPhone. It was just so obvious that the world had changed from the moment I picked it up. That’s undoubtedly too much to ask of Reality Pro and if the above is true, Apple seems to know that.
The initial headset will be made from aluminum, glass and cushions — and be reminiscent of Apple’s $550 AirPods Max headphones. The product will have a curved screen on the front that can outwardly show a wearer’s eyes, with speakers on the sides and a headband that helps fit the device around a user’s head.
This is in line with the early renders months ago from The Information. The other element to call out in the report is the mutliple statements that this headset will indeed be $3,000 over and over again. In other words, Gurman does not believe Apple is sandbagging here — under-promising on price to over-deliver when they reveal a far cheaper price. In fact, he notes that the “cheaper” $1,500 device, which is in development, would come later. And again, per the report, Apple is planning to not profit from the $3,000 device. This thing better be pretty wild.
