Apple Mixed Reality Headset Won’t Focus on Gaming; Jony Ive Remains Involved in Development: Report

BY Chandraveer Mathur

Published 20 May 2022

VR headset Unsplash

Earlier this week, The Information shared a report cataloging the difficulties Apple engineers and designers faced while developing the rumored mixed reality headset. The publication has shared more exciting details about the headset’s development journey in a follow-up to that report.

The report picks up from 2019 when Apple had been developing the headset for three years. At the time, one of the biggest unanswered questions was: Could the mixed reality headset work as a standalone device, or would it need a “powerful base station” to work as Apple expects it to.

“The headset that worked with a base station had superior graphics, including photorealistic avatars, while the stand-alone version depicted its avatars more like cartoon characters. Mike Rockwell, the Apple vice president in charge of the company’s AR/VR team, favored the headset with the base station, believing that Apple’s top brass wouldn’t accept the stand-alone version’s lower-quality visuals, according to the two people. He was wrong. Ive had pushed for the stand-alone version of the headset since the early days of the project, according to a person familiar with it.”

Although Ive left Apple in 2019, the report claims that he remains involved in developing the headset as a consultant.

“One person familiar with the matter said Ive’s consulting work for Apple since he left includes the headset, adding that he is often brought in to help his former team push through their preferences in areas such as battery, camera placement and ergonomics over those of engineers. Two people said even after Ive left Apple, some employees on the headset project were still required to make the trek from Cupertino to San Francisco, where Ive has a home, to get his approval on changes.”

“Ive has continued to tweak the headset’s design. While earlier prototypes had the battery in the headband, he prefers a design that would tether the headset to a battery the user wears, similar to Magic Leap’s headset design. It couldn’t be learned if this approach will make it into the final design.”

Another interesting detail revealed in the report is the use of up to 14 cameras to create lifelike virtual avatars that also replicate facial expressions.

“Other challenges, such as incorporating 14 cameras on the headset, have caused headaches for hardware and algorithm engineers. The cameras include those that will track the user’s face to ensure virtual avatars accurately represent their expressions and mouth movements, a marquee feature.”

Previous rumors cemented expectations that Apple’s mixed reality headset will focus on gaming, communication, and content consumption. However, The Information’s sources claim gaming isn’t one of the intended use cases for Apple’s headset. Considering that gaming has been a key element in the success of existing commercial-grade headsets such as Oculus products, the lack of focus on the use case was reportedly criticized internally at Apple.

“Four people who have worked on the project also criticized its lack of focus on gaming, a category of software that appeals to early adopters, which was important to the success of the iPhone and has been a big priority for Meta’s VR group. Those people said Rockwell’s group almost never mentioned games in internal presentations about possible uses for the headset. Apple isn’t developing game controllers for the device and is aiming to use hand tracking or in combination with a clothespin-like finger clip as inputs for the device, multiple people familiar with the project say.”

If you have the time (and a subscription), we suggest you take a look at the full report on The Information. Meanwhile, a separate report this week mentioned that Apple’s board convened to check on the headset project recently, suggesting that it could launch in the near future.

[Via The Information]