Apple Staffers Explain the Challenges in the Development of the Mixed Reality Headset

BY Chandraveer Mathur

Published 17 May 2022

VR Headset Oculus Unsplash

Since development efforts commenced in 2015, Apple’s mixed reality headset has faced numerous challenges and difficulties. In a detailed report cataloging them, The Information shares insights and the perspective of its sources inside Apple. It also chronicles the development journey besieged by “technical and leadership challenges.”

The publication’s report is based on interactions with over 10 people closely involved in developing Apple’s mixed reality headset. Most of the sources were a part of the project’s core team. Present-day rumors suggest the headset should debut by the end of 2023 with a hefty price tag and plenty of onboard processing power.

Starting with the mention of a conference in 2016, the sources say Apple’s board was shown very early prototypes of what could become future projects.

“Former Vice President Al Gore, then–Disney CEO Bob Iger and other Apple board members walked from room to room, trying out prototype augmented and virtual reality devices and software. One of the gadgets made a tiny digital rhinoceros appear on a table in the room. The creature then grew into a life-size version of itself, according to two people familiar with the meeting. In the same demo, the drab surroundings of the room transformed into a lush forest, showing how users could seamlessly transition from AR, in which they can still view the physical world around them, to the more immersive experience of VR—a combination known as mixed reality.”

At the time, some prototypes used existing AR/VR hardware such as HTC Vives. Others were so heavy they required additional support structures or a “small crane” so the board members could preview it “without straining their necks.” On the software front, most implementations ran Microsoft Windows. Although the descriptions of the prototypes suggest they were unconvincing and cumbersome to use, the iPhone maker decided to pour funds into the development efforts.

One of the major roadblocks the report chronicles is the company’s CEO, Tim Cook. The executive allegedly “supports the headset project,” but five staffers say he hasn’t been “as active in the effort as Jobs was with the iPhone’s development.”

“For example, he rarely visits the group at its offices away from the main Apple campus, those people said. The lack of a honcho of Cook’s stature to champion the headset, code-named N301, has made it harder at times for it to compete with other products such as the Mac and iPhone for head count and engineering resources, the people said.”

Reputed Apple executive Jony Ive was also a challenge. When the mixed reality headset team’s leader and founder, Mike Rockwell, sought assistance from other teams in the company, Ive’s team doubted that consumers won’t prefer wearing headsets for long durations.

“Rockwell, Meier and Rothkopf soon encountered pushback from Ive’s team. The three men had initially wanted to build a VR headset, but Ive’s group had concerns about the technology, said three people who worked on the project. They believed VR alienated users from other people by cutting them off from the outside world, made users look unfashionable and lacked practical uses. Apple’s industrial designers were unconvinced that consumers would be willing to wear headsets for long periods of time, two of the people said.”

This, subsequently, led to the creation of Apple’s mixed reality headset.

“The men came up with a solution to address the concerns of Ive’s team. For example, they proposed adding cameras to the front of the headset so that people wearing the device could see their surroundings, said the three people. But the feature that ultimately sold the industrial designers on the project was a concept for an outward-facing screen on the headset. The screen could display video images of the eyes and facial expressions of the person wearing the headset to other people in the room.”

“These features addressed the industrial design group’s worries about VR-induced alienation—they allowed other people in a room to interact and collaborate with a person wearing a headset in a way not possible with other VR gear. For years, the existence of such a display, internally code-named T429, was known only to a small circle of people even within Rockwell’s group.”

The Information says this is the first of a two-part report. The second will describe the “pivotal moment” for Apple’s headset in 2019.

[Via The Information]