Introducing Xicap - A Spatial Accessory for the Vision Pro
Portable outside-in tracking
It’s been a few months since Apple introduced active spatial accessories for the Vision Pro. Nathan Kong made a very nice presentation showing how these are made, how they are used and how they are integrated with the Vision Pro’s API. Spatial tags for the Vision Pro have existed for a while, they are simply QR code markers that get picked up by the AVP’s cameras.
Active spatial markers have the advantage that they can be tracked with higher accuracy - especially orientation - and they’re less prone to partial occlusions.
Active spatial tags in principle are similar to QR codes, they show a unique, known visual pattern defined by infrared light LEDs to the headset’s multiple cameras that allows the Vision Pro’s software to reconstruct the pose of the marker using some type of pnp algorithm.
Optical targets like these are quite common in human motion capture, out-side in tracking systems like ART have been using them almost for decades. We ourselves are using such camera systems frequently to do headset pose tracking either in multi-user setups or for in-vehicle motion capture.
What is different in case of the Vision Pro is that instead of placing cameras in static locations to define the tracking volume, the spatial accessory markers are being tracked by the cameras built into the Vision Pro. The user is taking the tracking volume with them.
Xicap spatial multi-purpose tracker
The Xicap tracker is a configurable multi-purpose spatial accessory for the Vision Pro. It is compact and has a rounded shape that can be easily attached to an object (eg. for tool tracking, as shown in the image above) or be worn on the body (eg. as game controller or motion capture device). We will offer several snap-on shells that for example allow the tracker to be attached to an object using a standard 1/4“-20 UNC screw. We will also offer straps to attach the tracker to arms, wrists or legs.
The image at the top of this post shows the Xicap tracker. To be as versatile as possible, the tracker has two push buttons on the side for users to access custom functionality. Additionally to the push buttons, the unit also has a multi-purpose IO connector to connect switches and other peripherals that could extend the functionality of the tracker - think haptic controls for custom game controllers.
Applications from tool tracking to robot teleoperation
There’s a wide range of applications for a spatial accessory like Xicap. Two areas specifically come to mind. As previously mentioned, accurately tracking tools during a critical maintenance task eg. in the defense field and therefore being able to provide enhanced guidance to maintenance personnel via the Vision Pro can have tremendous value by enhancing efficiency and safety.
As the whole world is talking about AI in the form of large language models, in San Francisco, Shenzhen and Munich, the age of physical artificial intelligence has started. Humanoid robots are being trained to perform benign household tasks, such as folding the laundry. While for training LLMs scientists just dumped the whole internet into their training clusters, gathering reliable data to train robotic models is a lot harder. Human motion capture based on trackers such as Xicap play an important role here.
Of course apart from these two evolving application areas there are no limits to what you can do with the Xicap tracker. It’s made to explore and experiment!
The rough specs
Here’s a quick look at what Xicap brings along. The first production version is still being finalized, so treat these as close-but-not-final:
| Tracking | Full 6-DoF - position and orientation - from an infrared LED constellation, with an onboard motion sensor bridging the moments the headset looks away |
| Seen by | The Vision Pro’s own cameras. No external cameras, no base stations - the tracking volume travels with you |
| Update rate | Up to 90 Hz through Apple’s spatial accessory API |
| Several at once | Every unit shows its own unique light pattern, so multiple trackers can work side by side without mixing each other up |
| Lighting | Any - bright workshop to complete darkness. The infrared light is invisible to the eye |
| Buttons | Two programmable push buttons for your own functions |
| Expansion | Multi-purpose IO to hook up switches, sensors or haptics of your own |
| Onboard memory | 16 MB of motion logging - it keeps recording even while out of the headset’s view |
| Connectivity | Bluetooth Low Energy - pairs like any standard accessory |
| Charging | USB-C, rechargeable battery |
| Size | About a wristwatch face: 52 mm across, 13 mm tall |
| Mounting | Snap-on shells (including a standard 1/4“-20 thread) and straps for wrist, arm or leg |
| Works with | Apple Vision Pro, visionOS 27 or later |
Work in progress
You probably noticed that we didn’t show any actual photos and videos of the actual Xicap unit in action yet. That is because we’re still working on finalizing the product. Right now the first PCBs of the electronics board of the unit are under production. Our designers are busy 3D printing to figure out the optimum shape for the shell, while the firmware engineers are finalizing Xicap’s MCU code. We will keep you updated on the time-line but expect the first product version to be ready at the end of October.
The Xicap tracker is available for pre-order here. You’re welcome to drop us a message to sign up for updates or with questions, ideas or feedback here.