Ahead of CES, Ultraleap announced today that it’s working with Prophesee to develop the world’s first system using event sensors and hand tracking in augmented reality (AR) devices. This announcement from Ultraleap and Prophesee demonstrates a solution to the unsolved challenge of AR devices that are always on, worn everywhere and require a higher level of interaction.
It leans on Ultraleap’s advanced computer vision and machine learning models together with Prophesee’s breakthrough GenX320 event-based Metavision® sensors, to tackle some of the largest challenges of augmented reality. Power consumption is a fraction of what’s used in existing models, and it enables an always-on AR device that can track user’s hand interactions natively for longer. The new Ultraleap models for event sensors are also more robust to adverse light conditions and have lower latency, empowering a new era of outdoor interaction capabilities. All of these features are provided with increased privacy as event sensors are not frame-based and capture only pixel movement information.
The GenX320 sensors from Prophesee were the first on the market to deliver ‘always on’ capabilities, making them the perfect candidate for Ultraleap to use in their first event-based machine learning vision system for hand tracking in AR. Together, these two technologies will enable robust and immersive interaction methods in a convenient glasses form factor that is required for market adoption in augmented reality.
Tom Carter, CEO and co-founder at Ultraleap said: “No product currently exists in the market that leverages event sensors and hand tracking for AR. Hand interaction adds impressive value to augmented reality, not only by providing the most natural way to interact, but by expanding functionalities and making the control system more ergonomic. Low power consumption, a high dynamic range, low latency and increased privacy are all critical features to reach the glasses form factor needed for mass adoption. These are the challenges that the Ultraleap and Prophesee teams are addressing with this new product.”
Luca Verre, CEO and co-founder at Prophesee said: “The integration of advanced hand tracking features is a pivotal step towards more immersive and interactive experiences in AR. Event-based vision has the proven ability to add more performance and usability improvements to the current state of AR systems which will expand the market potential and capabilities of this application. Our partnership with Ultraleap enhances the overall user experience, unlocking a new era of AR/VR devices that seamlessly blend the virtual and physical worlds, offering unparalleled levels of engagement for entertainment and productivity.”
The initial stage of this AR event-based vision product can be experienced at CES 2024, by appointment only. Please request a meeting through Ultraleap here.
