Leap Motion Co-Founders Talk ‘Dragonfly’ Made-for-VR Motion Input Camera

Ben from Road to VR interviews David and Michael from Leap about their demonstrations at CES 2015. They showcase their Dragonfly prototype, which features greater than HD resolution with RGB and infrared pixels, offering VR, AR, and hybrid experiences. They also demonstrate new UI features, allowing users to interact with holographic interfaces for complex settings adjustments and pass-through toggling.

Michael from Leap explains the company’s strategic shift towards VR, driven by the need for new input methods in virtual environments. With growing developer excitement and energy around VR, Leap sees an opportunity to define the interaction paradigm with new devices like Dragonfly. Their goal is to integrate hand tracking directly into VR headsets to reduce costs and enhance user experience.

David and Michael discuss the advantages of embedding Leap technology, like Dragonfly, directly into VR headsets. Embedding reduces costs and logistical issues compared to selling peripherals. It also provides developers with a broader user base, ensuring hand-tracking technology reaches more consumers. They highlight the importance of having hand tracking embedded into consumer devices to streamline development and user interaction.

Dragonfly is a prototype reflecting ongoing experiments, much like Oculus’ Crescent Bay. While hardware development continues, Michael emphasizes that Leap is more software-driven, with regular updates improving functionality. The hardware evolution focuses on frame rates, field of view, and color resolution, but future advancements will largely be driven by software innovations to enhance user experience and interactivity.

The addition of RGB cameras in the Dragonfly prototype enhances the realism of mixed-reality experiences. While infrared can detect objects like before, RGB brings color to the virtual world, making it feel more immersive. This encourages deeper blending of the real and virtual worlds. The potential to control transparency settings and integrate real-world objects into virtual settings is also explored.

Dragonfly’s wider field of view and human vision component distinguish it from previous Leap peripherals. The expanded inter-pupillary distance (IPD) enables better long-range tracking, solving limitations of the original Leap sensor. The prototype is built to handle complex machine vision tasks that adapt better to VR's unique demands, particularly for larger and more interactive virtual environments.

Michael discusses the challenges of hand tracking in VR, especially when users look away from their hands. Solutions like wider field-of-view sensors and memory APIs that recall hand positions when out of sight are being explored. These innovations will ensure seamless tracking in more dynamic VR interactions, supporting natural movement and object manipulation in virtual environments.

Leap’s team introduces a swipe gesture that smoothly transitions between the real world and virtual environments. This gesture-based transition, recently released as open-source, improves user experience by aligning movements with visual changes, making the interaction more intuitive. The UX team spent considerable time ensuring gestures were responsive and the transitions were not jarring, focusing on depth perception and movement cohesion.

Leap plans to release more information about Dragonfly’s integration into partner devices in the coming months. They emphasize that Dragonfly is just the beginning, and future advancements will focus on software. With constant software updates improving tracking and interaction, Dragonfly sets the stage for more significant technological leaps in VR and AR development.