Through the Looking Glass with Hand Tracking

Posted; February 4, 2021

Looking Glass Factory’s amazing holographic displays have brought the sci-fi world into real life.  Ultraleap’s Director, Spatial Computing, Matt Tullis talks about why holograms and hand tracking are a great combination and introduces his favourite gesture-controlled holographic experiences.

In 2018, Looking Glass Factory shipped the world’s first holographic display developer kits. The technology combines light field and volumetric display technologies to generate holographic images and video.

These holograms can be experienced by a number of people from different angles at the same time, from the same display. They’re building a world in which everyone from Unity developers to LiDAR photographers to Fortune 500 companies can leave Flatland and enter the third dimension.

But it’s not enough to just be able to see holograms – you need to be able to interact with them. Since the outset, hand tracking has often been the input method of choice by many Looking Glass creators.

It makes sense for this medium: in the 3D world we live in, we interact using our hands. We bring the same expectations when it comes to interacting with holograms, especially when they look real enough to touch. As it says of the Leap Motion Controller in the Looking Glass User Guide, “using the collider hands to interact with simple 3D physics never gets old.”

It looks as though the holographic future Looking Glass dreamed of is here to stay. In December 2020, they announced a Kickstarter Campaign for the first portrait version of their display.

The Holographic Capture and Hologram Collector bundles include a Leap Motion Controller. Their campaign blew past the initial fundraising goal, raising $1,000,000 in the first 24 hours and a total of £2,500,000!

As many more people get to discover Looking Glass holograms, here are some of our favorite gesture-controlled experiences created by Looking Glass and their community of “holo-hackers.”

Interactive software tool developer Visionaries 777 created an experience where attendees could see a miniature holographic version of themselves sitting in the driver’s seat of an INFINITI vehicle.

At the North American International Auto Show in 2019, attendees could explore the car, venture inside the spacious interior, and pop the hood for a peek at the engine.

At NRF in early 2020, Intel used the Looking Glass 8K immersive display to showcase the next generation of retail experiences. Powered by Ultraleap hand tracking, visitors were able to reach out with their bare hands and customize a 3D car. In a world where consumers are concerned about contactless interactions, interactive displays like this open up new possibilities.

Who wouldn’t want a cute 3D friend to play with? Hand tracking lets you care for, play with and stroke holographic characters. The affectionate interaction makes it hard not to think of them as a pet and develop an emotional connection.

And finally, a bit of fun with your own face! It’s possible to manipulate your live holo-portrait with iPhone’s TrueDepth Camera and a Leap Motion Controller. Find out from Albert Hwang, UX designer at Looking Glass Factory how it’s done.

To find out more about developing with hand tracking and Looking Glass, check out the Looking Glass User Guide for integration guides, tips, and best practice. Our developer site also has a quick setup guide for Unity users as well as other useful documentation.