Hand interacting with Ultraleap Leap Motion Controller

Through the Looking Glass with Hand Tracking

Looking Glass Factory’s amazing holographic displays have brought the sci-fi world into real life.  Ultraleap’s Director, Spatial Computing, Matt Tullis talks about why holograms and hand tracking are a great combination and introduces his favourite gesture-controlled holographic experiences.

In 2018, Looking Glass Factory shipped the world’s first holographic display developer kits. The technology combines light field and volumetric display technologies to generate holographic images and video.

These holograms can be experienced by a number of people from different angles at the same time, from the same display. They’re building a world in which everyone from Unity developers to LiDAR photographers to Fortune 500 companies can leave Flatland and enter the third dimension.

“We live in 3D, and in the 3D world we interact using our hands. As we leave the Flatland of 2D screens and enter the third dimension of holographic displays, hand tracking adds real value to our technology by enabling natural interaction with holograms.”
Shawn Frayne, Co-Founder and CEO - Looking Glass

The interactive element – adding hand tracking

But it’s not enough to just be able to see holograms – you need to be able to interact with them. Since the outset, hand tracking has often been the input method of choice by many Looking Glass creators.

It makes sense for this medium: in the 3D world we live in, we interact using our hands. We bring the same expectations when it comes to interacting with holograms, especially when they look real enough to touch. As it says of the Leap Motion Controller in the Looking Glass User Guide, “using the collider hands to interact with simple 3D physics never gets old.”

The holographic future

It looks as though the holographic future Looking Glass dreamed of is here to stay. In December 2020, they announced a Kickstarter Campaign for the first portrait version of their display.

Looking Glass and Ultraleap interaction with blobs
Interactive "Blobs" with the Leap Motion Controller on the new portrait Looking Glass.
Inspired by the work of Eliza SJ

The Holographic Capture and Hologram Collector bundles include a Leap Motion Controller. Their campaign blew past the initial fundraising goal, raising $1,000,000 in the first 24 hours and a total of £2,500,000!

As many more people get to discover Looking Glass holograms, here are some of our favorite gesture-controlled experiences created by Looking Glass and their community of “holo-hackers.”

Visonaries 777: Experiential marketing with hand tracking and holograms

Interactive software tool developer Visionaries 777 created an experience where attendees could see a miniature holographic version of themselves sitting in the driver’s seat of an INFINITI vehicle.

At the North American International Auto Show in 2019, attendees could explore the car, venture inside the spacious interior, and pop the hood for a peek at the engine.

Visionary 777 car interaction using looking glass and ultraleap hand tracking 1
Visionaries 777 created an interactive concept car experience
“The Looking Glass allowed us to display interactive and immersive visualizations of INFINITI cars and branding identity. Due to its holographic effect, it was very eye-catching but, more importantly increased the immersion and allowed the user to almost feel the car. With [Ultraleap hand tracking], users could interact without the need for controllers, making interactions seamless and natural.”
Frantz Lasorne, Co-Founder - Visionaries 777

Intel: Next-generation retail experiences

At NRF in early 2020, Intel used the Looking Glass 8K immersive display to showcase the next generation of retail experiences. Powered by Ultraleap hand tracking, visitors were able to reach out with their bare hands and customize a 3D car. In a world where consumers are concerned about contactless interactions, interactive displays like this open up new possibilities.

Intel interactive experience using looking glass and ultraleap hand tracking
Using Ultraleap hand tracking to engage with a Looking Glass display in Intel’s booth at NRF 2020 

Experimental artists: Holographic pets

Who wouldn’t want a cute 3D friend to play with? Hand tracking lets you care for, play with and stroke holographic characters. The affectionate interaction makes it hard not to think of them as a pet and develop an emotional connection.

Interactive pet Froggo using looking glass and ultraleap hand tracking
Froggo by experience artist Oliver Garcia-Borg

Live interaction with a holo-portrait

And finally, a bit of fun with your own face! It’s possible to manipulate your live holo-portrait with iPhone’s TrueDepth Camera and a Leap Motion Controller. Find out from Albert Hwang, UX designer at Looking Glass Factory how it’s done.

Interactive holographic portrait using looking glass and ultraleap hand tracking
Manipulation of a live holo-portrait from Albert Hwang at Looking Glass Factory

Developing with Looking Glass and hand tracking

To find out more about developing with hand tracking and Looking Glass, check out the Looking Glass User Guide for integration guides, tips, and best practice. Our developer site also has a quick setup guide for Unity users as well as other useful documentation.