<img height="1" width="1" style="display:none" alt="" src="https://analytics.twitter.com/i/adsct?txn_id=o7u1o&p_id=Twitter&tw_sale_amount=0&tw_order_quantity=0" /> <img height="1" width="1" style="display:none" alt="" src="//t.co/i/adsct?txn_id=o7u1o&p_id=Twitter&tw_sale_amount=0&tw_order_quantity=0" />
Lufthansa VR flight training with ULtraleap hand tracking technology

VR Flight Training: Lufthansa Aviation Training & NMY

Posted; February 17, 2022

Lufthansa Aviation Training’s VR cabin crew program was the first VR training program approved by the German Aviation Authority (Luftfahrt Bundesamt). Developed by NMY Mixed Reality Studio, since its introduction in 2019 around 20,000 Lufthansa flight attendants have used it to complete a part of their annual training. Hand tracking was essential to this innovative and effective immersive learning program. 

“VR training for flight attendants is a huge cost saver. But without hand tracking it would have been impossible to create – we would not have been able to get it approved by the German Aviation Authority. Using hand tracking instead of controllers was an indispensable part of developing an effective, authorized VR training program for cabin crew.”
Björn Eckermann, Referent Training Operations, Safety & Service Training - Lufthansa Aviation Training


In-person cabin crew training programs are enormously expensive in both time and money. Courses require physical dummy aircraft, or operational aircraft being temporarily grounded to allow training to take place.

Lufthansa Aviation Training wanted to develop a VR training program that could substitute part of this. Their focus initially was on the mandatory refresher training for security searches. These are searches flight attendants perform after every flight. They ensure no suspicious or potentially dangerous items have been left on board.

Lufthansa’s brief to NMY Mixed Reality Studio was very clear. This enterprise use of VR could only be successful with hand tracking, instead of gaming controllers.

Picture credit: NMY Mixed Reality Studio

The 4 reasons Lufthansa insisted on hand tracking

  1. Regulation: The German Aviation Authority required flight attendants to be able to interact naturally.
  2. Practicality: To search an aircraft flight attendants need to go down on their hands and knees to search under seats. This made controllers impractical.
  3. Ease of use: VR controllers would be too hard to explain within the timescale (the VR training takes, in total, just 30 minutes). Conversely, it’s very easy to explain to someone how to use their hands.
  4. Confidence and positivity: An important factor in training success is having a positive experience. The demographic of flight attendants is highly varied (18-70 year olds), generally inexperienced with VR, and largely non-gamers. Some participants don’t even have a smartphone. Hand tracking lowers the barriers to entry and improves confidence.

Selecting Ultraleap hand tracking

NMY selected Ultraleap hand tracking because of its reliability and flexibility. Unlike some other hand tracking solutions, Ultraleap hand tracking isn’t tied to any specific headset range.

Ultraleap hand tracking cameras range from headsets with fully integrated tracking through to peripherals. This meant NMY could easily integrate hand tracking with their preferred HTC Vive headset.

Lufthansa cabin crew member using Ultraleap hand tracking for VR Training

Ultraleap technology used

  • Core hand tracking software
  • Unity plugin
  • Leap Motion Controller camera attached to HTC Vive headset using custom clip

Interaction design for VR flight training

Participants must go through the VR flight training in just 30 minutes. NMY rapidly discovered that users had no time to learn complex gestures or intricate interactions.

What NMY realized, however, was that this didn’t matter. Counter-intuitively, while a broad similarity to the way flight attendants interact naturally in a real aircraft was essential, exact fidelity was unimportant.

A flight attendant may need to pick up a cushion to search under it. However, they can pick up the cushion any way they like. What they are learning is not how to pick up a cushion, but the correct process to search an aircraft effectively and deal with a potentially dangerous object.

Video credit: NMY Mixed Reality Studio

With this in mind, NMY implemented two important innovations behind the scenes:

  1. Created a large invisible box (“bounding box” in technical terms) surrounding each object.
  2. Reduced the complexity of the interactions by only requiring a simple grabbing motion when a user’s hand intercepts this box.
“Our client’s brief was very clear: VR controllers were not fit for purpose for VR flight training. Developing with hand tracking made it possible for us to deliver a VR experience that went beyond gaming. We successfully solved a real business need for an established, traditional industry.”
Sebastian Demmerle, CEO and Founder - NMY

In practice this means that attendants can “pick up” a cushion by simply putting their hand in the general area. The cushion sticks to their hand. They can move quickly through the experience and focus on the order of the security search – not on learning VR interactions ultimately irrelevant to the purpose of the training.

Picture credit: NMY Mixed Reality Studio

NMY also used a simple virtual wearable on the user’s left wrist. The virtual watch tells them how much time they have left. When they turn over their hand, they are provided with an itemized list of tasks to complete.

Virtual Reality images of inside a Lufthansa airplane

Key features of Lufthansa VR flight training

  • 5-6 training sessions per day
  • 9 participants per session
  • 30-minute session
  • Participants are supported by a virtual assistant and human call operator (one operator for all 9 participants)
  • 100-200 hand interactions per session

The first VR flight training approved by the German Aviation Authority

20,000 icon

20,000 flight attendants have used the VR program

5-6 icon

5-6 training sessions per day

9 icon

9 participants per session

With real lives at stake, the German Aviation Authority is rightly cautious when it comes to approving innovative training programs. This makes the fact that Lufthansa Aviation’s training was the first VR flight training program to have been approved a huge achievement and a big step forward in immersive training.

Since its introduction in 2019 around 20,000 Lufthansa flight attendants have used the VR program to complete a part of their annual training. VR flight training clearly demonstrated its efficiency and cost-effectiveness in a mainstream, highly safety-conscious industry. None of this would have been possible without hand tracking. 

Lufthansa VR flight training with ULtraleap hand tracking technology

The world’s best hand tracking

Ultraleap’s robust hand tracking enables transformational VR training experiences.

VR Training with Ultraleap