VR Flight Training: Lufthansa Aviation Training & NMY
Posted; February 17, 2022
Lufthansa Aviation Training’s VR cabin crew program was the first VR training program approved by the German Aviation Authority (Luftfahrt Bundesamt). Developed by NMY Mixed Reality Studio, since its introduction in 2019 around 20,000 Lufthansa flight attendants have used it to complete a part of their annual training. Hand tracking was essential to this innovative and effective immersive learning program.
In-person cabin crew training programs are enormously expensive in both time and money. Courses require physical dummy aircraft, or operational aircraft being temporarily grounded to allow training to take place.
Lufthansa Aviation Training wanted to develop a VR training program that could substitute part of this. Their focus initially was on the mandatory refresher training for security searches. These are searches flight attendants perform after every flight. They ensure no suspicious or potentially dangerous items have been left on board.
Lufthansa’s brief to NMY Mixed Reality Studio was very clear. This enterprise use of VR could only be successful with hand tracking, instead of gaming controllers.
The 4 reasons Lufthansa insisted on hand tracking
- Regulation: The German Aviation Authority required flight attendants to be able to interact naturally.
- Practicality: To search an aircraft flight attendants need to go down on their hands and knees to search under seats. This made controllers impractical.
- Ease of use: VR controllers would be too hard to explain within the timescale (the VR training takes, in total, just 30 minutes). Conversely, it’s very easy to explain to someone how to use their hands.
- Confidence and positivity: An important factor in training success is having a positive experience. The demographic of flight attendants is highly varied (18-70 year olds), generally inexperienced with VR, and largely non-gamers. Some participants don’t even have a smartphone. Hand tracking lowers the barriers to entry and improves confidence.
Selecting Ultraleap hand tracking
NMY selected Ultraleap hand tracking because of its reliability and flexibility. Unlike some other hand tracking solutions, Ultraleap hand tracking isn’t tied to any specific headset range.
Ultraleap hand tracking cameras range from headsets with fully integrated tracking through to peripherals. This meant NMY could easily integrate hand tracking with their preferred HTC Vive headset.
Ultraleap technology used
- Core hand tracking software
- Unity plugin
- Leap Motion Controller camera attached to HTC Vive headset using custom clip
Interaction design for VR flight training
Participants must go through the VR flight training in just 30 minutes. NMY rapidly discovered that users had no time to learn complex gestures or intricate interactions.
What NMY realized, however, was that this didn’t matter. Counter-intuitively, while a broad similarity to the way flight attendants interact naturally in a real aircraft was essential, exact fidelity was unimportant.
A flight attendant may need to pick up a cushion to search under it. However, they can pick up the cushion any way they like. What they are learning is not how to pick up a cushion, but the correct process to search an aircraft effectively and deal with a potentially dangerous object.
With this in mind, NMY implemented two important innovations behind the scenes:
- Created a large invisible box (“bounding box” in technical terms) surrounding each object.
- Reduced the complexity of the interactions by only requiring a simple grabbing motion when a user’s hand intercepts this box.
In practice this means that attendants can “pick up” a cushion by simply putting their hand in the general area. The cushion sticks to their hand. They can move quickly through the experience and focus on the order of the security search – not on learning VR interactions ultimately irrelevant to the purpose of the training.
NMY also used a simple virtual wearable on the user’s left wrist. The virtual watch tells them how much time they have left. When they turn over their hand, they are provided with an itemized list of tasks to complete.
Key features of Lufthansa VR flight training
- 5-6 training sessions per day
- 9 participants per session
- 30-minute session
- Participants are supported by a virtual assistant and human call operator (one operator for all 9 participants)
- 100-200 hand interactions per session
The first VR flight training approved by the German Aviation Authority
20,000 flight attendants have used the VR program
5-6 training sessions per day
9 participants per session
With real lives at stake, the German Aviation Authority is rightly cautious when it comes to approving innovative training programs. This makes the fact that Lufthansa Aviation’s training was the first VR flight training program to have been approved a huge achievement and a big step forward in immersive training.
Since its introduction in 2019 around 20,000 Lufthansa flight attendants have used the VR program to complete a part of their annual training. VR flight training clearly demonstrated its efficiency and cost-effectiveness in a mainstream, highly safety-conscious industry. None of this would have been possible without hand tracking.
The world’s best hand tracking
Ultraleap’s robust hand tracking enables transformational VR training experiences.VR Training with Ultraleap
Explore our blogs, whitepapers and case studies to find out more.