News Avionics
Munich university developing AR helmet display for ship, ocean platform landings

Researchers at the Technical University of Munich are developing an assistance system for helicopter landing approaches on ships, even in adverse weather conditions. Based on augmented reality, the system provides key information in the visor of a pilot’s helmet-mounted display.

“On the open seas, there are generally few points of visual reference to guide pilots,” said Tim Mehling, a doctoral candidate at chair at the university’s Institute of Helicopter Technology. “Pilots are, therefore, frequently forced to switch their focus from their landing target – the ship’s deck – to glance at the cockpit instrument panel.”

The group behind the new augmented reality (AR) system explains, that as a pilot aims for the helicopter landing platform on the deck, a ship can heave, pitch and roll violently in a storm, with visibility made worse by the sea spray and dense cloud cover. The team also notes a helicopter must maintain a safe distance from the superstructures on the upper deck of the ship to avoid contact with the aircraft’s rotor blades, but also needs to be far enough from the edge of the deck to avoid the overhanging fuselage pulling it backward into the sea.

The researchers have modelled this challenging landing environment by using the helicopter simulator at the Institute of Helicopter Technology. Their goal is to develop solutions to make it safer for pilots to execute approaches and landings on ships and ocean platforms – even in bad weather with high seas and low visibility.

“Even the best pilot’s concentration will slip after the 20th approach attempt,” says Mehling, noting many attempts at one landing may be needed depending on the conditions. As a result, Mehling is creating a helmet-mounted, real-time display that presents all relevant information from the helicopter and the ship – using graphics that are superimposed on the pilot’s view outside the cockpit.

Mehling began by developing a highly realistic simulation of the environment at sea. At the same time, he worked with experienced navy test pilots and applied national and international standards to determine what information pilots need for a safe approach. “We developed appropriate icons for every phase of the landing,” he said. “The coordinated icons are automatically displayed in the visor during the corresponding flight phase.”

The team behind the technology explains the helmet displays a visual attitude indicator as well as key helicopter parameters. The software can also use imaging data to automatically detect the ship’s deck and visualize the ideal approach, which resembles what Mehling describes as a “3D slide” in the display. “If visibility is lacking due to bad weather, the pilot can glide down this stable virtual slide to make a safe landing,” he explained.

Four test pilots with experience in seaborne operations with the German military and in the private sector have tested the pilot assistance system. In the next stage of development, it will be tested by pilots working in a range of flight ops scenarios. The results will then be used for further optimizations of the system.

More information about the development of the system can be found here.