Ensuring harmonious human–machine collaboration in the cockpit

Since autopilot was first demonstrated in 1914, pilots have found themselves increasingly “connected” to their aircrafts through numerous displays, knobs and instruments in the cockpit. This increasing degree of pilot assistance has been developed to help reduce pilot workload, always with aviation safety as a driver.

Paradoxically however there is the risk that when faced with an atypical event pilots encounter difficulties coping with the vast amounts of information generated by their instruments. It is therefore crucial to improve and optimize their situational awareness and the relationship between the flight crew and the aircraft controls.

PEGGASUS—optimizing human–machine interactions

Endorsed by the Clean Sky 2 initiative, the European project PEGGASUS aims to counter the “instruments paradox” by understanding crew members’ actions and behaviour and by moving towards a multimodal cockpit interactivity, thus allowing pilots greater levels of control. “We need to recognize when limitations such as confusion or drowsiness impact attention, mental workload, and decision-making on the flight deck,” explains Andrea Dunbar, Head of Embedded Vision Systems at CSEM. “Additionally, the new HMI we are developing will eventually enable a more intuitive and natural interaction so they can make quick, informed decisions across any situation, even when stressed.”

To design a powerful and tailored solution, CSEM and three partners will provide the essential complementary skills, ensuring the project’s success. “Our company is pleased to support the consortium partners with the expertise of a professional airline,” says Christoph Ammann, Vice-President, Head of Crew Training at Swiss International Air Lines, a member of the Lufthansa Group. “Mutual exchange with research and industry partners enables us to reflect on our training standards and on potential future applications."

The airline has previously collaborated with consortium member ETH Zurich to develop novel gaze-based techniques to monitor pilots’ cognitive states and situational awareness. In particular, these new methods were designed to allow more efficient and effective interaction between the pilot and the aircraft, while also expanding pilot training techniques. The consortium will build upon this excellent experience base and previously collected data when developing PEGGASUS.

Integrating HMI systems into the cockpit

“The aeronautic context poses unique challenges for our team,” comments Andrea Dunbar. “The vision systems and machine learning algorithms CSEM will develop must take into account both the pilots in the cockpit. The technology will be developed to remain accurate and robust during the course of a flight, considering aviation-specific environmental factors such as changing lighting conditions and vibrations.” Consortium partner SERMA Ingénierie will be responsible for integrating the PEGGASUS output into a cockpit prototype for testing.

The consortium’s work will also be supported by Thales, Clean Sky 2 leader. Thales’ Thierry Maret, Program Manager, is proud that PEGGASUS will mean another step forward in “providing pilots with new ways of interacting with the aircraft system so pilots can easily and efficiently adapt to the changing and complex needs of 21st century avionics.”