PhD graduation Luis Lutnyk
We’re glad to announce that our colleague Luis Lutnyk has successfully defended his dissertation on “Pilot Decision-Making Support through Intelligent Cockpit Technologies”. Congratulations, Luis!
- eyetracking@ethz.ch
- +41 44 633 71 59
Category Archives: PEGGASUS
- 21
Jun 23
-
- 10
Feb 23
-
New article in Applied Ergonomics – The effect of flight phase on electrodermal activity and gaze behavior: A simulator study
Our article “The effect of flight phase on electrodermal activity and gaze behavior: A simulator study” has been accepted for publication in the journal Applied Ergonomics:
Luis Lutnyk, David Rudi, Victor R. Schinazi, Peter Kiefer, Martin Raubal (2022). The effect of flight phase on electrodermal activity and gaze behavior: A simulator study , Applied Ergonomics, Volume 109, DOI: 10.1016/j.apergo.2023.103989 .
Highlights:
- Unobtrusive technologies were used to record electrodermal activity and gaze behavior in an instrument failure scenario.
- Participants’ electrodermal activity increased significantly during high workload phases of the failure scenario.
- AOI-based & non-AOI eye tracking metrics show significant differences when a secondary task needs to be solved during flight.
- The observed measures show great potential for future cockpits that can provide assistance based on the sensed pilot state.
Abstract. Current advances in airplane cockpit design and layout are often driven by a need to improve the pilot’s awareness of the aircraft’s state. This involves an improvement in the flow of information from aircraft to pilot. However, providing the aircraft with information on the pilot’s state remains an open challenge. This work takes a first step towards determining the pilot’s state based on biosensor data. We conducted a simulator study to record participants’ electrodermal activity and gaze behavior, indicating pilot state changes during three distinct flight phases in an instrument failure scenario. The results show a significant difference in these psychophysiological measures between a phase of regular flight, the incident phase, and a phase with an additional troubleshooting task after the failure. The differences in the observed measures suggest great potential for a pilot-aware cockpit that can provide assistance based on the sensed pilot state.
The article has been published as Open Access and you can get the PDF here:
https://www.sciencedirect.com/science/article/pii/S0003687023000273
The publication is part of PEGGASUS. This project has received funding from the Clean Sky 2 Joint Undertaking under the European Union’s Horizon 2020 research and innovation program under grant agreement No. 821461
- 02
Jun 22
-
Co-authored paper on PEGGASUS at SPIE Photonics West
A joint paper with our partners at CSEM on the eye tracking hard- and software developed in PEGGASUS was published on the SPIE digital library. We encourage you to take a deep dive into the technological achievements of PEGGASUS:
“The pipeline, which is a combination of data-driven and analytics approaches, runs in real time at 60 fps with a latency of about 32ms. The eye gaze estimation error was evaluated in terms of the point of regard distance error with respect to the 3D point location. An average error of less than 1.1cm was achieved over 28 gaze points representing the cockpit instruments placed at about 80-110cm from the participants’ eyes.”
Engin Türetkin, Sareh Saeedi, Siavash Bigdeli, Patrick Stadelmann, Nicolas Cantale, Luis Lutnyk, Martin Raubal, Andrea L. Dunbar (2022). Real time eye gaze tracking for human machine interaction in the cockpit In AI and Optical Data Sciences III (Vol. 12019, pp. 24-33). SPIE..
The paper was presented at SPIE’s Photonics West conference at San Francisco’s Moscone Center.
Find the full text and presentation video here: https://doi.org/10.1117/12.2607434
The publication is part of PEGGASUS. This project has received funding from the Clean Sky 2 Joint Undertaking under the European Union’s Horizon 2020 research and innovation program under grant agreement No. 821461
- 08
Feb 22
-
Successful finish of PEGGASUS and summary
We are happy to report that our aviation project PEGGASUS (Pilot Eye Gaze and Gesture tracking for Avionics Systems using Unobtrusive Solutions) successfully finished.
We would like to thank all partners involved in the project for the extensive efforts to finish the project successfully and deliver the results despite all the Covid-related restrictions and hurdles.
You can find a summary of the project outcomes at the EU Cordis Portal:
“The PEGGASUS consortium has developed a multi-camera vision system for tracking eye-gaze and gestures of the pilots in real time in the cockpit environment. This system allows a leap towards a more comprehensive human-machine interface in the cockpit to reduce the stress and cognitive load of the pilots, while bringing forward future pilot training techniques. Better awareness of the instruments will help the flight management to optimize trajectories and better manage fuel use, in line with the overall objectives of the Clean Sky JU.”
Two images showing the results of the algorithms including pupil detection and eye gaze estimation
Ultimate Prototype Hardware setup installed in the cockpit simulator
Excerpt and images taken from: https://cordis.europa.eu/project/id/821461/reporting
This project has received funding from the Clean Sky 2 Joint Undertaking under the European Union’s Horizon 2020 research and innovation program under grant agreement No. 821461
- 17
Nov 21
-
Joint Talk at the Royal Aeronautical Society Flight Simulation Conference
We had the honor of giving a presentation at the Royal Aeronautical Society‘s Flight Simulation Conference which took place from October 26nd to 27th in London.
In the joint talk with Gilad Scherpf of Lufthansa Group, we presented results from the PEGGASUS project and showcased how eye and gesture tracking can support the assessment of EBT competencies. (Evidence-Based Training)
We want to thank the hosts at the RAeS for the invitation and the attendees for the very positive feedback and interesting questions during the panel discussion. A special thank you also goes out to our partners at SWISS and CSEM.
The full programme of the conference can be found here.
The talk was given as part of PEGGASUS. This project has received funding from the Clean Sky 2 Joint Undertaking under the European Union’s Horizon 2020 research and innovation program under grant agreement No. 821461
- 14
Dec 20
-
Workshop Paper published from ICCAS 2020
Our paper “Recognizing Pilot State: Enabling Tailored In-Flight Assistance Through Machine Learning” has been published in the proceedings of the 1st International Conference on Cognitive Aircraft Systems:
Lutnyk, L., Rudi, D., Kiefer, P., & Raubal, M. (2020). Recognizing Pilot State: Enabling Tailored In-Flight Assistance Through Machine Learning ICCAS 2020.
Abstract. Moving towards the highly controversial single pilot cockpit, more and more automation capabilities are added to today’s airliners. However, to operate safely without a pilot monitoring, avionics systems in future cockpits will have to be able to intelligently assist the remaining pilot. One critical enabler for proper assistance is a reliable classification of the pilot’s state, both in normal conditions and more critically in abnormal situations like an equipment failure. Only with a good assessment of the pilot’s state, the cockpit can adapt to the pilot’s current needs, i.e. alert, adapt displays, take over tasks, monitor procedures, etc.
The publication is part of PEGGASUS. This project has received funding from the Clean Sky 2 Joint Undertaking under the European Union’s Horizon 2020 research and innovation program under grant agreement No. 821461
- 24
Apr 20
-
Workshop Paper published from ETAVI 2020
Our paper “Towards Pilot-Aware Cockpits” has been published in the proceedings of the 1st International Workshop on Eye-Tracking in Aviation (ETAVI 2020):
Lutnyk L., Rudi D., and Raubal M. (2020). Towards Pilot-Aware Cockpits. In Proceedings of the 1st International Workshop on Eye-Tracking in Aviation (ETAVI 2020), ETH Zurich. DOI: https://doi.org/10.3929/ethz-b-000407661
Abstract. Eye tracking has a longstanding history in aviation research. Amongst others it has been employed to bring pilots back “in the loop”, i.e., create a better awareness of the flight situation. Interestingly, there exists only little research in this context that evaluates the application of machine learning algorithms to model pilots’ understanding of the aircraft’s state and their situation awareness. Machine learning models could be trained to differentiate between normal and abnormal patterns with regard to pilots’ eye movements, control inputs, and data from other psychophysiological sensors, such as heart rate or blood pressure. Moreover, when the system recognizes an abnormal pattern, it could provide situation specific assistance to bring pilots back in the loop. This paper discusses when pilots benefit from such a pilot-aware system, and explores the technical and user oriented requirements for implementing this system.
Edit. The publication is part of PEGGASUS. This project has received funding from the Clean Sky 2 Joint Undertaking under the European Union’s Horizon 2020 research and innovation program under grant agreement No. 821461
- 15
Apr 20
-
ETAVI 2020 Proceedings Online
The proceedings of the 1st International Workshop on Eye-Tracking in Aviation (ETAVI) 2020 have been published here.
We thank all program committee members for their efforts and great support, it is very much appreciated.
Furthermore, we thank all authors of the 15 excellent articles that were accepted for ETAVI 2020. We regret that the event had to be cancelled due to COVID-19.
Edit. Some of the organizers from ETH are part of the PEGGASUS project. This project has received funding from the Clean Sky 2 Joint Undertaking under the European Union’s Horizon 2020 research and innovation program under grant agreement No. 821461
- 17
Sep 19
-
Visit by the Vice President for Research and Corporate Relations
On 11 September, Prof. Dr. Detlef Günther, the Vice President for Research and Corporate Relations of ETH Zurich, has visited the D-BAUG department and informed himself about the exciting research activities of the different institutes.
Our institute was represented by Peter Kiefer, who summarized the research of the GeoGazeLab. The slides provide an overview on our research interests and current projects.
Edit. The presentation includes the PEGGASUS project. This project has received funding from the Clean Sky 2 Joint Undertaking under the European Union’s Horizon 2020 research and innovation program under grant agreement No. 821461
- 20
Aug 19
-
ETAVI – 2nd Call for Papers
A quick reminder for the “1st International Workshop on Eye-Tracking in Aviation (ETAVI)” that is going to take place March 2019 in Toulouse, France.
The submission deadlines are:
- Abstracts: 9th September 2019
- Paper: 30th September 2019
Feel free to also forward the Call for Papers to any interested colleagues.
We look forward to seeing you there!
Edit. Some of the organizers from ETH are part of the PEGGASUS project. This project has received funding from the Clean Sky 2 Joint Undertaking under the European Union’s Horizon 2020 research and innovation program under grant agreement No. 821461
- 29
Jul 19
-
PEGGASUS in the news
Our research project PEGGASUS (Pilot Eye Gaze and Gesture tracking for Avionics Systems using Unobtrusive Solutions) has attracted quite a bit of coverage in the media.
See for yourself in this little press review:
-
- “Jetzt kommt der Robo-Copilot” on higgs magazine (article, German)
- “Le CSEM coordonne un projet européen d’assistance aux pilotes d’avion” on Radio Télévision Suisse (radio broadcast – click on image for audio, French)
- “Ensuring harmonious human–machine collaboration in the cockpit” in Eurotec magazine (arcticle, English)
- “Hand in Hand im Cockpit” in Aktuelle Technik (article, German)
- “Mensch-Maschine-Schnittstelle im Cockpit” in Digital Engineering Magazine (article, German)
Edit. This project has received funding from the Clean Sky 2 Joint Undertaking under the European Union’s Horizon 2020 research and innovation program under grant agreement No. 821461
- 26
Feb 19
-
New aviation project: PEGGASUS
PEGGASUS (Pilot Eye Gaze and Gesture tracking for Avionics Systems using Unobtrusive Solutions)
We’re glad to announce the start of a new aviation project at the GeoGazeLab.
Check out our vision for pilot interactions in the cockpit of the future at the project page.
Edit. This project has received funding from the Clean Sky 2 Joint Undertaking under the European Union’s Horizon 2020 research and innovation program under grant agreement No. 821461
- 25
Jan 19
-
Luis Lutnyk joins the team
We are happy to welcome Luis Lutnyk as a new PhD student in the GeoGazeLab! His research will be about eye tracking in aviation.
Edit. Luis is part of the PEGGASUS project. This project has received funding from the Clean Sky 2 Joint Undertaking under the European Union’s Horizon 2020 research and innovation program under grant agreement No. 821461
