RE: Gaze tracking: military instructors take a fresh look at pilot training25 Nov 2021 12:08
2nd part....
Results have shown that the CTS helped instructors move from anecdotal evidence to factual evidence to evaluate where the trainees were looking, and also helped to understand their decision-making, as Seeing Machines Aviation General Manager Patrick Nolan explained to AeroTime.
“One Qualified Flying Instructor (QFI) watched a scenario playback without eye-tracking and remarked that they may have failed that trainee based on their observed performance. The accuracy was not at the required standard in terms of speed, height, and heading maintenance – all signs of cognitive overload,” Nolan said. “We then showed the instructor the same scenario playback but with the trainees’ gaze tracking overlaid on the instruments.
Nolan continued: “In this specific scenario the QFI was open to an alternative decision because what he could see with this additional information was that the trainees’ performance issues were not related to their cognitive load. The student pilot actually had a very good, structured, and disciplined scan pattern, and was not cognitively overloaded, but simply required some more time in the Hawk simulator getting comfortable from a ‘hands and feet’ coordination perspective.”
He added: “So, instead of failing the trainee, adding a remedial training package, and adding significant cost, the instructor determined that he would give this trainee an extra 10-15 minutes of free flight on his own to get more comfortable flying the aircraft.”
While Jericho’s scope of research was on gaze tracking, Seeing Machines continue to develop features using signals that CTS collects, such as pupil dilation, fixation, and dwell times. Through the company’s Human Factors and Advanced Engineering capability, these signals can then be developed into features that are useful to all aspects of the training process including such as measurements of stress and fatigue, situational awareness and cognitive workload.
A limitation identified during the Jericho engagement was that the capability is constrained by its central, fixed mounting position. While the field of view of the camera is wide and suitable for all aspects of visual and instrument flying, if the camera can no longer track a pilot’s eye(s), during dynamic air combat maneuvers, the system loses gaze tracking, this is a physical human limitation from a single sensor, but was agreed that further investigation of the requirement was required.
“If they're flying tactically and, particularly basic fighter maneuvering, they're spending a lot of time looking up and looking back, but the sensor is sitting right in front of them,” Nolan said. “So, all the system is really seeing then is the back or the side of their head. This is an opportunity to leverage our multi-camera solutions, but only if there is a clear value proposition to support a more complex capability. Additionally, another strong use case is Crew Resource Management in a multi-crew side-by-side ****pit en