RE: Waymo 2021-01-0713 Jan 2021 19:30
[0029] In another aspect, the technology relates to passively detecting driver fatigue of an assist driver in a vehicle configured to operate in a fully or partially autonomous mode, or configured for manual operation. Gaze patterns may serve as a reliable indicator of cognitive states, such as for example distractions. Gaze patterns may also serve as a reliable indicator of fatigue. For instance, a fatigued driver typically does not vigilantly monitor the road but rather gazes in one direction, e.g., straight ahead. An autonomous vehicle includes sensors and systems that can detect objects, etc., surrounding the vehicle and derive scene understanding with respect to the vehicle, and may be able to detect the direction of the driver's gaze. As such, one or more computing devices of the vehicle may determine one or more expected gaze direction, including an expected or normative gaze direction, pattern of directions, group of directions or sequence of directions (collectively referred to herein as “gaze direction or patterns”), for a driver based on the derived scene understanding. If the driver is not looking in an expected gaze direction or pattern on one or more occasions, such as if the driver's gaze patterns are not well correlated with an expected gaze pattern, this may serve as an indicator of passive fatigue.
[0030] For example, if a scene understanding derived from the vehicle's perception system indicates that based on naturally occurring stimuli outside the vehicle a driver should be looking in a particular direction, the actual gaze direction or pattern of the driver may then be compared to that particular direction or pattern to provide a measure of the driver's attentiveness and thus passive fatigue. As a further example, if the driver is making a right turn, the driver would be expected to be mainly looking to the left for oncoming traffic. However, if the detected gaze direction or pattern of the driver indicates that the driver is looking straight ahead that may signal that the driver is no longer sufficiently engaged in the driving task (e.g., due to low vigilance or fatigue). As the system can tell when and where drivers are expected to be looking, it allows for dynamically assessing a driver's vigilance and/or fatigue passively. The actual gaze direction (or pattern of gaze direction) may be detected through use of cameras within the compartment of the vehicle.