Member Info for johnchucka

System, Method, and Program for Predicting Diminished Attentiveness State, and Storage Medium on Which Program Is Stored
A reduced attention state estimation system includes an eyeball movement and eyelid activity measurement unit that measures an eyeball movement and an eyelid activity of a subject to obtain eyeball movement and eyelid activity data, a section determination unit that determines an eye opening section, an eye closing section, an eye-blinking section and a cluster section based on the eyeball movement and eyelid activity data, an eyeball movement and eyelid activity-related information calculation unit that calculates a sharpness of microsaccade or the like for each eye opening section based on the eyeball movement and eyelid activity data, and an attention assessment unit that determines an attention assessment or the like for an eye opening/cluster section, which is an eye opening section and cluster section, based on the sharpness of microsaccade or the like.
https://worldwide.espacenet.com/publicationDetails/biblio?CC=US&NR=2020218935A1&KC=A1&FT=D&ND=3&date=20200709&DB=&locale=en_EP
https://www.freepatentsonline.com/20200218935.pdf

A reminder of some of the other closely dated Aircraft / Cokepit Patents which may or may not be related and lastly a list of the CAST members.

AIRBUS OPERATIONS SAS 2020-06-11
AIRCRAFT ****PIT AND METHOD OF DISPLAYING IN AN AIRCRAFT ****PIT

GE AVIAT SYSTEMS 2020-06-11
AIRCRAFT AND METHOD OF CONTROLLING

GE AVIAT SYSTEMS LTD 2020-06-17
METHOD OF ASSESSING A PILOT EMOTIONAL STATE

GE AVIATION SYSTEMS LLC 2020-06-11
AIRCRAFT AUGMENTED REALITY SYSTEM AND METHOD OF OPERATING

AIRBUS OPERATIONS SAS 2020-06-11
AIRCRAFT ****PIT AND METHOD OF DISPLAYING IN AN AIRCRAFT ****PIT

THALES SA 2020-06-11
METHOD AND SYSTEM FOR DISPLAY AND INTERACTION EMBEDDED IN A ****PIT

Industry CAST Members (Commercial Aviation Safety Team)

Aerospace Industries Association
Airbus
Airports Council International
Airlines for America
The Boeing Company
Flight Safety Foundation
General Electric (representing all engine manufacturers)
National Air Carrier Association
Regional Airline Association
https://www.faa.gov/news/fact_sheets/news_story.cfm?newsId=23035

Airline Pilot Forums (2018)
I'm sure everyone has seen the advertisements, the solicitation for volunteers, for the "Human Performance Initiative — Be All You Can Be!" cosponsored by Corporate Safety and our MEC chairman. (Funny, seems like an endorsement from our Association Safety Committee might be appropriate.)

The description of the project is quite vague.
"Safety and Airworthiness, in partnership with ALPA Safety, is investigating tools to improve individual performance on the flight deck. We are launching a Human Performance Initiative (HPI), which is a research project to better understand behavior and drive improvements by investing in the crew force. The HPI is working in collaboration with third-party experts, including flight operational researchers and human performance researchers."
I'm told participants will wear some sort of headband which will monitor physical responses, and the entire event will be "recorded" to evaluate the pilot throughout the event. It is not clear to me whether the recording will be a simple video camera, or more sophisticated eye movement tracker or monitor.

Given the FDX Chairman's interest in single-pilot Operations, it's not hard to draw a line between these "safety" studies and a plan to replace one pilot with a video camera to monitor the other one.
https://www.airlinepilotforums.com/fedex/111846-video-cameras-****pit.html

Fact Sheet – Commercial Aviation Safety Team (2020)
Looking Ahead
Established in 1997, CAST has evolved beyond the "historic" approach of examining past accident data to a proactive approach that focuses on detecting risk and implementing mitigation strategies before accidents or serious incidents occur. In 2010, CAST set a new goal: to reduce the U.S. commercial fatality risk by another 50 percent by 2025.

The transition to prognostic safety analysis requires greater emphasis on acquiring, sharing, and analyzing aviation safety data from across the aviation community. CAST works closely with the Aviation Safety Information Analysis and Sharing (ASIAS) program to leverage data from across the aviation community to identify emerging and changing risks, and proactively adopts voluntary safety enhancements to mitigate the risks.

Voluntary safety programs such as the Aviation Safety Action Program (ASAP), Flight Operational Quality Assurance (FOQA) program, and Air Traffic Safety Action Program (ATSAP) in ASIAS provide air carriers and the Government insight into millions of operations so potential systemic safety issues and trends are identified.

The ASIAS program connects a wide variety of safety data and information sources across industry and works closely with Government-industry safety teams like CAST and the General Aviation Joint Steering Committee (GAJSC) to monitor known risks, evaluate the effectiveness of deployed mitigations, and detect emerging risks.
https://www.faa.gov/news/fact_sheets/news_story.cfm?newsId=23035

ARTIFICIAL EYE FOR CALIBRATION OF AN EYE-TRACKER, EYE TRACKING CALIBRATION SYSTEM AND METHOD THEREOF
An artificial eye for eye tracking calibration of an eye-tracker, comprising: an eye model comprising a foveal region; an optical position sensor arranged at the foveal region for transducing the position of a foveal light spot into a sensor position signal; a two-axis rotational pivoting support upon which the eye model is mounted; a two-axis actuator for rotating the eye model in said pivoting support; a feed-back electronic circuit, connected between the sensor and actuator, arranged to cause rotation of the eye model by said actuator such that the sensor position signal matches a reference position signal. The reference position signal may be adjustable to cause an off-set between the pupillary-axis of the eye model and a visual-axis defined by the reference position signal, in particular the reference position signal being adjustable by configuration of the feed-back electronic circuit.
https://worldwide.espacenet.com/publicationDetails/biblio?CC=WO&NR=2020016631A1&KC=A1&FT=D&ND=3&date=20200123&DB=&locale=en_EP
https://www.freepatentsonline.com/WO2020016631A1.pdf

SYSTEM AND METHOD FOR IMPROVING SIGNAL TO NOISE RATIO IN OBJECT TRACKING UNDER POOR LIGHT CONDITIONS
Described herein is a system and method for performing eye tracking. One embodiment provides a system (100) including a camera (106) for capturing images of a vehicle driver's (102) eye and light emitting diodes (LEDs—108 and 110) configured to selectively illuminate the driver's eye during image capture by the camera (106). A processor (118) is configured to process at least a subset of the captured images to determine one or more eye tracking parameters of the subjects eye and to determine one or more illumination characteristics of the images. A controller (120) is configured to send an LED control signal to the LEDs (108 and 110) to control the drive current amplitude and pulse time of the LEDs (108 and 110). The controller (120) selectively adjusts the drive current amplitude and/or pulse time based on the determined illumination characteristics of a previous captured image or images.
https://worldwide.espacenet.com/publicationDetails/biblio?II=0&ND=3&adjacent=true&locale=en_EP&FT=D&date=20201015&CC=US&NR=2020327323A1&KC=A1
https://www.freepatentsonline.com/20200327323.pdf

Back in 2015 @4.30m KK expected 5000 units to be in action during the first 6 months.
https://www.youtube.com/watch?v=rRUjObrkQl4

Forgot to add.
This Patent references Langdale-Smith, N., Jan 27, 2015. CES 2015--Seeing Machine

Kong, L. et al., "Non-contact detection of oxygen saturation based on visible light imaging device using ambient light," Optics Express, vol. 21, No. 15, pp. 17464-17471, Jul. 29, 2013. cited by applicant .
Kuboyama Yuta, "Motion Artifact Cancellation for Wearable Photoplethysmographic Sensor", B.S. Electrical Engineering and Computer Science, MIT, 2009, 66 pages. cited by applicant .

Langdale-Smith, N., Jan 27, 2015. CES 2015--Seeing Machines: The Future of Automotive Safety. Retrieved from https://www.youtube.com/watch?v=obPnLufAu7o. cited by applicant .

Moharir, P.S. et al., "Optical Barker Codes", Electronics Letters, published May 2, 1974, vol. 10, No. 9, Mar. 28, 1974, 2 pages. cited by applicant .
Murata et al.: "Noninvasive Biological Sensor System for Detection of Drunk Driving", IEEE Transactions on Information Technology in Biomedicine, vol. 15, No. 1, Jan. 2011. cited by applicant .
Nakayama, O. et al., "Development of a Steering Entropy Method for Evaluating Driver Workload," SAE Technical Paper Series 1999-01-0892, Mar. 1-4, 1999, Detroit, Michigan, USA. cited by applicant .
Nobata et al., Study of the Personal Authentication Technique Using ECG Signal toward Driver Recognition, 2 pages. cited by applicant .
Ofalt, Martin M., Jr., "Ford, MIT Partnering to Increase Driver Wellness and Safety," The College Driver.com, Jan. 24, 2010. cited by applicant .
https://uspto.report/patent/grant/10,759,438#:~:text=SEEING%20MACHINES

I may have posted one of the earlier iterations of this Honda patent but it's worth a re visit simply to demonstrate the huge range of vehicle sensors Honda are using in combination with Driver state monitor described here.

Method for controlling vehicle systems in motor vehicles
Method for controlling vehicle systems in motor vehicles. A method for controlling vehicle systems includes receiving monitoring information from one or more monitoring systems and determining a plurality of driver states based on the monitoring information from the one or more monitoring systems. The method includes determining a combined driver state based on the plurality of driver states and modifying control of one or more vehicle systems based on the combined driver state.
https://worldwide.espacenet.com/publicationDetails/biblio?CC=CN&NR=110871809A&KC=A&FT=D&ND=3&date=20200310&DB=&locale=en_EP

PROGRAM AND TRAIN OPERATION SIMULATOR
To present gazing tendency during train operations by operation simulation to a user.SOLUTION: In a train operation simulator 1000, an eye tracking 104 tracks and detects a line of sight of a user during the execution of operation simulation. A gazed object specification part 216 specifies an object in an operation screen to which the line of sight of the user is directed at a current travel position as a gazed object at the travel position during the execution of the operation simulation. A statistical processing part 220 performs predetermined statistical processing based on the gazed object at each travel position specified by the gazed object specification part 216 after the execution of the operation simulation, and calculates a statistical value related to a time or distance when or where a predetermined type of object is gazed. A gazing result display control part 222 controls display of a gazing result based on the statistical value.SELECTED DRAWING: Figure 7
http://translationportal.epo.org/emtp/translate/?ACTION=description-retrieval&COUNTRY=JP&ENGINE=google&FORMAT=docdb&KIND=A&LOCALE=en_EP&NUMBER=2019135505&OPS=ops.epo.org/3.2&SRCLANG=ja&TRGLANG=en

[0046] In some implementations, the eye tracking sensors 208 are provided as infrared sensors. In some implementations, the eye tracking sensors 208 are provided as visible light, infrared-free tracking sensors. In some implementations, the eye tracking sensors 208 are incorporated into a control panel of the aircraft 204. In some implementations, the eye tracking sensors 208 are detachably fixed to the aircraft 204, and can be moved between aircrafts. In some implementations, the eye tracking sensors 208 are communicably coupled to the IRM computing device 150 to provide eye tracking data to the computing device 150 to be processed by risk management application 144b stored on the IRM computing device 150. In some implementations, the eye tracking sensors 208 are communicably coupled to the server 102 to provide eye tracking data to the server 102 to be processed by risk management application 144a stored on the server 102.

MITIGATING OPERATIONAL RISK IN AIRCRAFT
An in-flight risk management system includes an eye sensor, a seat sensor, and one or more in-flight risk management computers that perform operations including: obtaining history information for a flight crew member, obtaining flight information associated with a flight on which the flight crew member will be present, determining, based on the history information and the flight information, a predicted fatigue profile for the flight crew member, determining, based on the predicted fatigue profile for the crew member, one or more fatigue indicator profiles for the flight crew member, obtaining, from at least one of the eye sensor or seat sensor, one or more measured fatigue indicator values associated with the flight crew member, generating, based on the one or more measured fatigue indicator values, one or more updated fatigue indicator profiles, and generating, based on the one or more updated fatigue indicator profiles, an updated predicted fatigue profile.

[0045] In some implementations, IRM system 200 includes one or more eye tracking sensors 208a, 208, 208c, 208d (referred to generically as eye tracking sensors 208). In some implementations, the eye tracking sensors 208 provide data to the IRM system 200 about the eye movement of user 206a and user 206b (referred to collectively as users 206 or generically as user 206). For example, eye tracking sensors 208 can determine the number of eyelid closures (i.e. blinks) that a flight crew member experiences in a minute. In some examples, the eye tracking sensors 208 can determine the length of time that a user's 206 eyelids are closed. In some examples, the eye tracking sensors 208 monitor the eye movement of a pilot (e.g., one of users 206) to determine whether the pilot's gaze is fixed on a particular point, and determine the length of time the pilot is fixed on that point. In some examples, the eye tracking sensors 208 determine whether a pilot is scanning the field of vision. In some implementations, the eye tracking sensors 208 track the eye movement of multiple users 206. For example, the eye tracking sensors can track the eye movements of a pilot of the aircraft 204 (e.g., one of users 206) and the eye movement of a co-pilot of the aircraft 204 (e.g., one of users 206). Data provided by the eye tracking sensors 208 to the IRM system 200, such as prolonged gazes, increased blink rate, or a combination thereof, can provide an indication of an increasing fatigue level of the respective flight crew member.
https://worldwide.espacenet.com/publicationDetails/description?CC=US&NR=2020290740A1&KC=A1&FT=D&ND=3&date=20200917&DB=&locale=en_EP

VEHICULAR DRIVER MONITORING SYSTEM
A vehicular driver monitoring system includes an illumination source that, when powered, emits light that illuminates at least a portion of a driver of the vehicle. A camera is disposed at the interior of the vehicle and views the portion of the driver. A control includes an image processor that processes captured image data. The illumination source is controlled to modulate intensity of light emitted by the illumination source. The image data output is processed to distinguish a component of the image data output that arises from the illumination source illuminating the portion of the driver from a component of the image data output that arises from ambient light illuminating the portion of the driver. The control filters the image data output to reduce the distinguished component that arises from ambient light and, responsive to processing of the filtered image data output, monitors the portion of the driver.

The driver monitoring camera 14 captures image data representative of the driver's head. The interior cabin-viewing camera may be disposed at any suitable location in interior cabin of the vehicle, such as at a dashboard or instrument panel of the vehicle or within the movable head of an interior rearview mirror or part of a windshield electronics module (that includes a forward-viewing camera for advanced driver assistance functions such as described in U.S. Pat. Nos. 9,896,039; 9,871,971; 9,596,387; 8,256,821; 7,480,149; 6,824,281 and/or 6,690,268, which are all hereby incorporated herein by reference in their entireties) with the field of view of the cabin-viewing camera encompassing the driver head region in the vehicle (either directly or via a reflector). The light source 16 is operable to illuminate the driver front head region in a manner that enhances capturing and processing of image data in high ambient lighting conditions, as discussed below.
https://worldwide.espacenet.com/publicationDetails/biblio?CC=US&NR=2020320320A1&KC=A1&FT=D&ND=3&date=20201008&DB=&locale=en_EP
https://patents.justia.com/patent/20200320320
https://www.freepatentsonline.com/20200320320.pdf

SEE recent follow on Twitter.
I'm guessing it's an artist the tweeter at SEE likes or more cunningly the title of her track 'No More Hiding' is a clue about how the next RNS is going to wake the world up to SEE. One lives in Hope.
https://twitter.com/ginabrooklynn

Yes

DISTRACTED STATE DETERMINATION DEVICE AND DISTRACTED STATE DETERMINATION METHOD
This distracted state determination device (20) is provided with a stimulation instruction unit (27) which instructs the stimulation of a person if the person may be in an distracted state, a user information acquisition unit (21) which acquires detection information about the person, and a distracted state determination unit (25) which determines whether the person has fallen into a distracted state. On the basis of change in the detection information about the person before and after the stimulation instruction, the distraction determination unit (25) determines whether the person had fallen into a distracted state before the stimulation instruction.
https://worldwide.espacenet.com/publicationDetails/biblio?CC=WO&NR=2020195168A1&KC=A1&FT=D&ND=3&date=20201001&DB=&locale=en_EP

METHOD AND CONTROL UNIT FOR DETECTING DROWSINESS OF A DRIVER FOR A DRIVER ASSISTANCE SYSTEM FOR A VEHICLE
A method for detecting drowsiness of a driver for a driver assistance system for a vehicle includes a read-in step, an ascertainment step, and a step of determining a drowsiness signal. In the read-in step, at least a first indicator signal that represents a first drowsiness parameter of the driver determined by a first drowsiness-detection device of the vehicle, a second indicator signal that represents a second drowsiness parameter of the driver determined by a second drowsiness-detection device of the vehicle, and optionally a third indicator signal that represents a third drowsiness parameter of the driver determined by a third drowsiness-detection device of the vehicle are read in. In the ascertainment step, a first validity of the first indicator signal, a second validity of the second indicator signal, and optionally a third validity of the third indicator signal are ascertained. In the determination step, the drowsiness signal that represents the detected drowsiness of the driver is determined, utilizing the first indicator signal, the second indicator signal, optionally the third indicator signal, the first validity, the second validity, and optionally the third validity.

For example, one drowsiness parameter can indicate a state or degree of drowsiness of the driver. One drowsiness parameter, for instance, can indicate whether the driver is judged to be very tired, somewhat tired, or not tired. A validity of an indicator signal can indicate a probability for correctness of the drowsiness parameter represented by the indicator signal. The individual drowsiness parameters can differ, since they can be determined in a variety of ways. Accordingly, different validities can be assigned to the drowsiness parameters. In the ascertainment step, the first validity, the second validity, and additionally or alternatively the third validity can be ascertained, e.g., utilizing a function test of the first, second, and additionally or alternatively third drowsiness-detection device. Thus, for example, if the function test indicates a malfunction of one of the drowsiness-detection devices, the validity of the indicator signal of this drowsiness-detection device can then be ascertained accordingly, e.g., can be classified as low. For instance, an operational function of a camera of a drowsiness-detection device can be tested in the function test, and additionally or alternatively, a quality of a camera image of the camera can be checked. Advantageously, the validity of the individual drowsiness parameters can be inferred very accurately by way of such a function test.
https://worldwide.espacenet.com/publicationDetails/biblio?CC=US&NR=2020317211A1&KC=A1&FT=D&ND=3&date=20201008&DB=&locale=en_EP
https://patents.justia.com/patent/20200317211
https://www.freepatentsonline.com/y2020/0317211.html

VISION SYSTEM AND METHOD FOR A MOTOR VEHICLE
A vision system (10) for a motor vehicle with a stereo imaging apparatus (11) with imaging devices (12) adapted to capture images from a surrounding of the motor vehicle, and a processing device (14) adapted to process images captured by the imaging devices (12) and to detect objects, and track detected objects over several time frames, in the captured images. The processing device (14) is adapted to obtain an estimated value for the intrinsic yaw error of the imaging devices (12) by solving a set of equations, belonging to one particular detected object (30), using a non-linear equation solver method, where each equation corresponds to one time frame and relates a frame time, a disparity value of the particular detected object, an intrinsic yaw error and a kinematic variable of the vehicle.
https://worldwide.espacenet.com/publicationDetails/biblio?II=0&ND=3&adjacent=true&locale=en_EP&FT=D&date=20201008&CC=US&NR=2020322588A1&KC=A1
https://www.freepatentsonline.com/20200322588.pdf

SENSOR DATA SHARING BETWEEN VEHICLES
Various aspects of the present disclosure generally relate to sensor systems. In some aspects, a method may include receiving information associated with a sensor system of a first vehicle. The method may include determining, based on the information, a score associated with the sensor system. The score may be representative of a reliability of the sensor system. The method may include determining whether the score satisfies a reliability threshold. The method may include enabling, based on determining that the score satisfies the reliability threshold, use of sensor data, associated with the sensor system, for an operation associated with the second vehicle. Numerous other aspects are provided.
https://worldwide.espacenet.com/publicationDetails/biblio?II=4&ND=3&adjacent=true&locale=en_EP&FT=D&date=20201008&CC=WO&NR=2020199183A1&KC=A1

ROUTE GUIDANCE APPARATUS AND METHOD
Accordingly, the present invention provides a route guidance apparatus and method that substantially obviate one or more problems due to limitations and disadvantages of the related art. An object of the present invention is to provide a route guidance apparatus and method which may match image data collected by a camera with a driver's visible region and flexibly determine the size of a direction indicator according to a remaining distance, to clearly recognize change in direction without hindering realism.

Additional advantages, objects, and features of the invention will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the invention. The objectives and other advantages of the invention may be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

To achieve these objects and other advantages and in accordance with the purpose of the invention, as embodied and broadly described herein, a route guidance apparatus may include an image acquisition unit configured to collect image data of an area in front of a vehicle, photographed by a camera, a driver's view recognition unit configured to calculate the range of a driver's viewing angle by detecting a driver's eye height and a driver's gaze direction, an image processing unit configured to extract a region of interest that corresponds to the range of the driver's viewing angle from the image data and to generate a route guidance image by overlaying a direction indicator that corresponds to a driving route on the region of interest, and a display unit configured to output the route guidance image.

The region of interest may be a region that corresponds to the driver's gaze direction, out of the image data. The route guidance apparatus may further include a road configuration recognition unit configured to detect a guide point on the driving route. The image processing unit may be configured to determine at least one of a size, a shape or a position of the direction indicator based on a remaining distance from a current position of the vehicle to the guide point. The guide point may include at least one of a forked road, an entrance/exit section, a roundabout, an intersection or a turn point.
https://worldwide.espacenet.com/publicationDetails/biblio?II=1&ND=3&adjacent=true&locale=en_EP&FT=D&date=20201008&CC=US&NR=2020318989A1&KC=A1
https://patents.justia.com/patent/20200318989
https://www.freepatentsonline.com/20200318989.pdf

Eye Tracking
[0003] Therefore, a need exists for advanced eye tracking techniques. In particular, need exists for eye tracking techniques which enable dynamic activation of menu items of a GUI based an eye gaze.

[0004] This need is met by the features of the independent claims. The dependent claims define embodiments.

[0005] In an example, a method comprises displaying a GUI. The GUI comprises a plurality of menu items. The method further comprises detecting a first user action. The first user action comprises an eye gaze associated with the eyes of a user. The method further comprises, based on the orientation of the eye gaze of the first user action: selecting at least one first menu item from the plurality of menu items. The method further comprises detecting a second user action. The method further comprises activating a function associated with a given one of the at least one first menu item based on the second user action.

[0006] In an example, a device comprises at least one display, a memory, and at least one processor. The memory is configured to store program code. The at least one processor is coupled with the memory and the at least one display. The at least one processor is configured to execute the program code. Execution of the program code causes the at least one processor to perform the following: controlling the display to display a GUI comprising a plurality of menu items; detecting a first user action comprising an eye gaze associated with the eyes of a user; based on the orientation of the eye gaze of the first user action: selecting at least one first menu item from the plurality of menu items; detecting a second user action; and, based on the second user action: activating a function associated with a given one of the at least one first menu item.

[0007] In an example, a computer program product comprises program code to be executed by at least one processor. Executing the program code causes the at least one processor to perform a method. The method comprises displaying a GUI. The GUI comprises a plurality of menu items. The method further comprises detecting a first user action. The first user action comprises an eye gaze associated with the eyes of a user. The method further comprises, based on the orientation of the eye gaze of the first user action: selecting at least one first menu item from the plurality of menu items. The method further comprises detecting a second user action. The method further comprises activating a function associated with a given one of the at least one first menu item based on the second user action.

https://worldwide.espacenet.com/publicationDetails/description?CC=US&NR=2020319705A1&KC=A1&FT=D&ND=3&date=20201008&DB=&locale=en_EP
https://www.freepatentsonline.com/20200319705.pdf

Toyota The Future of Vehicle UX
https://www.tri-ad.global/areas-of-focus/design-system