London South East | The UK's favourite investor portal.

V2V Patent describes drowsy driver detection using driver gaze and other sensors , then if driver fails to respond to action requests describes a process for alerting other close by vehicles to the hazard.

APPARATUS SYSTEM AND METHOD FOR MANAGING DROWSY DRIVING
An apparatus for managing drowsy driving according to one embodiment of the present invention includes: a communication unit for transmitting vehicle information to a server and receiving a driver's drowsiness probability from the server; and a control unit which generates a command when the driver's drowsiness probability exceeds a reference value and determines a driver's drowsiness state based on the driver's input information corresponding to the command. When it is determined that a driver's state is a drowsy state, a driver is guided to carry out the command provided by the vehicle, thereby improving driving concentration without an additional increase in cost.

The sensor 22 may acquire driver information. Here, the driver information may include the driver's drowsiness information, and the drowsiness information may include the driver's eyes closed state and a gaze direction.
The sensor 22 may acquire vehicle information about a lane in front of the vehicle, an obstacle around the vehicle, and a driving environment by detecting the front of the vehicle while the vehicle is driving.
As an example, the sensor 22 is a distance sensor (radar sensor or lidar sensor) that acquires lane information, a speed sensor that detects the vehicle speed, a DATC (Dual Auto Temperature Control) sensor that detects the temperature inside the vehicle, and the amount of carbon dioxide inside the vehicle. It may include a carbon dioxide measurement sensor to measure, an illuminance sensor to detect the brightness of the vehicle outside, an ODS (Occupant Detection System) sensor to detect a passenger's boarding, and the like.
However, the sensor 22 is not limited to the above-described embodiment, and may include sensors capable of acquiring driver's sleepiness information or vehicle information.
The navigation 23 may provide vehicle location information, surrounding environment information, and vehicle information.

The communication unit 41 may receive the driver's drowsiness information obtained from the camera 21, the sensor 22, and the navigation 23 from the other vehicle 11, and vehicle information at a time when drowsiness is detected.
The communication unit 41 may receive vehicle information obtained from the camera 31, the sensor 32, and the navigation 33 from the own vehicle 12 at the current time point.
The controller 42 may learn to calculate a drowsiness probability based on the driver's drowsiness information received from the other vehicle 11 and vehicle information at the time when drowsiness is detected.
https://worldwide.espacenet.com/publicationDetails/biblio?CC=KR&NR=20200063669A&KC=A&FT=D&ND=3&date=20200605&DB=&locale=en_EP
https://www.freepatentsonline.com/2020

Fusion of electronic mirror systems and driver monitoring for increased convenience and added safety.
An apparatus includes an interface and a processor. The interface may be configured to communicate video data to a display. The processor may be configured to receive video frames from a capture device, generate video data for the display in response to the video frames, perform operations to detect an object in the video frames and determine eye position information. The eye position information may be used to determine whether a driver is looking at the display. The processor may be configured to adjust content presented by the display when the driver is looking at the display

Embodiments of the present invention include providing fusion of electronic mirror systems and driver monitoring for increased convenience and added safety that may (i) track where a driver is looking, (ii) intelligently adapt notifications to a gaze direction of a driver, (iii) suppress notifications in response to a gaze direction of a driver, (iv) adapt a brightness of a display in response to a gaze direction in dark environments, (v) provide internal and external computer vision, (vi) detect objects around the vehicle and/or (vii) be implemented as one or more integrated circuits.

Embodiments of the present invention may be configured to adjust output to displays. Vehicles may be designed with multiple displays integrated. The displays may provide information and/or e-mirror solutions (e.g., left, right and center rear-view displays). Embodiments of the present invention may comprise implementing e-mirror systems enhanced with driver state monitoring (DSM) and/or driver monitoring systems (DMS). DSM and/or DMS may be used to detect a driver and/or detect and track a gaze direction of a driver. Embodiments of the present invention may further comprise computer vision (CV). Computer vision may be implemented in order to detect approaching vehicles and/or objects from behind in the field of view of the e-mirror system and alert/warn the driver.
https://patents.justia.com/patent/10810966
https://www.freepatentsonline.com/10810966.html
https://worldwide.espacenet.com/publicationDetails/biblio?CC=US&NR=10810966B1&KC=B1&FT=D&ND=3&date=20201020&DB=&locale=en_EP

May have posted this before but came across it again and noted camera 326 in fig 3/10 drawing is 3 circles in rectangle box in a perception system which has inward /outward camera fusion capacity.

SYSTEMS AND METHODS FOR DETECTING AND DYNAMICALLY MITIGATING DRIVER FATIGUE
This technology relates to dynamically detecting, managing and mitigating driver fatigue in autonomous systems. For instance, interactions of a driver in a vehicle may be monitored to determine a distance or time when primary tasks associated with operation of the vehicle or secondary tasks issued by the vehicle computing were last performed (610). If primary tasks or secondary tasks are not performed within given distance thresholds or time limits (620), then one or more secondary tasks are initiated by the computing device of the vehicle (630). In another instance, potential driver fatigue, driver distraction or overreliance on an automated driving system is detected based on gaze direction or pattern of a driver. For example, a detected gaze direction or pattern (810) may be compared to an expected gaze direction or pattern given the surrounding environment in a vicinity of the vehicle (860).

[0031] For example, if a scene understanding derived from the vehicle’s perception system indicates that based on naturally occurring stimuli outside the vehicle a driver should be looking in a particular direction, the actual gaze direction or pattern of the driver may then be compared to that particular direction or pattern to provide a measure of the driver’s attentiveness and thus passive fatigue. As a further example, if the driver is making a right turn, the driver would be expected to be mainly looking to the left for oncoming traffic. However, if the detected gaze direction or pattern of the driver indicates that the driver is looking straight ahead that may signal that the driver is no longer sufficiently engaged in the driving task (e.g., due to low vigilance or fatigue). As the system can tell when and where drivers are expected to be looking, it allows for dynamically assessing a driver’s vigilance and/or fatigue passively. The actual gaze direction (or pattern of gaze direction) may be detected through use of cameras within the compartment of the vehicle.
https://worldwide.espacenet.com/publicationDetails/biblio?II=0&ND=3&adjacent=true&locale=en_EP&FT=D&date=20200625&CC=WO&NR=2020131803A1&KC=A1

FIG3/10
https://worldwide.espacenet.com/publicationDetails/mosaics?CC=WO&NR=2020131803A1&KC=A1&FT=D&ND=3&date=20200625&DB=&locale=en_EP

APPARATUS, METHOD, PROGRAM, SIGNAL FOR DETERMINING INTERVENTION EFFECTIVENESS INDEX
A method for determining an intervention effect index for a person executing a task is provided and comprises: obtaining sensing information by at least one sensor coupled to the person, the sensing information includes first sensing information relating to performance in executing the task and second sensing information relating to an emotional state; determining, on the basis of said first sensing information, a performance value difference indicating a variation between performance in executing the task before and after an intervention is applied, the intervention representing an excitation affecting the person; estimating, on the basis of said second sensing information, an emotional value difference indicative of a variation between emotional states before and after the intervention is applied; determining the intervention effect index on the basis of said performance value difference and said emotional value difference, the intervention effect index representing an indication on effectiveness of the intervention on the person.

[0104] In the present example, the performance M indicates how correctly the driving task is executed, which can be determined e.g. by measuring certain driving parameters like how correctly the vehicle follows certain predetermined routes (e.g. comparing how smoothly the actual driving route correspond to an ideal route obtained from a map), how smooth the control of the vehicle is (e.g. whether or how often any sudden change of direction occurs), the degree of the driver recognizing an obstacle, etc. Suitable sensors could be provided, like for instance: a camera similar to (or the same as) the CM camera above described, conveniently installed to point inside and/or outside the vehicle to capture the driven route, the driving pattern, and/or the driver's movements; positioning measurement systems, e.g. to determine the driven route; vehicle speed sensors; vehicle inertial systems for obtaining information on current driving parameters, etc. The information provided by the sensors can also be used, according to other examples, to compare the distance covered over a certain period and an expected distance for a given period, or to determine whether in reaching two points a certain route has been followed compared to predetermined available routes, etc.: the obtained values provide also a measure of the performance M in performing a driving task. When such measurements are performed before and after an intervention, a performance value difference ?M can be determined. The sensors (and information thereby produced by their sensing or capturing) are thus examples of sensors capable of providing first sensing information, on the basis of which the performance variation ?M can be determined.
https://worldwide.espacenet.com/publicationDetails/biblio?CC=US&NR=2020210917A1&KC=A1&FT=D&ND=3&date=20200702&DB=&locale=en_EP

MONITORING TREATMENT COMPLIANCE USING PATIENT ACTIVITY PATTERNS
Methods and systems for monitoring compliance of a patient with a prescribed treatment regimen are described. Patient activity is detected unobtrusively with an activity sensor at the patient location, and activity data is transmitted to a monitoring location. Patient speech detected during use of a communication system such as a mobile telephone by the patient may also be used as an activity signal. Patient activity and/or speech is processed at the patient location or monitoring location to identify activity parameters or patterns that indicate whether the patient has complied with the prescribed treatment regimen. The activity sensor and other components at the patient location may be incorporated into, or associated with, a cell phone, computing system, game system, or vehicle system, for example. The system may provide a report to an interested party, for example a medical care provider or insurance company, regarding patient compliance with the prescribed treatment regimen.

0272] Commercial eye tracking software packages can analyze eye tracking and show the relative probability of eye fixation at particular locations. This allows for a broad analysis of which locations received attention and which ones were ignored. Other behaviors such as blinks, saccades, and cognitive engagement can be reported by commercial software packages. A gaze tracking system for monitoring eye position is available from Seeing Machines Inc., Tucson, Ariz. (see e.g., the Specification Sheet: "faceLAB.TM. 5 Specifications" which is incorporated herein by reference). Eye position, eye rotation, eye gaze position against screen, pupil diameter and eye vergence distance may be monitored. Eye rotation measurements of up to +/-45 degrees around the y-axis and +/-22 degrees around the x-axis are possible. Typical static accuracy of gaze direction measurement is 0.5-1 degree rotational error.
https://www.freepatentsonline.com/20200090805.pdf
https://worldwide.espacenet.com/publicationDetails/biblio?II=17&ND=3&adjacent=true&locale=en_EP&FT=D&date=20200319&CC=US&NR=2020090805A1&KC=A1

One reason why I think SEE will take 50%++ of the DMS market is due to the essential systems requirement to recognise and manage V2V Driver1 line of sight to Driver 2 line of sight to understand driver intention and awareness. Here is another example which is similar to GM imo so I'm guessing they are both SEE related.

IN VEHICLE CONTROL APPARATUS AND METHOD FOR CONTROLLING THE SAME
An in-vehicle control apparatus which executes various functions of a vehicle and a method for controlling the same are provided. The in-vehicle operating apparatus includes an operating unit assembly and a drive unit that is configured to rotate the operating unit assembly about one axis. The operating unit assembly includes a body, a first operating unit disposed at a first side of the body and configured to receive a first command to execute at least a first function, and a second operating unit disposed at a second side of the body opposite to the first side of the body and configured to receive a second command to execute at least a second function.

[0042] In some forms of the present disclosure, when there is a vehicle intervening (cutting in) while a smart cruise control (SCC) function is enabled, early recognizing the vehicle intervening and corresponding to the vehicle intervening to prevent sudden deceleration control or the like and correspond to the vehicle intervening depending on a corresponding intent of a user.
Claims.
1. A vehicle controller, comprising: a processor configured to: detect an intervening vehicle based on a line of sight direction of a user; and respond to the intervening vehicle based on an intent of the user when the intervening vehicle is detected; and a storage configured to store information regarding the line of sight direction of the user or the detected intervening vehicle.
https://worldwide.espacenet.com/publicationDetails/biblio?CC=US&NR=2020324772A1&KC=A1&FT=D&ND=3&date=20201015&DB=&locale=en_EP
https://uspto.report/patent/app/20200255013#:~:text=KIM%20JIN%20KWON

DROWSY DRIVING MANAGEMENT DEVICE, SYSTEM INCLUDING THE SAME, AND METHOD THEREOF
A drowsy driving management device includes: a processor configured to determine whether slow eye movement of a user occurs and whether there is no change in steering torque and to determine that the user drives while drowsy when the slow eye movement of the user occurs and when there is no change in the steering torque for a predetermined period of time; and a storage that stores information indicating whether the slow eye movement occurs and a result of determining whether there is no change in the steering torque.

[0012] The processor may calculate eye movement information with respect to a vehicle based on information about a fixation of the user and information about a head pose of the user based on image data and may calculate eye movement information with respect to an external environment based on the eye movement information with respect to the vehicle.

[0013] The information about the fixation may include a gaze yaw angle and a gaze pitch angle. The information about the head pose may include a head pose yaw angle and a head pose pitch angle.

[0014] The processor may calculate an eye movement yaw angle with respect to the vehicle in the eye movement information with respect to the vehicle by subtracting the head pose yaw angle from the gaze yaw angle and may calculate an eye movement pitch angle with respect to the vehicle in the eye movement information with respect to the vehicle by subtracting the head pose pitch angle from the gaze pitch angle.

[0015] The processor may calculate an eye movement yaw angle with respect to the external environment in the eye movement information with respect to the external environment using the eye movement yaw angle with respect to the vehicle and a vehicle yaw rate and may calculate an eye movement pitch angle with respect to the external environment in the eye movement information with respect to the external environment using the eye movement pitch angle with respect to the vehicle and a vehicle pitch angle.

[0016] The processor may calculate an eye movement distance during the predetermined period of time using a value obtained by adding the eye movement yaw angle with respect to the external environment and the eye movement pitch angle with respect to the external environment and may determine that the slow eye movement occurs, when the eye movement distance is less than a predetermined reference value.
https://worldwide.espacenet.com/publicationDetails/biblio?CC=US&NR=2020317208A1&KC=A1&FT=D&ND=3&date=20201008&DB=&locale=en_EP
https://www.freepatentsonline.com/20200317208.pdf

Colin Barden
In my most recent story in EETimes, I speculated that Qualcomm may be working on an integrated ADAS/infotainment domain controller. These 5 links provide details of why I believe this is a well developed program, with General Motors top candidate to be the first announced automaker, possibly as soon as December 1-2:
https://www.linkedin.com/in/colin-barnden-1081376/detail/recent-activity/

JC post Patent Stacker 10Jun2020

SEE.
SYSTEM AND METHOD FOR IDENTIFYING A CAMERA POSE OF A FORWARD FACING CAMERA IN A VEHICLE
A method includes capturing images of a vehicle driver's face from a driver facing camera and images of a forward road scene from a forward facing camera. The images are analyzed to derive gaze direction data in a vehicle frame of reference. The gaze direction data are statistically collated to determine a frequency distribution of gaze angles. One or more peaks in the frequency distribution are identified and associated with corresponding reference points in the images to determine one or more reference gaze positions in the vehicle frame of reference. The one or more reference gaze positions are correlated with a position of the reference points in a forward facing camera frame of reference to to determine a camera pose of a forward facing camera in the vehicle frame of reference.

GM
METHODS AND SYSTEMS FOR PROCESSING DRIVER ATTENTION DATA
Methods and systems are provided for processing attention data. In one embodiment, a method includes: receiving, by a processor, object data associated with at least one object of an exterior environment of the vehicle; receiving upcoming behavior data determined from a planned route of the vehicle; receiving gaze data sensed from an occupant of the vehicle; processing, by the processor, the object data, the upcoming behavior data, and the gaze data to determine an attention score associated with an attention of the occupant of the vehicle; and selectively generating, by the processor, signalsto at least one of notify the occupant and control the vehicle based on the attention score.

Qualcomm
EARLY FUSION OF CAMERA AND RADAR FRAMES
Disclosed are techniques for fusing camera and radar frames to perform object detection in one or more spatial domains. In an aspect, an on-board computer of a host vehicle receives, from a camera sensor of the host vehicle, a plurality of camera frames, receives, from a radar sensor of the host vehicle, a plurality of radar frames, performs a camera feature extraction process on a first camera frame of the plurality of camera frames to generate a first camera feature map, performs a radar feature extraction process on a first radar frame of the plurality of radar frames to generate a first radar feature map, converts the first camera feature map and/or the first radar feature map to a common spatial domain, and concatenates the first radar feature map and the first camera feature map to generate a first concatenated feature map in the c

V2

System, Method, and Program for Predicting Diminished Attentiveness State, and Storage Medium on Which Program Is Stored
A reduced attention state estimation system includes an eyeball movement and eyelid activity measurement unit that measures an eyeball movement and an eyelid activity of a subject to obtain eyeball movement and eyelid activity data, a section determination unit that determines an eye opening section, an eye closing section, an eye-blinking section and a cluster section based on the eyeball movement and eyelid activity data, an eyeball movement and eyelid activity-related information calculation unit that calculates a sharpness of microsaccade or the like for each eye opening section based on the eyeball movement and eyelid activity data, and an attention assessment unit that determines an attention assessment or the like for an eye opening/cluster section, which is an eye opening section and cluster section, based on the sharpness of microsaccade or the like.
https://worldwide.espacenet.com/publicationDetails/biblio?CC=US&NR=2020218935A1&KC=A1&FT=D&ND=3&date=20200709&DB=&locale=en_EP
https://www.freepatentsonline.com/20200218935.pdf

A reminder of some of the other closely dated Aircraft / Cokepit Patents which may or may not be related and lastly a list of the CAST members.

AIRBUS OPERATIONS SAS 2020-06-11
AIRCRAFT ****PIT AND METHOD OF DISPLAYING IN AN AIRCRAFT ****PIT

GE AVIAT SYSTEMS 2020-06-11
AIRCRAFT AND METHOD OF CONTROLLING

GE AVIAT SYSTEMS LTD 2020-06-17
METHOD OF ASSESSING A PILOT EMOTIONAL STATE

GE AVIATION SYSTEMS LLC 2020-06-11
AIRCRAFT AUGMENTED REALITY SYSTEM AND METHOD OF OPERATING

AIRBUS OPERATIONS SAS 2020-06-11
AIRCRAFT ****PIT AND METHOD OF DISPLAYING IN AN AIRCRAFT ****PIT

THALES SA 2020-06-11
METHOD AND SYSTEM FOR DISPLAY AND INTERACTION EMBEDDED IN A ****PIT

BAE SYSTEMS PLC 2020-07-09
APPARATUS AND METHOD FOR DEFINING AND INTERACTING WITH REGIONS OF AN OPERATIONAL AREA

BAE 2020-07-09
EAD-MOUNTED DISPLAY AND CONTROL APPARATUS AND METHOD

BOEING CO 2020-07-09
SYSTEM AND METHOD FOR FOVEATED SIMULATION

Industry CAST Members (Commercial Aviation Safety Team)

Aerospace Industries Association
Airbus
Airports Council International
Airlines for America
The Boeing Company
Flight Safety Foundation
General Electric (representing all engine manufacturers)
National Air Carrier Association
Regional Airline Association
https://www.faa.gov/news/fact_sheets/news_story.cfm?newsId=23035

Latest Share Chat

Member Info for johnchucka

Followers

Following

Posts (Last 30 Days)

Posts (All Time)

Last Post

Member Since

Share Chat Post Distribution (Last 30 Days)

Activity Log (Last 30 Days)

Latest Share Chat

Member Info for johnchucka

Followers

Following

Posts (Last 30 Days)

Posts (All Time)

Last Post

Member Since

Share Chat Post Distribution (Last 30 Days)

Activity Log (Last 30 Days)

Please enter at least 2 characters

No results found

Search results

Recently viewed

Trending