Member Info for johnchucka

Cognitive Heat Map: A Model for Driver Situational Awareness
A system includes a camera configured to capture image data of an environment, a monitoring system configured to generate a gaze sequences of a subject, and a computing device communicatively coupled to the camera and the monitoring system. The computing device is configured to receive the image data from the camera and the gaze sequences from the monitoring system, implement a machine learning model comprising a convolutional encoder-decoder neural network configured to process the image data and a side-channel configured to inject the gaze sequences into a decoder stage of the convolutional encoder-decoder neural network, generate, with the machine learning model, a gaze probability density heat map, and generate, with the machine learning model, an attended awareness heat map.
https://worldwide.espacenet.com/publicationDetails/biblio?CC=US&NR=2022153278A1&KC=A1&FT=D&ND=3&date=20220519&DB=&locale=en_EP
https://www.freepatentsonline.com/20220153278.pdf

PREDICTIVE IMPAIRMENT MONITOR SYSTEM AND METHOD
A driver monitor system and method for predicting impairment of a user of a vehicle. The system includes video cameras, an input device for inputting a list of medications being taken by the driver. Processing circuitry predicts side effects of the medications based on the half-life of the medication, detecting eye gaze movement, eye lid position, and facial expression of the user using images from the video camera, predicting whether the user is transitioning into an impaired physical state that is a side effect of the medications, verifying the side effect of the medications, determining whether the user is fit to drive using the verified side effects of the medications, and outputting to the vehicle an instruction to operate the vehicle in a level of automation that makes up for the at least one side effect or to perform a safe pull over operation of the vehicle.
https://worldwide.espacenet.com/publicationDetails/biblio?II=1&ND=3&adjacent=true&locale=en_EP&FT=D&date=20220519&CC=US&NR=2022153302A1&KC=A1
https://www.freepatentsonline.com/20220153302.pdf

Driver assistance system for drivers using bioptic lenses
Assistance is provided to a driver of a road vehicle using bioptic eyewear which has a carrier lens and a telescopic lens. Presence of bioptic eyewear on the driver is detected while driving the vehicle. Spatial positioning of the driver with bioptic eyewear is tracked to identify a telescopic event in which the driver is looking through the telescopic lens. A gaze direction through the telescopic lens is detected during the telescopic event. Remote sensing data is collected covering a region of interest corresponding to a projection of the gaze direction. An object is detected within the region of interest. A driving indicator is identified according to the object. The driving indicator is presented to the driver as supplemental information to highlight key driving indicators such as identity of signage and color of a traffic light.

The invention increases situational awareness for a user of a bioptic device by detecting events when the telescopic lens is being used based on in-vehicle monitoring by a driver-state monitoring camera. Upon detection of the usage of the telescopic lens, the gaze of the eye(s) looking through the telescopic lens(es) is analyzed to determine the direction of the gaze. The gaze direction points to a region of interest in the distance outside the vehicle. Remote sensors in the vehicle (e.g., a forward looking camera) identify the surrounding scene. Particular objects and/or key driving indicators (e.g., traffic directives such as color of traffic lights or traffic signage) are automatically identified using known methods such as pattern recognition and image analysis. An HMI interface (audio and/or visual) provide situational awareness by communicating the driving indicators to the driver. To improve the robustness of detecting telescopic lens use and gaze direction which are based on monitoring the positions of the driver's head and their bioptic device, a customization or configuration process can be utilized to calibrate a monitoring system according to each user.
https://uspto.report/patent/grant/11,328,591#:~:text=gaze
https://www.freepatentsonline.com/11328591.pdf
https://patents.justia.com/patent/11328591

[0107] In an embodiment, vehicle 700 may contain one or more camera(s) 735. In an embodiment, the camera(s) 735 may comprise a camera sensor and mounting assembly. Different mounting assemblies may be used for different cameras on vehicle 700. For example, front facing cameras may be mounted in the front bumper, in the stem of the rear-view mirror assembly or in other front facing areas of the vehicle 700. Rear facing cameras may be mounted in the rear bumper/fender, on the rear windshield, on the trunk or other rear facing areas of the vehicle. The side facing mirrors may be mounted on the side of the vehicle such as being integrated into the mirror assembly or door assemblies. The cameras may provide object detection and distance estimation, particularly for objects of known size and/or shape (e.g., a stop sign and a license plate both have standardized size and shape) and may also provide information regarding rotational motion relative to the axis of the vehicle such as during a turn. When used in concert with the other sensors, the cameras may both be calibrated through the use of other systems such as through the use of LIDAR, wheel tick/distance sensors, and/or GNSS to verify distance traveled and angular orientation. The cameras may similarly be used to verify and calibrate the other systems to verify that distance measurements are correct, for example by calibrating against known distances between known objects (landmarks, roadside markers, road mile markers, etc.) and also to verify that object detection is performed accurately such that objects are accordingly mapped to the correct locations relative to the car by LIDAR and other system. Similarly, when combined with, for example, accelerometers, impact time with road hazards, may be estimated (elapsed time before hitting a pot hole for example) which may be verified against actual time of impact and/or verified against stopping models (for example, compared against the estimated stopping distance if attempting to stop before hitting an object) and/or maneuvering models (verifying whether current estimates for turning radius at current speed and/or a measure of maneuverability at current speed are accurate in the current conditions and modified accordingly to update estimated parameters based on camera and other sensor measurements).

[0108] In some embodiments, at least some of camera(s) 735 may be inward facing. The camera(s) 735 may be utilized to capture one or more images from which vehicle data may be derived (e.g., an image of the speedometer from which speed can be derived, an image of a heading indicator from which a heading may be derived, etc.). In some embodiments, camera(s) 735 may be utilized to capture one or more images of at least some portion of one or more of the vehicle occupants

TLS,
Do you think this maybe relevant to the Seeing Machines patent PHONE DOCKING STATION FOR ENHANCED DRIVING SAFETY ?

https://worldwide.espacenet.com/publicationDetails/biblio?II=14&ND=3&adjacent=true&locale=en_EP&FT=D&date=20180125&CC=US&NR=2018026669A1&KC=A1

TECHNIQUES FOR UTILIZING A MOBILE DEVICE AS A PROXY FOR A VEHICLE
Techniques described herein include utilizing a mobile device as a proxy receiver for a vehicle such that a driver of the vehicle can be presented driving assistance information based on messages received by the mobile device. The mobile device can determine that it is temporally located with the vehicle. If so, the mobile device can be configured to receive messages from other entities (e.g., vehicles, roadside units, traffic signals, and the like) in a vehicle-to-everything network. On reception, the mobile device may determine whether a received data message is relevant to the vehicle. In response to determining that the data message is, in fact, relevant to the vehicle, the mobile device may provide driving assistance information that is generated based at least in part on the information provided in the received data message.

[0003] Legacy vehicles, which may or may not include vehicles with sensors and/or ADAS capabilities, may lack V2X capabilities and, thus, may not take advantage of the valuable data being communicated, or their involvement may be suboptimal. Many benefits of a vehicle-to-everything (V2X) environment depend on knowing the accurate inter-vehicle distances and relative locations as well as on having environmental and situational awareness. This type of location and location-related measurements are regularly communicated between vehicles, for example, through V2X using messages such as a Basic Safety Message (BSM). Both legacy vehicle drivers and autonomous (or semi-autonomous) vehicle drivers can benefit from this shared interaction and communication.
https://worldwide.espacenet.com/publicationDetails/biblio?CC=US&NR=2022028264A1&KC=A1&FT=D&ND=3&date=20220127&DB=&locale=en_EP
https://www.freepatentsonline.com/20220028264.pdf

SYSTEMS AND METHODS FOR MOVING ONE OR MORE DISPLAYS IN A VEHICLE BASED ON VEHICLE OCCUPANT LOCATION
A vehicle comprising a sensor, a moveable display, and a controller. The controller is configured to determine a location of an occupant within the vehicle and, based on the location of the occupant within the vehicle, move the moveable display.
https://worldwide.espacenet.com/publicationDetails/biblio?II=41&ND=3&adjacent=true&locale=en_EP&FT=D&date=20220505&CC=US&NR=2022134877A1&KC=A1
https://www.freepatentsonline.com/20220134877.pdf

EYE-GAZE INPUT APPARATUS
An eye-gaze input apparatus includes a hardware processor functioning as an eye-gaze determination unit, an input/output processing unit, and a careless-state determination unit. The eye-gaze determination unit determines a first input by an eye gaze of a user with respect to first images of input elements displayed in a display region set in front of a windshield and a driver seat of a vehicle. The careless-state determination unit determines whether the user is in a careless state. The input/output processing unit confirms the first input in a case where the eye-gaze determination unit determines that there is the first input on any of the one or more first images and the careless-state determination unit determines that the user is not in the careless state, and does not confirm the first input in a case where the careless-state determination unit determines that the user is in the careless state.
https://worldwide.espacenet.com/publicationDetails/biblio?II=9&ND=3&adjacent=true&locale=en_EP&FT=D&date=20220505&CC=US&NR=2022135050A1&KC=A1
https://www.freepatentsonline.com/20220135050.pdf

A CAMERA SYSTEM AND METHOD FOR GENERATING AN EYE CONTACT IMAGE VIEW OF A PERSON
The disclosure relates to a camera system (100) for generating an eye contact image view of a person with a gaze direction that is offset from a camera position, the camera system (100) comprises a camera (10a, 10b, 10c, 10d) configured to obtain image data of the person, a processing circuitry (102a, 102b, 102c) configured to be operatively connected to the camera (10a, 10b, 10c, 10d) and configured to obtain a camera view of the person from a camera position, and generate a synthetic camera view of the person from a virtual camera position. The disclosure further relates to a method for generating an eye contact image view of a person with a gaze direction that is offset from a camera position and a processing circuitry program product (500).
https://worldwide.espacenet.com/publicationDetails/biblio?II=3&ND=3&adjacent=true&locale=en_EP&FT=D&date=20220505&CC=WO&NR=2022089222A1&KC=A1
https://www.freepatentsonline.com/WO2022089222A1.pdf

SYSTEM AND METHOD TO DETERMINE RESPONSIVENESS OF A DRIVER OF A VEHICLE TO FEEDBACK REGARDING DRIVING BEHAVIORS
This disclosure relates to a system and method for determining responsiveness of a driver of a vehicle to feedback regarding driving behaviors. The system may include a sensor configured to generate output signals conveying first driving behavior information, which may characterize operation of the vehicle by the driver. The system may include one or more processors configured to obtain the first driving behavior information. The one or more processors may effectuate provision of feedback defined by feedback information based on the first driving behavior. The sensor may be configured to output signals conveying second driving behavior information, which may characterize operation of the vehicle by the driver during and/or subsequent to the provision of the feedback. The one or more processors may be configured to obtain the second driving behavior information and assess responsiveness of the driver to the feedback based on the second driving behavior information.

Driver status information may include one or more driver status information regarding the driver of vehicle 11. By way of non-limiting example, driver status information regarding the driver may include whether the driver is alert, whether driver is sleepy or sleeping, whether driver is using a headphone, whether driver is watching the road, whether the driver is distracted, whether the driver is watching TV, whether the driver is eating, whether the driver is driving with one hand, whether the driver is talking on the phone, the driver's historical accident information, the driver's historical traffic violation information, the driver's historical driving pattern information, and/or other driver status information.

Driver status information may be generated by one or more sensors. By way of non-limiting example, an image sensor may count the number of time vehicle 11 swerves within a certain duration of time to determine whether the driver is sleepy or sleeping. For example, an image sensor may be used to track the driver's eye or head movements and a physiological sensor may be used to track the driver's heart rate to determine whether the driver is sleepy or sleeping. As another example, user input devices (e.g., keyboard, keypad, touchscreen, mouse, etc.) may be used to identify the driver.
https://worldwide.espacenet.com/publicationDetails/biblio?CC=US&NR=2022126839A1&KC=A1&FT=D&ND=3&date=20220428&DB=&locale=en_EP
https://www.freepatentsonline.com/20220126839.pdf
https://www.youtube.com/channel/UC8xjpwsT5vsxywcyqjXbNkQ

DW, I would say that from my very shallow skim of the patent world that the core technology is keenly scrutinised as we would expect and that's where the critical nuances are debated between interested parties but once it migrates into systems where it is one of many complimentary moving parts those patented systems start to look to have very homogenous presentations but differently described.

SYSTEMS AND METHODS FOR DETECTING SYMPTOMS OF OCCUPANT ILLNESS
Systems and methods for detecting symptoms of occupant illness is disclosed herein. In embodiments, a storage is configured to maintain a visualization application and data from one or more sources, such as an audio source, an image source, and/or a radar source. A processor is in communication with the storage and a user interface. The processor is programmed to receive data from the one or more sources, execute human-detection models based on the received data, execute activity-recognition models to recognize symptoms of illness based on the data from the one or more sources, determine a location of the recognized symptoms, and execute a visualization application to display information in the user interface. The visualization application can show a background image with an overlaid image that includes an indicator for each location of recognized symptom of illness. Additionally, data from the audio source, image source, and/or radar source can be fused.
https://worldwide.espacenet.com/publicationDetails/biblio?II=22&ND=3&adjacent=true&locale=en_EP&FT=D&date=20220428&CC=US&NR=2022129687A1&KC=A1
https://www.freepatentsonline.com/20220129687.pdf

Object notification systems for identifying and notifying vehicles of relevant objects
As noted above, the primary vehicle system 102 includes the one or more interior sensors 126 provided within an interior of the primary vehicle. It should be appreciated that the one or more interior sensors 126 may include the same components and capabilities as the exterior sensors 124. However, the one or more interior sensors 126 are provided within an interior of the primary vehicle and directed within an occupant compartment of the primary vehicle to detect a direction of a gaze of an occupant of the primary vehicle. Accordingly, the interior sensors 126 are configured to capture data of a gaze of one or more occupants of the primary vehicle. The data is then communicated to the one or more processors 132 to determine a direction of the gaze of the one or more occupants. In response to an object being identified by the one or more exterior sensors 124, the one or more processors 132 determine whether the one or more occupants are gazing in a direction of the object.

At step 416, the object data captured by the exterior sensors 124 is analyzed to confirm that the relevant object 310 is present and correctly identified.

After the relevant object 310 is identified, either at step 408 or at step 416, and determined to be within the notification zone 308 of primary vehicle 302, the interior sensors 126 of the primary vehicle 302 are utilized at step 418 to detect the gaze 306 of the associated occupant for whom the object 310 was deemed relevant. After detecting the gaze 306 of the occupant, it is determined at step 420 whether the gaze 306 of the occupant is directed in a direction matching a direction of the relevant object 310 in relation to the primary vehicle 302. As referred to herein, when the direction of the occupant's gaze 306 is determined to “match” a direction of the relevant object 310, it is determined that the occupant is likely to be looking at, or at least in the direction of, the relevant object 310. Further, it may be determined that the occupant's gaze 306 matches the direction of the object 310 if an angle of the occupant's gaze 306 is within a threshold angle of the direction of the relevant object 310 such as, for example, 10 degrees, 20 degrees, 30 degrees, or the like. Similarly, it may be determined that the occupant's gaze 306 matches the direction of the relevant object 310 if movement of the occupant's gaze 306 appears to correspond with movement of the relevant object 310 as the primary vehicle 302 passes the relevant object 310.
https://worldwide.espacenet.com/publicationDetails/description?CC=US&NR=11312300B1&KC=B1&FT=D&ND=3&date=20220426&DB=&locale=en_EP
https://www.freepatentsonline.com/11312300.pdf

VEHICLE OCCUPANT GAZE DETECTION
A computer includes a processor and a memory, the memory storing instructions executable by the processor to determine respective probabilities of a direction of a gaze of a vehicle occupant toward each of a plurality of points in an image, determine a gaze distance from a center of the image based on the probabilities, and, upon determining that the gaze distance exceeds a threshold, suppress manual control of at least one vehicle component.
https://worldwide.espacenet.com/publicationDetails/biblio?CC=US&NR=2022129686A1&KC=A1&FT=D&ND=3&date=20220428&DB=&locale=en_EP
https://www.freepatentsonline.com/20220129686.pdf
https://patents.justia.com/patent/20220129686

VEHICLE BEHAVIORAL MONITORING
Vehicle behavioral monitoring includes determining a measure of distraction of the operator of a target vehicle, characterizing the type or category of distraction, determining level of risk that the target vehicle poses, and invoking various responses including host vehicle notifications and evasive actions and external notification and information sharing.
[0001] This disclosure is related to situational awareness in road vehicles.
[0002] Vehicle systems are known to monitor the region surrounding the vehicle for improving a vehicle operator's situational awareness. Common examples include forward and rear range, range-rate and vision systems. Such systems may be enablers in autonomous and semi-autonomous vehicle controls, for example adaptive cruise controls, assisted parking, lane departure warning, lane keeping and blind spot warnings.
[0003] Vehicle systems are proposed to monitor a vehicle operator. Such systems may employ vision systems, infrared sensors, gyroscopic sensor and other technologies to monitor eye gaze, eye blinking/closure, head posture and other metrics of the vehicle's operator. These metrics may be used in determining vehicle operator distraction, drowsiness, fatigue and the like.
https://worldwide.espacenet.com/publicationDetails/description?CC=US&NR=2022111867A1&KC=A1&FT=D&ND=3&date=20220414&DB=&locale=en_EP
https://www.freepatentsonline.com/20220111867.pdf

TLS,
Do you foresee the OMS/DMS being incorporated into this assembly to create triggers for the system to prompt, alert and ensure the driver has seen external vehicles etc since as I read it the external cameras feed to a screen on the rear-view mirror?

VEHICULAR FULL MIRROR DISPLAY SYSTEM WITH AUXILIARY FORWARD AND REARWARD VIEWS
A vehicular display system includes an electronic control unit (ECU), a plurality of cameras including a driver-side rear corner camera and a passenger-side rear corner camera disposed at respective driver-side and passenger-side rear corner regions of the vehicle. The ECU includes an image processor operable to process image data captured by any of the plurality of cameras and generate video images. A video mirror display screen is disposed at an interior rearview mirror assembly of the vehicle and is operable to display video images provided by the ECU. Responsive to a triggering event, the ECU, via processing of image data captured by at least one of the driver-side and passenger-side rear corner cameras, generates cross-traffic view video images and provides the cross-traffic view video images to the video mirror display screen. The cross-traffic view video images are displayed in a split-screen format at the video mirror display screen.
https://worldwide.espacenet.com/publicationDetails/biblio?CC=US&NR=2022118915A1&KC=A1&FT=D&ND=3&date=20220421&DB=&locale=en_EP
https://www.freepatentsonline.com/20220118915.pdf

POWER CONTROL BASED AT LEAST IN PART ON USER EYE MOVEMENT
In some aspects, a user device may receive, from a vision sensor of a user device, first image data associated with a first set of images. The user device may determine, using an image processing model, that a set of images depict a particular type of eye activity of an eye of a user. The user device may cause, based at least in part on determining that the first set of images depict the particular type of eye activity, a power level of a camera of the user device to be reduced.
https://worldwide.espacenet.com/publicationDetails/biblio?II=32&ND=3&adjacent=true&locale=en_EP&FT=D&date=20220421&CC=US&NR=2022124242A1&KC=A1
https://www.freepatentsonline.com/20220124242.pdf

TLS..could these be related?

GM GLOBAL TECH OPERATIONS LLC / Pub -2022-04-21 - Filling 21.10.2020
FACILITATING TRANSFERS OF CONTROL BETWEEN A USER AND A VEHICLE CONTROL SYSTEM
A system for facilitating transfers of control includes an input module configured to acquire a set of user preferences relating control states to contexts, the control states including at least a manual control state, and an autonomous and/or semi-autonomous control state. The system includes a processing device configured to automatically perform, during operation of a semiautonomous system, generating a transfer of control (TOC) policy based on the set of user preferences and a current context state, the TOC policy prescribing transitions between control states in response to actions. The processing device is also configured to perform, based on an action performed by the user or a control system, determining whether to perform a transition from a current control state to a second control state, and in response to the TOC policy prescribing the transition, transitioning from the current control state to the second control state.
https://worldwide.espacenet.com/publicationDetails/biblio?CC=US&NR=2022119015A1&KC=A1&FT=D&ND=3&date=20220421&DB=&locale=en_EP
https://register.epo.org/ipfwretrieve?apn=US.202017075996.A&lng=en
https://www.freepatentsonline.com/20220119015.pdf

NVIDIA CORP / Pub -2022-04-21 - Filling 21.10.2020
OCCUPANT ATTENTIVENESS AND COGNITIVE LOAD MONITORING FOR AUTONOMOUS AND SEMI-AUTONOMOUS DRIVING APPLICATIONS
In various examples, estimated field of view or gaze information of a user may be projected external to a vehicle and compared to vehicle perception information corresponding to an environment outside of the vehicle. As a result, interior monitoring of a driver or occupant of the vehicle may be used to determine whether the driver or occupant has processed or seen certain object types, environmental conditions, or other information exterior to the vehicle. For a more holistic understanding of the state of the user, attentiveness and/or cognitive load of the user may be monitored to determine whether one or more actions should be taken. As a result, notifications, AEB system activations, and/or other actions may be determined based on a more complete state of the user as determined based on cognitive load, attentiveness, and/or a comparison between external perception of the vehicle and estimated perception of the user.
https://worldwide.espacenet.com/publicationDetails/biblio?CC=US&NR=2022121867A1&KC=A1&FT=D&ND=3&date=20220421&DB=&locale=en_EP
https://register.epo.org/ipfwretrieve?apn=US.202017076690.A&lng=en
https://www.freepatentsonline.com/20220121867.pdf

OCCUPANT ATTENTIVENESS AND COGNITIVE LOAD MONITORING FOR AUTONOMOUS AND SEMI-AUTONOMOUS DRIVING APPLICATIONS
In various examples, estimated field of view or gaze information of a user may be projected external to a vehicle and compared to vehicle perception information corresponding to an environment outside of the vehicle. As a result, interior monitoring of a driver or occupant of the vehicle may be used to determine whether the driver or occupant has processed or seen certain object types, environmental conditions, or other information exterior to the vehicle. For a more holistic understanding of the state of the user, attentiveness and/or cognitive load of the user may be monitored to determine whether one or more actions should be taken. As a result, notifications, AEB system activations, and/or other actions may be determined based on a more complete state of the user as determined based on cognitive load, attentiveness, and/or a comparison between external perception of the vehicle and estimated perception of the user.
https://worldwide.espacenet.com/publicationDetails/biblio?II=2&ND=3&adjacent=true&locale=en_EP&FT=D&date=20220421&CC=US&NR=2022121867A1&KC=A1
https://www.freepatentsonline.com/20220121867.pdf

INTERFACE SHARPNESS DISTRACTION MITIGATION METHOD AND SYSTEM
A driver monitor system and method of optimizing use of cognitive resources of a user in a vehicle is disclosed. The system includes in-cabin video cameras, a display device, and processing circuitry. The method performed by the processing circuitry includes monitoring eye gaze direction using images from the in-cabin video cameras, gradually softening image regions and lines displayed in the display device that are outside a first vision area of the user of the vehicle, and using the eye gaze direction to predict whether the user is transitioning into a new focus vision area that had been outside the first vision area. When the user's eyes begin to move, begin sharpening softened image regions and lines such that by the time eyes have shifted into the new focus vision area, the image regions and lines in the new focus vision area reach full sharpness.
https://worldwide.espacenet.com/publicationDetails/biblio?II=3&ND=3&adjacent=true&locale=en_EP&FT=D&date=20220421&CC=US&NR=2022121865A1&KC=A1
https://www.freepatentsonline.com/20220121865.pdf