Member Info for johnchucka

HUMAN-ASSISTED MACHINE LEARNING THROUGH GEOMETRIC MANIPULATION AND REFINEMENT
Systems and methods are provided for feature detection such that users can apply advanced machine learning and artificial intelligence (AI) without the need for a deep understanding of existing algorithms and techniques. Embodiments of the present disclosure provide systems and methods than enable easy access to a suite of machine learning algorithms and techniques, an intuitive interface for training an AI to recognize image features based on geometric "correct and refine" recursion, and real-time visualizations of training effectiveness.
https://worldwide.espacenet.com/publicationDetails/biblio?II=3&ND=3&adjacent=true&locale=en_EP&FT=D&date=20200305&CC=WO&NR=2020047466A1&KC=A1#

VEHICLE HUMAN MACHINE INTERFACE IN RESPONSE TO STRAINED EYE DETECTION
A vehicle system in a vehicle, comprising a first sensor configured to survey an environment proximate to the vehicle, including an object outside of the vehicle, a second sensor located within a vehicle cabin and configured to obtain data indicative of facial information of an occupant of the vehicle, and a processor in communication with the first and second sensors and programmed to output information regarding the object on a display of the vehicle system in response to the data indicative of the facial information indicating that the occupant is viewing the object outside of the vehicle.
https://patents.justia.com/patent/20200074197
http://www.freepatentsonline.com/20200074197.pdf

Last week, we showcased Collaborative Driving in Stockholm traffic. Hands off the steering wheel, eyes still on the road - safe and convenient (and fun!)

https://www.linkedin.com/posts/veoneer_last-week-we-showcased-collaborative-driving-activity-6639202023965282304-_p7u

https://www.linkedin.com/company/veoneer/

100 years of GM
Guardian Angel on City streets @ 1.15m
https://www.gm.com/masthead-story/future-transportation-vision.html

REAL-TIME STEREO CALIBRATION BY DIRECT DISPARITY MINIMIZATION AND KEYPOINT ACCUMULATION
A stereoscopic imaging device is configured to capture multiple corresponding images of objects from a first camera and a second camera. The stereoscopic imaging device can determine multiple sets of keypoint matches based on the multiple corresponding images of objects, and can accumulate the keypoints. In some examples, the stereoscopic imaging device can determine a vertical disparity between the first camera and the second camera based on the multiple sets of keypoint matches. In some examples, the stereoscopic imaging device can determine yaw errors between the first camera and the second camera based on the sets of keypoint matches, and can determine a yaw disparity between the first camera and the second camera based on the determined yaw errors. The stereoscopic imaging device can generate calibration data to calibrate one or more of the first camera and the second camera based on the determined vertical disparity and/or yaw disparity.
https://patents.justia.com/patent/20200077073
Drawings
http://www.freepatentsonline.com/20200077073.pdf

LOAD REDUCTION IN A VISUAL RENDERING SYSTEM
In one implementation, an electronic visual-rendering device includes an eye-tracking sensor and at least a first component. The eye-tracking sensor is configured to detect an eye-close event and, in response, output an eye-close-event message. The first component is configured to operate in at least a normal-power mode and a first low-power mode. The first components is configured to transition from operating in the normal-power mode to operating in the first low-power mode in response to the eye-tracking sensor's output of the eye-close-event message.

SUMMARY
The following presents a simplified summary of one or more embodiments to provide a basic understanding of such embodiments. This summary is not an extensive overview of all contemplated embodiments, and is not intended to either identify key critical elements of all embodiments or delineate the scope of all embodiments. The summary's sole purpose is to present some concepts of one or more embodiments in a simplified form as a prelude to the more detailed description that is presented later.
In one embodiment, an electronic visual-rendering device comprises an eye-tracking sensor and a first component. The eye-tracking sensor is configured to detect an eye-close event and, in response, output an eye-close-event message. The first component is configured to operate in at least a normal-power mode and a first low-power mode. The first component is configured to transition from operating in the normal-power mode to operating in the first low-power mode in response to the eye-tracking sensor's output of the eye-close-event message.

In another embodiment, a method for an electronic visual-rendering device comprises detecting, by an eye-tracking sensor, an eye-close event, outputting, by the eye-tracking sensor, an eye-close-event message in response to the detecting of the eye-close event, operating a first component in a normal-power mode, and transitioning the first component from operating in the normal-power mode to operating in a first low-power mode in response to the eye-tracking sensor outputting the eye-close-event message.

In yet another embodiment, a system comprises means for electronic visual-rendering, means for detecting an eye-close event, means for outputting an eye-close-event message in response to detecting the eye-close event, means for operating a first component in a normal-power mode, and means for transitioning the first component from operating in the normal-power mode to operating in a first low-power mode in response to the outputting of the eye-close-event message.
https://patents.justia.com/patent/20200073465

Drawings
http://www.freepatentsonline.com/20200073465.pdf

VISUAL ASSISTANCE AND CONTROL SYSTEM FOR A WORK MACHINE
A work machine includes a frame and a set of ground engaging elements movably supported by the frame and driven by an engine to drive movement of the machine in a worksite. The machine includes a location sensor configured to generate a machine location sensor signal indicative of a location of the mobile work machine. The machine includes a communication component that communicates with a worksite server and retrieves object location data. The machine also includes virtual model generator logic that determines the object is within a field of view of an operator of the machine, based on the machine location sensor signal and object location data, and generates an augmentation indication indicative of the determination. The machine includes augmentation logic that generates and displays an augmented reality overlay, based on the augmentation indication, and displays an indication of the object proximate the object within the field of view of the operator.

Operator sensor(s) 151 sense a characteristic or location of operator 350 in operator cab 101. Operator sensor(s) 151 can include position sensors, orientation sensors, eye tracking sensors, etc. Operator sensor(s) 151 may generate sensor signals useful for generating augmented displays on augmented display device 128. For example, to properly augment the view of operator 350, such that generated indications appear in the correct locations (or perspective), it is helpful to know where the eyes of operator 350 are positioned and oriented (relative to the augmented display device 128).
https://patents.justia.com/patent/20200071912
https://worldwide.espacenet.com/publicationDetails/biblio?II=9&ND=3&adjacent=true&locale=en_EP&FT=D&date=20200305&CC=US&NR=2020071912A1&KC=A1#
Drawings
http://www.freepatentsonline.com/20200071912.pdf

SYSTEM AND METHOD FOR AUTOMATICALLY TRIGGERING INCIDENT INTERVENTION
A method includes determining, based at least in part on sensor data about a worksite, first and second conditions associated with a location at the worksite. The method also includes determining, based at least in part on the first condition and the second condition, a first risk factor associated with a first portion of the worksite, the first portion including the location and area proximate the location and determining a second risk factor associated with a second portion of the worksite adjacent the first portion. The method may also include generating a graphical user interface comprising a graphical representation of the first portion of the worksite and the second portion of the worksite and visual indications of the risk factors. Further aspects may include controlling machines and/or electronic devices at the worksite when a value associated with the risk factors is greater than a threshold value.
https://patents.justia.com/patent/20200074383
https://worldwide.espacenet.com/publicationDetails/biblio?II=11&ND=3&adjacent=true&locale=en_EP&FT=D&date=20200305&CC=US&NR=2020074383A1&KC=A1#
Drawings
http://www.freepatentsonline.com/20200074383.pdf

INDUSTRIAL APPLICABILITY
Alertness monitoring is crucial in many operating environments, especially those where heavy machinery, vehicles, or dangerous equipment is in use. The present document discusses alertness monitoring, instead of fatigue monitoring, to provide a positive connotation of measuring how alert a person is, rather than how fatigued they are. Supervisors, crew, and other people are able to review visual representations of alertness to recognize operators needing help and to compare operators over different time periods. The alertness score may be used for analytics with other data such as productivity data, safety data, or equipment telematics data. Additionally, the alertness score may be used to initiate remedial responses to ensure the safety of the operator and of others. The remedial responses may also avoid or reduce damage to equipment, which will reduce downtime, lost profits, or scheduling conflicts.

This disclosure includes all modifications and equivalents of the subject matter recited in the claims appended hereto as permitted by applicable law. Moreover, any combination of the above-described elements in all possible variations thereof is encompassed by the disclosure unless otherwise indicated herein or otherwise clearly contradicted by context.

SYSTEM AND METHOD FOR MONITORING FATIGUE DURING OPERATION OF EQUIPMENT
A system and method for monitoring fatigue during operation of equipment is provided. A system includes a processor subsystem; and a memory comprising instructions, which when executed by the processor subsystem, cause the processor subsystem to: determine a number of fatigue events of that an operator experienced while operating equipment over a time period; determine a weighted alertness score based on the number of fatigue events, the weighted alertness score determined using a non-linear function; and initiate a remedial response based on the weighted alertness score.

DETAILED DESCRIPTION
In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of some example embodiments. It will be evident, however, to one skilled in the art that the present disclosure may be practiced without these specific details.

An operator is a human who operates equipment, machinery, vehicles, or other human-controllable devices. Examples of operators include, but are not limited to truck drivers, car drivers, crane operators, drone operators, forklift drivers, press machine operator, and the like. While operators may include people who work at desks, such as a telephone service or a computer technician, monitoring the fatigue of such operators may not be as critical as monitoring heavy machine operators. As such, although some example embodiments discussed herein refer to vehicle operators, and in particular to heavy equipment, such as mining vehicles, it is understood that the concepts and mechanisms may be applied to a wide variety of situations.

The systems and methods described herein monitor fatigue events. A fatigue event is an event where the operator exhibits fatigue, exhaustion, tiredness, distraction, or other aspects of non-alertness. Fatigue events may be defined by a system architect, system implementer, or other administrative user. Examples of fatigue events include, but are not limited to long eye blinks, extended staring (e.g., reduced eye motion activity), head nods, slouched posture, slower breathing rhythms, reduced heart rate, or the like. Additionally, fatigue events may be defined by contextual or environmental parameters, such as sudden or late braking, reduced steering wheel adjustments, inconsistent vehicle acceleration, or the like.
https://patents.justia.com/patent/20200070838
https://worldwide.espacenet.com/publicationDetails/biblio?II=12&ND=3&adjacent=true&locale=en_EP&FT=D&date=20200305&CC=US&NR=2020070838A1&KC=A1#
Drawings
http://www.freepatentsonline.com/20200070838.pdf

FYI .Driver in toxification detection based on driver speech characteristics ++++

MANAGEMENT OF AUTONOMOUS VEHICLE DRIVING FEATURES BASED ON DRIVER STATE
According to one aspect, driver management is provided. One or more speech segments of a driver of an autonomous vehicle may be recorded. A position, a destination, or a previous destination of a driver or an autonomous vehicle may be tracked. An estimated state of the driver may be determined based on one or more of the speech segments, the position, the destination, or the previous destination, and a calendar event associated with the driver or a passenger of the autonomous vehicle. Autonomous driving features may be automatically enabled, disabled, or operation of the autonomous vehicle may be enabled or disabled in different modes based on the estimated state of the driver. Additionally, notifications may be displayed, rideshare applications may be launched, or warnings may be sent based on the estimated state of the driver.
https://patents.justia.com/patent/20200075043

DRIVER STATE DETECTION BASED ON GLYCEMIC CONDITION
https://patents.justia.com/patent/20200070840
Embodiments of the disclosure relate to systems and methods for detecting driver state based on glucose levels of a driver of the vehicle. In one embodiment, a method of detecting driver state based on glucose levels of a driver of the vehicle includes receiving measurements of glucose levels of the driver from a continuous glucose monitoring (CGM) device and determining a glycemic state of the driver based on the measurements and a driving performance parameter based on the glycemic state. The method further includes determining a degree of impairment of the driver based on whether the driving performance parameter satisfies a predetermined threshold and determining a level of driver state severity based on driver performance characteristics, features of the driving environment, and the degree of impairment of the driver. A response is then activated based on the driver state severity.

The driver behavior detection module 302 receives images of the driver's face, upper body, and feet captured using the image sensor 190. The driver behavior detection module 302 then applies computer vision tools to extract facial and whole-body features indicating the driver behavior and posture at any given time. The driver behavior detection module 302 estimates and classifies the different patterns of driver behavior and posture using one or more machine learning algorithms. In some embodiments, the driver behavior detection module 302 uses a computer vision tool such as OpenPose to extract features of the face, upper body, legs and feet and estimate in real-time the driver behavior and posture. For example, the driver behavior detection module 302 may determine that the driver is drowsy, inattentive, restless or otherwise not in an appropriate driving posture, based on the features extracted by OpenPose. Non-limiting examples of detection of driver behavior and posture may be found in U.S. patent application Ser. No. 15/861,009 filed Jan. 3, 2018, assigned to Toyota Research Institute, Inc. and titled “Vehicle Systems and Methods for Detecting and Mitigating an Incapacitated Driver”, the entire contents of which are incorporated herein by reference. The driver behavior and posture thus estimated is then inputted to the driver responsiveness estimation module 310.

The insulin effect estimation module 304 receives measurements of glucose levels and physiological parameters of the driver from the receiver 142. The insulin effect estimation module 304 may then optionally evaluate the measurements against any dose of insulin pumped to the body of the driver by an insulin pump. Otherwise, the measurements of glucose levels and physiological parameters of the driver are inputted to the driver responsiveness estimation module 310. The measurements of physiological parameters may include body temperature, drowsiness, blood pressure, heart rate, respiratory rate, b

METHOD AND PRODUCT FOR HANDLING DRIVER INTERVENTION IN AN AUTONOMOUS STEERING SYSTEM
In a number of illustrative variations, an autonomous steering system may comprise a number of logic modules intended to autonomously address a number of areas of control within the realm of vehicle steering and travel including but not limited to vehicle acceleration, vehicle braking, and lateral control of the vehicle. The logic for the modules of the autonomous steering system may account for driver assistance or intervention.
https://patents.justia.com/patent/20200070888
Drawings
http://www.freepatentsonline.com/20200070888.pdf

SYSTEMS AND METHODS FOR SALIENCY-BASED SAMPLING LAYER FOR NEURAL NETWORKS
A method includes receiving, with a computing device, an image, identifying one or more salient features in the image, and generating a saliency map of the image including the one or more salient features. The method further includes sampling the image based on the saliency map such that the one or more salient features are sampled at a first density of sampling and at least one portion of the image other than the one or more salient features are sampled at a second density of sampling, where the first density of sampling is greater than the second density of sampling, and storing the sampled image in a non-transitory computer readable memory.

In general, the saliency-based distortion layer reconfigures an image or video feed such that important features (e.g., a driver's eyes or a road hazard) may be automatically recognized and preserved at a higher resolution than other less salient portions of the image (e.g., the background portion of a cabin of a vehicle). This may allow for the down-sampling of an image or video without losing the important and salient features otherwise lost when reducing an image size for implementation in a CNN. Implementation of the saliency-based distortion layer within a CNN may allow a system to automatically decide important features (e.g., through a trained saliency network) without specific knowledge and/or input from a user. The CNN implementing the saliency-based distortion layer can learn what is salient in order to pre-select what is important and subject to further processing. In some embodiments, the saliency-based distortion layer can also use time-based compression, such as finding a key portion of a video (e.g., a driver falling asleep) and preserve that portion of the video, while compressing the other portions of the video.
https://worldwide.espacenet.com/publicationDetails/biblio?II=1&ND=3&adjacent=true&locale=en_EP&FT=D&date=20200305&CC=US&NR=2020074589A1&KC=A1#
https://patents.justia.com/patent/20200074589
Drawings;
http://www.freepatentsonline.com/20200074589.pdf

Every little install helps;

Sadleirs
The second oldest family business in Australia, Providing logistics & packaging solutions Australia wide & Worldwide.

Seeing Machines is setting an industry standard when it comes to driver fatigue and distraction. Guardian provides immediate intervention to the driver when signs of fatigue or distraction occur. Audio alarms and seat vibration alerts are activated instantaneously to warn the driver to re-focus their attention to the road.
This technology was recently installed into a number of our Chevron prime movers. Safety is paramount at Sadleirs!

https://www.linkedin.com/company/sadleirs/

S2020/Lewbo18,
It was that rear view mirror thing that was interesting about the John Deere patent I posted yesterday.

[0029] Inside the driver's cab 22 is a camera 40 for optically detecting the eye position or head posture of a vehicle operator 42. The information obtained in this respect is fed to the control unit 18 to determine the instantaneous viewing direction of the vehicle operator 42 using image processing software. The camera 40 is integrated in a rear-view mirror 44 or a housing 46 covered by the rear-view mirror 44.
https://worldwide.espacenet.com/publicationDetails/description?CC=US&NR=2018236928A1&KC=A1&FT=D&ND=3&date=20180823&DB=&locale=en_EP#

HEAD-MOUNTED DISPLAY AND CONTROL APPARATUS AND METHOD
An apparatus and method for displaying an operational area, the apparatus comprising a headset (10) for placing over a user's eyes, the headset including a viewing device (12) configured to provide to said user, in use, a view of a real-world environment, a screen depicting an operational area, said operational area being defined within said real-world environment and comprising a plurality of functional regions each defining a different one or more selectable functions or operations that can be performed in respect of said operational area, the apparatus being configured to project image data displayed on said screen into said user's view of said real-world environment through said visor to generate an augmented reality environment, the apparatus further comprising a control module (26) including a control device (28) configured to be selectively communicably coupled to all of said functional regions to enable a user to selectively perform the respective one or more functions or operations associated therewith, wherein the control device (28) is communicably coupled to a selected functional region only in response to a respective actuation signal, and an eye tracker module (24) configured to monitor said user's gaze relative to said augmented reality environment, in use, and, when said user's gaze is directed at a selected functional region displayed therein, generate and transmit to said control module (26) a said actuation signal and when said user's gaze is no longer directed at said selected functional region, cause said control device (28) to be decoupled therefrom.
https://worldwide.espacenet.com/publicationDetails/description?CC=EP&NR=3454175A1&KC=A1&FT=D&ND=3&date=20190313&DB=&locale=en_EP#

ADAPTIVE LIGHTING SYSTEM OF AN OFF-ROAD UTILITY VEHICLE
An adaptive lighting system of an off-road utility vehicle includes an illumination device controllable with respect to its emission characteristic or light intensity, and a control unit in communication with the illumination device. The control unit is configured to adapt the emission characteristic or light intensity level by controlling the illumination device based upon a determined gaze direction of a vehicle operator, an identified extraneous light effects, a determined relative position of an external off-road utility vehicle, or cartographic location information.

[0013] Furthermore, the possibility exists that the control device can determine the view direction of the vehicle operator by optical sensing of the eye position or head posture. Such devices are known, for example, from the automotive sector in connection with systems for warning against falling asleep and comprise a camera which is integrated, for example, in a rear-view mirror or a housing enclosed by the rear-view mirror inside a passenger compartment in order to derive using associated image processing software, the degree of fatigue of the driver from characteristic indicators of this obtained from a recorded movement of the eyelids or pupils.

[0029] Inside the driver's cab 22 is a camera 40 for optically detecting the eye position or head posture of a vehicle operator 42. The information obtained in this respect is fed to the control unit 18 to determine the instantaneous viewing direction of the vehicle operator 42 using image processing software. The camera 40 is integrated in a rear-view mirror 44 or a housing 46 covered by the rear-view mirror 44.
https://worldwide.espacenet.com/publicationDetails/description?CC=US&NR=2018236928A1&KC=A1&FT=D&ND=3&date=20180823&DB=&locale=en_EP#

Matt Ligertwood
Business Development Manager at Haddtrack

Whilst speeding is top of the list for most cause of accidents, fatigue and distraction are right up there however there is a solution to assist and prevent breaches in your fleet.

100,000 NSW drivers were identified through the new mobile phone detection cameras over a 3mth period, which is very alarming stats.

This morning at Eastern Creek, Chief Inspector Phillip Brooks and Haddtrack discuss the Seeing Machines solution being deployed in many transport verticals in the country, and the finer details of how the real-time intervention works to alert operators of fatigue breaches within minutes. If an event is triggered and a mobile is seen in use within the recording, be sure this won’t go unnoticed.

Contact me today to organise a live demonstration and showcase the amount of actionable data pulled from the Seeing Machines platform to protect your fleet.

https://www.linkedin.com/posts/matt-ligertwood-48045814_driversafety-fatigue-roadsafety-activity-6639994768938098688-QZ5m

Semicast Research@semicast_res
Collaborative Driving and the Future of Trust in Mobility
https://iis50.ces.tech/Videos/2020/Collaborative-Driving-and-the-Future-of-Trust-in-M.aspx