Member Info for johnchucka

'Unprecedented Collaboration' or JConfirmation Bias ? you decide.

Toyota
SYSTEMS AND METHODS OF PROCESSING AN IMAGE 2019-11-21
https://worldwide.espacenet.com/publicationDetails/biblio?II=6&ND=3&adjacent=true&locale=en_EP&FT=D&date=20191121&CC=US&NR=2019354780A1&KC=A1#
Systems, methods, and other embodiments described herein relate to processing an image having a distortion and an object. The method according to some embodiments includes acquiring an image having a distortion and performing a first object detection analysis on a first portion of the image. Next, the method includes performing an undistortion process on the image, and after, performing a second object detection analysis on a second portion of the image. The second portion of the image is different than the first portion of the image. The method can then combine a first result of the first object detection analysis with a second result of the second object detection analysis.

GM 2019-11-28
CONTROL SYSTEMS, CONTROL PROCEDURES AND CONTROLS FOR AN AUTONOMOUS VEHICLE
[0035]
Cameras (or image sensors) may be arranged to provide a three hundred and sixty (360) degree image coverage of the surroundings of the vehicle 10. The cameras capture images (eg. B. frames) and give image data (z. A distorted image in YUV format), which can then be processed into rectified (or undistorted) camera images. An image pre-processing module can process the image data by equalizing / correcting the image data, pre-processing the corrected image data (e.g. B. image size change and average subtraction) and the corrected, preprocessed image data in corrected camera images (z. In a normal RGB format) that can classify a neural network of an image classification module. The image data may be corrected to correct for distortions in the image, which may cause straight lines (in reality) to appear curved, for example, when point clouds in 3D space were projected onto the uncorrected image data, they might actually become distorted due to the distortions be the wrong place in the picture.
By equalizing the image, the projections from the 3D space correspond to the correct parts of the image. The rectified camera images may then be sent to an image classification module along with other inputs, including three-dimensional locations of objects from an object tracking module, and processed to generate the image classification data that may be provided to an object classification module and used to generate object classification data that is then sent to an object tracking module which processes the objects, radar tracking information and object classification data to generate object tracking information.
https://worldwide.espacenet.com/publicationDetails/description?CC=DE&NR=102019111402A1&KC=A1&FT=D&ND=3&date=20191128&DB=&locale=en_EP#

Honda 2019-11-14
SYSTEM AND METHOD FOR LEARNING AND EXECUTING NATURALISTIC DRIVING BEHAVIOR
https://worldwide.espacenet.com/publicationDetails/biblio?CC=US&NR=2019346844A1&KC=A1&FT=D&ND=3&date=20191114&DB=&locale=en_EP#
A system and method for learning and executing naturalistic driving behavior that include classifying a driving maneuver as a goal-oriented action or a stimulus-driven action based on data associated with a trip of a vehicle. The system and method also include determining a cause associated with the driving maneuver classified as a stimulus-driven action and determining an attention capturing traffic related object associated with the driving maneuver. The system and method additionally include building a naturalistic driving behavior data set that includes at least one of: an annotation of the driving maneuver based on a classification of the driving maneuver, an annotation of the cause, and an annotation of the attention capturing traffic object. The system and method further include controlling the vehicle to be autonomously driven based on the naturalistic driving behavior data set.
[0042] Additionally, the one or more cameras 112 of the vehicle camera system 110 may be disposed at internal portions of the vehicle 102 including the vehicle dashboard (e.g., dash mounted camera), rear side of a vehicle rear view mirror, etc. The camera(s) 112 may also be positioned in a direction of the driver of the vehicle 102 to capture images of the driver as the vehicle 102 is being driven. The camera(s) may be calibrated to clearly capture specific body movements that may include head angle of the driver as the driver moves their head towards one or more traffic related objects while conducting the one or more driving maneuvers. Additionally, the camera(s) may be calibrated to clearly capture eye gaze movements as the driver moves their eyes to view one or more traffic related objects while conducting the one or more driving maneuvers.

[0043] The one or more cameras 112 may provide untrimmed images/video of the surrounding environment of the vehicle 102 and the internal portions of the vehicle 102. As discussed below, the naturalistic driving application 106 may execute image logic to determine the image data. The application 106 may additionally execute the image logic to determine one or more sets of image coordinates associated with one or more objects that may include, but may not be limited to, traffic participants (e.g., pedestrians, bikers, other vehicles), roadway attributes (e.g., lane markings, off-ramps, curbs), and road side objects (e.g., traffic light, stop sign). The naturalistic driving application 106 may additionally execute the image logic to determine driver eye gaze coordinates and driver head angle coordinates that may be utilized by the application 106 to determine the one or more attention capturing traffic related objects that may capture the driver's attention as they conduct the one or more driving maneu

TLS,
Lets hope I'm not just paddling in the deep end of conformational bias;)

Bosch.
METHOD AND DEVICE FOR ESTIMATING A DIRECTION OF A VIEW OF A VEHICLE OCCUPANT, METHOD AND DEVICE FOR DETERMINING A HEAD MOVEMENT GAIN PARAMETER SPECIFIC FOR A VEHICLE OCCUPANT, AND METHOD AND DEVICE FOR ESTIMATING THE DIRECTION OF VIEW OF A VEHICLE OCCUPANT 2019-10-17
A method for estimating a direction of view of an occupant of a vehicle. The method includes a step of ascertaining a head fixation position of the head of the vehicle occupant, terminating a current head movement of the head, using sensor data concerning an angular speed of the current head movement, and a step of combining the head fixation position and a head movement gain parameter in order to estimate the direction of view of the vehicle occupant, the head movement gain parameter being a function of a specific human head movement pattern during a visual fixing of an object.
https://worldwide.espacenet.com/publicationDetails/biblio?CC=US&NR=2019318183A1&KC=A1&FT=D&ND=3&date=20191017&DB=&locale=en_EP#

METHOD AND SYSTEM FOR DETERMINING AN OBJECT LOCATION BY USING MAP INFORMATION 2019-11-21
A system and method for determining an object location by using map information is disclosed. The method includes receiving, by a controller, an image data of a scene. An object is located within the scene. The method also includes determining, by the controller, a location of the object based on the image data of the scene. The method can also include receiving, by the controller, a map information. The map information includes at least one path information. The method also includes determining an association between the location of the object based on the image data and the map information of the scene. The method also includes determining a 3-dimensional location of the object based on the determined association.
https://worldwide.espacenet.com/publicationDetails/description?CC=US&NR=2019354781A1&KC=A1&FT=D&ND=3&date=20191121&DB=&locale=en_EP#

Patrickt,
Which DMS provider would YOU go with, the one that is 2nd/3rd best and only looks inward with a few valuable features or would you choose the most technically advanced model that allows for the most accurate fusion with your other outward facing sensors which also will gives you inward/outward driver/object data / Lidar sensor alignment to enable you keep up the machine learning capacity with your best in class peers at the AVCC where the next global Baseline safety architecture is being established?
Now ask your self which one would Toyota choose?

TLS,
By looking into the future thru Patents we can work out the dystopian desires we never knew we had well before they persuade us we need to buy them.

Lewbo,
Where I see what I consider a relevant (to SEE) numbered paragraph I include it in the posting . What I'm noting on this 2019-11-21 batch is I'm not finding as many direct references to gaze/Driver/inward facing / tracking camera references but to the sensor system as a whole. For example earlier patents noted eye tracking then micro sleeps, then many described Gaze tracking, then Gaze and Object , to inward and outward facing cameras and sensor fusion/alignment along with the rest of the detection components, Lidar etc . Now those elements look to be being described and included as 'sensors' from a brief scan of each patent thereby losing it's distinct identity within the overall sensor tool kit. My guess is that SEE tech has been described in detail historically and now has merged in Patent terms into the whole 'sensor' system.

As to your point about Toyota catching up. If they are behind it's hard for me as a non techy to determine but with the AVCC announcement speaks of 'unprecedented collaboration' my personal view (aka JC logic) and interpretation is that they are all closely aligned and many companies are collaborating on a baseline technology platform.

SYSTEMS AND METHODS FOR CONDITIONAL IMAGE TRANSLATION
System, methods, and other embodiments described herein relate to conditionally generating custom images by sampling latent space of a generator. In one embodiment, a method includes, in response to receiving a request to generate a custom image, generating a component instruction by translating a description about requested characteristics for the object instance into a vector that identifies a portion of a latent space within a respective generator. The method includes computing the object instance by controlling the respective one of the generators according to the component instruction to produce the object instance. The respective one of the generators being configured to generate objects within a semantic object class. The method includes generating the custom image from at least the object instance to produce the custom image from the description as a photorealistic image approximating a real image corresponding to the description.
https://worldwide.espacenet.com/publicationDetails/biblio?II=2&ND=3&adjacent=true&locale=en_EP&FT=D&date=20191121&CC=US&NR=2019354804A1&KC=A1#

MODELING GRAPH OF INTERACTIONS BETWEEN AGENTS
A method for configuring a behavior for a subject vehicle is presented. The method includes observing a first behavior of an agent during a first time period and estimating a first set of behavioral constraints followed by the agent based on the observed first behavior. The method also includes observing a second behavior of the agent during a second time period and determining whether the agent is behaving in accordance with the first set of behavioral constraints based on the second behavior. The method still further includes adjusting the first set of behavioral constraints when the agent is not operating in accordance with the estimated set of behavioral constraints. The method also includes adjusting the behavior of the subject vehicle based on the adjusted first set of estimated behavioral constraints.

[0037] Based on the observed behavior of the subject vehicle 200, the second agent vehicle 210 adjusts its set of behavioral constraints according to the estimated set of behavioral constraints of the subject vehicle 200. The estimated set of behavioral constraints may include the one-way hypothesized constraining relationship 226, the third one-way relationship 224, and the two-way constraining relationship 222. For example, a prediction system of the second agent vehicle 210 may predict that the subject vehicle 200 is merging into the gap between the first agent vehicle 204 and the second agent vehicle 210. Thus, the second agent vehicle 210 adjusts its set of behavioral constraints to consider a merging vehicle (e.g., subject vehicle 200) while also considering that the first agent vehicle 204 is in front of the second agent vehicle 210. For example, based on the updated set of behavioral constraints, the second agent vehicle 210 may maintain a distance from the first agent vehicle 204 while also providing space for the subject vehicle 200 to merge.
https://worldwide.espacenet.com/publicationDetails/biblio?II=3&ND=3&adjacent=true&locale=en_EP&FT=D&date=20191121&CC=US&NR=2019354105A1&KC=A1#

Toyota 2019-11-21
PROVIDING USER ASSISTANCE IN A VEHICLE BASED ON TRAFFIC BEHAVIOR MODELS
Providing user assistance in a vehicle includes evaluating information about the manual operation of the vehicle and information about an environment surrounding the vehicle, and identifying the driving behavior of the vehicle based on the evaluation of the information about the manual operation of the vehicle and the information about the environment surrounding the vehicle. The user assistance further includes receiving a traffic behavior model that describes a predominating driving behavior of a like population of reference vehicles, and issuing both prospective instructions and remedial instructions to a user on how to make the driving behavior of the vehicle match the predominating driving behavior of the like population of reference vehicles.
https://worldwide.espacenet.com/publicationDetails/biblio?II=0&ND=3&adjacent=true&locale=en_EP&FT=D&date=20191121&CC=US&NR=2019355249A1&KC=A1#

Toyota 2019-11-21
SYSTEMS AND METHODS OF DETERMINING STEREO DEPTH OF AN OBJECT USING OBJECT CLASS INFORMATION
Systems, methods, and other embodiments described herein relate to a method of determining stereo depth of an object. One method includes obtaining an image captured with a stereo camera arrangement. The stereo camera arrangement can be installed in one of a vehicle and a robotic apparatus, for example. The image captures a portion of the environment associated with the stereo camera arrangement. The method can further include identifying an object in the image, determining an object class for the object, determining a size parameter of the object, determining a size parameter of the object class, determining a maximum disparity for the object with the size parameter of the object and the size parameter of the object class, and determining a stereo depth of the object based on the maximum disparity.
https://worldwide.espacenet.com/publicationDetails/biblio?CC=US&NR=2019355140A1&KC=A1&FT=D&ND=3&date=20191121&DB=&locale=en_EP#

[0041] The vehicle 100 can include an input system 220. An input system includes any device, component, system, element or arrangement or groups thereof that enable information/data to be entered into a machine. The input system 220 can receive an input from a vehicle passenger (e.g. a driver or a passenger). The vehicle 100 can include an output system 225. An output system includes any device, component, or arrangement or groups thereof that enable information/data to be presented to a vehicle passenger (e.g. a person, a vehicle passenger, etc.).

4.45pm – 5.00pm, Augmenting Learning with Eye-tracking: An Evaluation in an Operational Fighter Pilot Training Setting, Kyle Wilson phd
https://www.hfesa2019.org/uploads/1/0/2/1/102108368/draft_program5.pdf

Gemma Read@gemma_read
@KyleWilsonPhD of @seeingmachines
giving us an overview of research to test if visualisation of student pilot’s eye movements can be useful in training #hfesa2019

SYSTEMS AND METHODS OF PROCESSING AN IMAGE
Systems, methods, and other embodiments described herein relate to processing an image having a distortion and an object. The method according to some embodiments includes acquiring an image having a distortion and performing a first object detection analysis on a first portion of the image. Next, the method includes performing an undistortion process on the image, and after, performing a second object detection analysis on a second portion of the image. The second portion of the image is different than the first portion of the image. The method can then combine a first result of the first object detection analysis with a second result of the second object detection analysis.
https://worldwide.espacenet.com/publicationDetails/originalDocument?CC=US&NR=2019354780A1&KC=A1&FT=D&ND=3&date=20191121&DB=&locale=en_EP#

Page 10/15
022
'The vehicle 100 discussed herein has a single camera. However it is envisioned that the vehicle 100 can include many cameras operatively connected to a surface including inwardly and/or outwardly facing surfaces'

S2020,
I'm looking forward to when Cenkos/SEE stop being Conservative and give us decent figures and projections and when they tell us about Guardian3/4 and it's Patented unique multi camera Lidar/ ADAS aligned calibration capacity to enhance our own Data haul with it's commercial value to Big Data/ AVCC partners/ telematics companies etc along with it's deep learning Algorithm characteristics which makes our DMS data Gold++ .

Vehicle telematics data could unlock $1.5 trillion in future revenue for automakers

'The data that connected cars and autonomous vehicles produce open up entirely new revenue streams that the automaker can control (and share with partners in other sectors). According to McKinsey, monetizing onboard services could create USD $1.5 trillion – or 30 percent more – in additional revenue potential by 2030, which will more than offset any decline in car sales.
And this data on how a driver and vehicle interact can also give automotive manufacturers, logistics companies, fleet managers, and insurance companies valuable information on how to make transportation safer, more efficient, and more enjoyable — but they must be able to handle the new huge streams of data and analyze those to extract insights.'
https://venturebeat.com/2018/12/04/vehicle-telematics-data-could-unlock-1-5-trillion-in-future-revenue-for-automakers/

TLS/Sch,
I also looks to me (with my very limited understanding if this tech) that SEE is creating the IP (as they dial down on the R&D ;) that moves beyond Driver gaze tracking. It is designing a system where multiple camera's can be used to identify and track objects within the vehicle and outside scene across different cameras and 'able to track and map the positions of objects from the field of view of one camera to another' (004) (089) and the data can compliment other perception systems (013/ 065). Their IP describes a new calibration solution which has previously required 'significant time and effort and the accuracy of the positions/orientations are limited by the technician's competence' and ' each time a camera is to be repositioned or a new camera added to the monitoring system, the calibration procedure must be performed again at the cost of system downtime and technician's charges(005), so making it cheaper and more accurate
https://worldwide.espacenet.com/publicationDetails/description?

Tell me I'm wrong but I think SEE are designing a whole of vehicle camera system that compliments Lidar type systems and can be used on multiple vehicle types without the associated technical issues and costs and as such could be fitted by less qualified engineers which will allow for the more rapid role out of the technology amongst fleets and offer more accurate/reliable Data .
https://worldwide.espacenet.com/publicationDetails/description?CC=US&NR=2019164310A1&KC=A1&FT=D&ND=3&date=20190530&DB=&locale=en_EP#

TLS,
Yes a big jump which looks to me to be coordinated imo with SEE/ Toyota/Hyundai/GM/Continental /Bosch all issuing patents which appear to have common properties of describing a relationship between driver facing and outward facing sensors and all with relatively close dates.

For reference and comparison.
GM GLOBAL TECH OPERATIONS LLC 2019-10-17
METHODS AND SYSTEMS FOR PROCESSING DRIVER ATTENTION DATA
https://worldwide.espacenet.com/publicationDetails/biblio?II=0&ND=3&adjacent=true&locale=en_EP&FT=D&date=20191017&CC=US&NR=2019318180A1&KC=A1#
Methods and systems are provided for processing attention data. In one embodiment, a method includes: receiving, by a processor, object data associated with at least one object of an exterior environment of the vehicle; receiving upcoming behavior data determined from a planned route of the vehicle; receiving gaze data sensed from an occupant of the vehicle; processing, by the processor, the object data, the upcoming behavior data, and the gaze data to determine an attention score associated with an attention of the occupant of the vehicle; and selectively generating, by the processor, signals to at least one of notify the occupant and control the vehicle based on the attention score.

[0029] In various embodiments, the attention determination system 22 further receives data from driver monitoring systems 14 of other vehicles (not shown) or a central processing system (not shown), for example, via a communication system. In such embodiments, the data from the vehicles is generally indicates the gaze of other drivers when at the same location as the vehicle. In such embodiments, the attention determination system 22 further determines the attention of the driver based on the general gaze direction of other drivers determined from the received data.

Toyota / Honda for comparison .

TOYOTA RES INSTITUTE INC 2019-07-25
VEHICLE SYSTEMS AND METHODS FOR DETERMINING A TARGET BASED ON A VIRTUAL EYE POSITION
https://worldwide.espacenet.com/publicationDetails/biblio?CC=US&NR=2019228242A1&KC=A1&FT=D&ND=3&date=20190725&DB=&locale=en_EP#

[0028] The user detection system 130 is communicatively coupled to the electronic control unit 102 over the communication path 104. The user detection system 130 may include any device configured to detect the presence of an object within the surrounding environment of the vehicle 100. More specifically, the user detection system 130 is configured to detect the presence of an object within the vicinity of the vehicle 100. The user detection system 130 may include a user detection sensor 132 configured to output an object signal indicative of the presence of one or more objects within the vicinity of the vehicle 100. Based on the object signal of the user detection sensor 132, the electronic control unit 102 may execute object recognition logic to detect an object and classify the detected object into a classification. The user detection sensor 132 may include, but is not limited to, a camera, a LiDAR sensor, a RADAR sensor, a sonar sensor, a proximity sensor, and the like. In some embodiments, the user detection system 130 includes more than one user detection sensor 132.

Honda 2019-07-04
VEHICLE CONTROL DEVICE, VEHICLE CONTROL METHOD, AND PROGRAM
https://worldwide.espacenet.com/publicationDetails/biblio?CC=WO&NR=2019130483A1&KC=A1&FT=D&ND=3&date=20190704&DB=&locale=en_EP#

[0037]
In addition, the other vehicle occupant gaze direction recognition unit 132 of the recognition unit 130 identifies the other vehicle m from the recognized objects, and recognizes an occupant riding on the other vehicle m. Then, the other vehicle occupant gaze direction recognition unit 132 recognizes the recognized direction of the occupant's face or the direction of the line of sight of the other vehicle m. For example, the other vehicle occupant gaze direction recognizing unit 132 recognizes the direction of the occupant's face or the direction of the line of sight based on the position of the occupant's pupils, irises, eyes, and the like, as well as the skin and hair regions.

[0073]

Further, in the first embodiment described above, the face of the driver of the other vehicle m or the direction D of the line of sight is described as being recognized on the own vehicle M side, but the present invention is not limited to this, and the face of the driver of the other vehicle m Alternatively, the direction D of the line of sight may be recognized on the other vehicle m side. For example, an occupant monitoring camera may be installed in the cabin of another vehicle m. The occupant monitoring camera is installed so that the driver of the other vehicle m is included in the angle of view. In such a case, the device control unit 170 controls the communication device 20 to receive an image captured by the occupant monitoring camera from the other vehicle m. And the recognition part 130 performs image processing with respect to the image received from the other vehicle m, and recognizes the direction D of the driver | operator's face or eyes | visual_axis of the other vehicle m. On the other vehicle m side, the driver's face or line-of-sight direction D may be recognized from an image captured by the occupant monitoring camera. In this case, gaze direction information is transmitted from the other vehicle m to the host vehicle M.