Member Info for TheLongestShot

New concerns have forced Omnivision to announce that they are not a new variant, although tests have shown a marked increase in road safety spreading to other chip vendors infected with Fovio

They already walk among us

That is really positive news for Seeing Machines. I hope no-one has been caught in the crossfire though.

The Intel patent application is about scoring advertising signs or objects of interest viewed by passenger. It even discounts glances in the direction of signs where the passenger may have been looking at an approaching car. This of course means that they can charge advertisers for knowing how effective their signs are, or identify where perhaps there is little competition for eyeballs, so they can blight it.

Ha, no rest for me when JC is on a roll.

Mike, I rembember dissecting this one (or it's predecessor) a while back. Essentially it is limited to detecting distraction or lack of attention by monitoring the drivers gaze and checking that the gaze areas follow the key parts of the road, adjusting for curves and hills. I guess that can coexist with SEE's DSM patent, but I am no lawyer.

Note, this is the publication of the application. But the original Australian application was 2018.while there is still scope for a challenge, that would have been made by now if they had 'prior art'. So they can still challenge in the next 18 months? (I am no patent expert) But no one has and none of our competitors look to be using anything other than ML for driver state just like they do for happy or sad drivers.

This is they key point. Jumping from thousands of photos of distracted drivers to ML that gives confidence levels for various mental states is fragile and difficult to verify when it fails, the only solution is to throw more training data at it.

Affectiva are "cheating" with synthetic data to get better gender and ethnicity coverage, but in many ways this isn't fixing the problem, just moving it to how well people match your synths!

But why is this patent important - trustable, verifiable science.

Rather than go from pictures of a face to "magically" train a Machine Learning model to identify driver state, Seeing Machines are using eye gaze data - already tested and easily verified. Then it is mapping the eye gaze data onto a 2d representation of the world. You can verify the attention patterns on the instruments, mirrors, central console the road outside, street signs etc.
Next step, you can observe the scan patterns of an alert driver, a sleepy driver, someone watching a movie on their phone resting in front of the instruments.

By having separate verifyable steps you can have confidence. Contrast this with Tesla, who made great strides and hoped to crack self driving with more data, but the more they add, the more fragile their systems have become.

Thanks JC for keeping an eye on these. It is another sigh of relief as these finally convert into full patents!

If you wondered by Smart Eye are so focused on buying emotional companies? Well it is because this patent shuts them out of using eye gaze mapping on a 2d representation of the world and the resulting hot spots of attention and using the flicking between zones to identify an alert driver as opposed to being fixared on just one area as being zoned out (or watching a screen)

NOTICE OF PUBLICATION, music to our ears. Another patent safely in the bag. Seems like yesterday but also a lifetime ago when we first discussed this patent that explained why SEE IR LEDs flash alternately. We are on the home straight now.

Back then cameras in mirrors, A-Pillars or central console seemed so exotic. But with Magna (VW?) , Ford F-150 etc they will soon be commonplace.

Now we also know that eye tracking in Simulators is definitely a thing, even if the imminent news still isn't official

Good article Schlem,
The other part that worried me is the implication that pedestrians and cyclists would need to have connected devices and have alerts sent to them, or to be tracked by them? Well my wife's phone is safely encased in her bag so she doesn't hear phone calls. My phone seems a little sluggish when it wakes up. Cyclists don't use phones while cycling and as for young children who aren't being monitored by a flag waving adult?

Nice Volvo patent JC. This must be one of Trent Victor's last before he moved to Waymo. He is also a former Seeing Machine's director.

Schlem, Kelly Funkhauser's Tweet was the start of a good thread https://twitter.com/CRcarsKFunk/status/1456219111030247428?t=kdhw88J6CoBzvqArx27LGQ&s=19

I also like the fact that Magna have not patented "the" mirror, but a range of mirrors to cover various price points with optional extras such as OMS, built in displays, visible or hidden DMS camera and LEDs etc. They really have this all sewn up!

What can I say JC? This patent spells it out pretty clearly!
There are some clues about implementation "first processor" and "second processor" looks like there is a core using Occula to do the DMS and the second core that is handling the front facing camera for object detection. We also see two versions of the rear view camera, one where the lens and LEDs are visible below the mirror, and another where they are embedded behind the mirror.

The classifications are getting more interesting too:
(i) driver (or occupant) head found, (ii) abnormal head posture, (iii) false head detection in the driver's seat / area or in the passenger seat / area, (iv) eyes of the driver (or occupant) closed, (v) eyes of the driver not facing the road in front of the vehicle, (vi) position of the driver in the driver's seat (and / or position of the occupant in the Passenger seat), (vii) driver looking at a target, (viii) microsleep, (ix) driver (or occupant) asleep, (x) level of drowsiness of the driver (or occupant), (xi) category of the driver's hand position (such as using aspects of the systems described in U.S. Publication No. US-2018-0231976 (which is hereby incorporated by reference in its entirety), (xii) classification of a driver-held object, (xiii) driver (or occupant) nods with the head, (xiv) driver (o the occupant) speaks, (xv) driver (or occupant) laughs, (xvi) driver (or occupant) coughs, (xvii) driver (or occupant) sneezes, (xviii) driver or occupant) yawns, (xix) driver (or Occupant) turns the body, (xx) driver (or occupant) smokes, (xxi) driver (or occupant) holds a cell phone to his head, (xxii) driver (or occupant) changes, (xxiii) face identification, (xxiv) presence of an occupant, (xxv) presence of a child, (xxvi) presence of a child seat, (xxvii) video conference of the driver (or occupant) and / or (xxviii) the driver (or occupant) takes a selfie.

DMS eye gaze linked to identified objects in front of the car:
For example, the processor can determine the driver's attention and / or the driver's gaze direction (by processing image data captured by the driver monitoring camera) and by determining a hazard in front of the vehicle (by processing image data captured by the forward-looking camera) and in an area that is from the driver is not seen at the time, the system can generate a warning to the driver to inform the driver of the potential hazard that requires their attention.

When the mirror is moved, DMS has to re-orient itself by mapping pre-determined features in the cabin so that it can map the drivers head position and eye gaze relative to the car (not the moveable mirror) this has been mentioned before in SEE patents

I do like the idea of the mirror setting itself up for the driver automatically by working with the DMS camera!

I presume this is more about patent protection rather than allowing for our Swedish friends "The system can use any suitable face tracking software or algorithm."

Apologies, I am way behind on my reading. Good, no, bloody amazing find JC. Thanks also to Jazz for standing up to patent duties. If you are in any doubt, look at the pictures in the pdf, this is not pie in the sky, this is a mirror unit not far from production and if you know your US car logos it won't take long to ID the vehicle.

We have separate cameras for DMS and OMS hidden in the rear view mirror. Plus temperature readings of the passengers too (to spot anyone left behind - or for climate control) our friend Colin will like the heat pipes to dump the excess heat from all of the cameras and processing.
So hit this link, and enjoy the pictures. One day I may catch up and read the words too! https://www.freepatentsonline.com/20210291739.pdf

JC, remember that film Wall-E where the passengers reclined in mobile seats that moved automatically, until they all became obese and lacking in enough muscles to allow them to stand. Well this patent is one of the first steps.

For the record, I have never met Terry, we must work in different offices

Terry, have you been making these news stories yourself? The link points to your PC

HI JC,

Not much point having a DMS camera and a separate one for the HUD, and I don't think this will be up to much so it isn't going to steal our business

Schlem, Google has that story as being from Jan 2021

Hi JC,
This one shows how convoluted patents can get in order to solve one problem. TL;DR; It won't work!

CONTENT ADJUSTMENT BASED ON VEHICLE MOTION AND EYE GAZE.
Passengers get travel sick when reading on their phones. With AVs drivers will get this opportunity too! Honda's solution is to replicate what you were reading on your car's display but to stabilise the image so that as the car hits a bump on the road, the image will be constant relative to the surroundings as perceived from the position of the driver's eyes, so motion sickness is reduced. This will also require you to navigate on the mobile and eye tracking keeps everything together.

Of course this is not going to work and if it makes the driver feel better, the moving text will appear worse and more distracting for the passengers who have a different motions based on their size, weight and position in the car.