We would love to hear your thoughts about our site and services, please take our survey here.
Good find JC.
We know that Conti have "shy displays" that can be hidden behind trim for example or this could relate to ordinary screens.
Essentially if you aren't looking then it is not illuminated, or it is done a a lower resolution and/or brightness/refresh rate.
If urgent traffic or vehicle information needs to be displayed then it can quickly brighten to catch the driver's attention, but normally it will gently increase so as not to startle or distract.
The other reason for the patent is that low power or sleep mode reduces the power consumption and reduces the heat build up, which reduced cooling requirements and will provide a longer life for the screen.
‘In your car, you only can see and hear and activities, and you’re limited in the things that you can do,’ says Durach. ‘That’s the big differentiator if you think about what your smartphone can do.’
Smartphones get more of our attention than touchscreens, but they’re far less aware of what we’re doing – and that’s totally different to when we’re driving. OEMs are able to pick up more data about our current activities than Apple or Google could ever dream of – and that’s actually a good thing for making a more intelligent UX.
‘[In a car] we know the speed you’re driving, your navigation address, the status of the car, the state of the battery or how much petrol you have, your navigation when getting to a certain point where we can find your parking spot like this, we always think about the context,’ explains Durach.
‘It’s really important for us to give you a better user experience, because we know so much what you’re doing. We know even where you’re looking at the interior cameras.’
https://www.carmagazine.co.uk/car-news/motor-shows-events/consumer-electronics-show-ces/2023/
Well done JC, I haven't read it yet, but when I saw heart, the word that jumped into my head was "Photoplethysmography" (OK maybe not the word, but rPPG, or something like that) . I did some research into Harman's DMS earlier in the week that did heart rate using that thing that can't be spelt.
It is about measuring the change in reflected light levels caused by your pulse moving your skin, so that different areas scatter the light source. Not visible to humans but the subtle diffence can be measured as long as environmental light is filtered out.
No doubt this is one of the features that at least two OEMs put into their shopping list so they can be seen to add value.
When JC first posted about
SYSTEM AND METHOD FOR IDENTIFYING A CAMERA POSE OF A FORWARD FACING CAMERA IN A VEHICLE
And CAMERA REGISTRATION IN A MULTI-CAMERA SYSTEM
Both Seeing Machines patent applications
about SEE linking the gaze with the outside world, my first thoughts were they don't have the resources to identify and track the outside world. That was a problem that many other companies were burning cash on. But Tier 1s didn't need SEE for the whole solution they just needed a 3d gaze vector and a known position in the car. We then saw examples of Mercedes asking about the surroundings (via Cerence) and more practically Veoneer warning about a cyclist about to cross the path, that the driver HADN'T SEEN. Now we see the democratisation of the technology to a single system on chip that can we widely used, initially for Conti, but later who knows.
dr777, don't lose any sleep over these enthusiastic amateurs when it comes to DMS.
Looks like it is an single person project using free face mapping software from Google. This only works with videos of the face, no eye tracking, purely working on PERCLOS (with their "custom" algo) don't expect it to be deployed in Automotive. They may have an NVidia logo, and have qualifications in SPICE and ISO 26262, but they don't have any actual DMS or optical path experience, never mind Human factors.
Reading between the lines, Valeo got a contract with BMW, (Valeo released their news in June https://www.valeo.com/en/valeo-signs-major-contract-with-the-bmw-group-in-advanced-driving-assistance-systems/)
However, for the life of them, they can't remember which Tier 2 they were working with, but they know that whoever it was is a friend of Qualcomm. Hey, Qualcomm are busy people, but I know that they are at CES 2023, so we will ask them which Tier 2 we were really impressed with when we SEE them. I think that they will be at the press days, so if I get a chance to see Christiano or Nakul I will get the Tier 2 phone number and let them know then. If not Nakul is around on the 5th
https://www.qualcomm.com/company/events/ces
Another find by JC that shows some of the more unique edge cases of DMS.
This is focussed on Land Rover, and in particular their off-road capabilities. Now anyone who has driven properly off road, knows that driver distraction and attention are different when there is no smooth straight road to follow. Hands constantly at 10 to 2, quarter to nine or dread the thought twenty to eight, just don't happen so capacitive touch sensors will give irregular readings. Thumbs of course are outside of the wheel, and with the rough road, the torque sensors may not get an even signal to determine if there is intelligent input, or a slumped driver. There is no easily identifiable lanes to detect if you are following the road, or a GPS map route to follow, so the system can't detect if you leave the lane.
Then add in some high end crawler features and the car may be driving itself. Add in the extreme steering wheel movements and the cameras may be obscured by the wheels or the driver's arms.
Is the driver looking in the correct place? Well they may be hanging out of the window, or getting all their "vision" from someone who is talking them through a tricky situation from outside of the vehicle as they can "see" or they could be staring intently at a screen inside the car with a "see through" functionality, so they can see a virtual view of the road ahead through the bonnet so they can steer around the obstructions or holes. So DMS is not a one size fits all!
OK, key thing to remember here is that Veoneer is primarily an airbag and seat belt company. So this is from the that side of the company, and imagines a world where DMS doesn't exist. So they have Near Infra red cameras for seat belt monitoring functionality. They even seem to embed a pattern that is visible in NIR to make detecting the seat belt easier and show the image pre-processing to make this detection easier. But no matter how cheap they could make the processing, a stand alone seat belt monitoring system will not fly. In reality, it will be done by the same system that provides the DMS and OMS
Red letter day - CFP is correct, chutzpahtaker is wrong!
It is not "he gets paid less shares" but "he gets paid FEWER shares"
Now let's move on
Transportation
BMW taps Valeo for driving assistance systems in its next-gen EVs
Jaclyn Trop
@jaclyntrop / 5:23 PM GMT+1•June 28, 2022
Neue Klasse
Image Credits: BMW Group
BMW Group has tapped Valeo to provide the advanced driver assistance system for the automaker’s new electric vehicle platform due to launch in 2025.
The deal, a significant win for Valeo, also holds importance for BMW. The automaker is aiming to capture a 50% market share of global EV sales by the end of the decade and sees its new EV platform, called Neue Klasse, as the key to reaching that target. The partnership also gives a bit more insight into BMW’s changing strategy around EVs.
Valeo will provide the sensors and software for parking and maneuvering BMW’s next generation of EVs, as well as the advanced driver assistance system domain controller that manages the flow of data between the sensors and the vehicle.
Developing a dedicated EV platform is an about-face for the German automaker. BMW tested the concept with its first EV, the i3, before hedging its bets with a hybrid approach to building gas-powered, hybrid and battery-electric vehicles on the same line. However, this delayed the development and launch of new BMW models.
Configurable platforms that can underpin a range of EVs, from subcompact cars to vans, will be crucial to automakers’ sustained growth, according to the 2022 AlixPartners Global Automotive Outlook released June 22.
GM’s EVs are based on the automaker’s new Ultium battery platform, whose modular architecture can produce 19 different battery and drive configurations, help GM scale its battery-electric operations and reduce costs.
About 30% of Volkswagen Group’s EVs are based on its MEB (Modular Electric Drive Matrix) architecture, including the ID.AERO sedan the automaker debuted yesterday for the Chinese market. The automaker expects that share to surpass 80% by 2025.
BMW “Neue Klasse” will also use Valeo’s next-generation ultrasonic sensors, surround-view cameras and **new multifunctional interior camera that can detect signs of drowsiness, fatigue or driver distraction**.
Powered by Qualcomm Snapdragon SoCs, the ADAS domain controller will host Valeo’s software platform for low-speed driving, as well as machine learning software from BMW and Qualcomm. The system will add new features and upgrades during the car’s lifecycle, as well as collect real-time mapping and location data.
“Beyond the opportunities resulting from our integrated software stack, the domain controller itself is an important cornerstone in our strategy for safer and smarter mobility,” Marc Vrecko, Valeo’s president of comfort and driving assistance systems, said in a statement.
BMW did not immediately respond to a request for comment Tuesday.
https://techcrunch.com/2022/06/28/bmw-taps-valeo-for-driving-assistance-systems-in-its-next-gen-evs/?guccounter=1&guce_rePowered
Both are using artistic license - so we can see "how robots see and think" just like when watching Terminator - I don't think robots need to read their source code via their eyes!
WhatIf, no patent for IR using light sources in DMS, but SEE have patents for how they are used - varying intensity, and whether one or more are illuminated used each image and selecting the best image
Seize, this may be a light bulb moment but Osram make light sources.
Light (NIR) is essential for DMS, OMS, but there doesn't need to be exclusivity in who they sell to.
They need to sell themselves, so they are attaching their name to the systems where they are essential, even of they are not unique suppliers
Lewbo, I think that was more of a recruitment / STEM session, so they were showing the kind of things you would be involved in rather than actively promoting SEE, that was just an ideal and portable demo
Certainly hyms,
Adding a feature to estimate size, weight, presence and position of a passenger allows far better air bag deployment.
Takata went bust because of faulty airbags. They are a safety feature, but they can harm you if they fire at the wrong time or if you are too close.
If you think of an airbag, it is a shaped balloon that is inflated rapidly by explosives to just in front of you and then it does a controlled deflation as you hit it so it becomes a soft cushion slowing your forward progress so you don't hit anything and so slow your deceleration even though the vehicle may have come to an abrupt halt.
Now if you are too close, you will hit a solid bag as it is still inflating
If your limbs are in front of your body, they will be pushed back and hit your face/body.
If you are tall and heavy, you need more air in the bag before inflation starts
If you are small and light, you need a softer "pillow"
baby seat in the front passenger seat - air bag should be disabled
So now airbags can have more than one explosive "setting" so that the bag has a variety of inflation levels and with an OMS, it can detect if you are too close and judge how much power is required to safely protect you, or even disable it entirely if there is no passenger in the seat - reducing ear damage and also the cost of repair
Phil, as you say, it doesn't matter.
Qualcomm bought the driving and "policy" stack - "how to drive" and "how" to drive (what are the rules, what style of driving - aggressive, or smooth) and also supporting the driver with overrides.
Veoneer keep active safety - see this for details of what that means (Note I don't think DMS is part of this particular example)
https://www.veoneer.com/en/press-releases?page=/press/perma/2068791
Veoneer Awarded Active Safety System by Major Asian OEM
November 15, 2022 08:30
Stockholm, Sweden, November 15, 2022: The automotive technology company Veoneer has signed an agreement with a major Asian OEM to equip an electric vehicle joint-venture in China with an active safety system comprising of a monovision camera, a package of forward-looking and corner radars, and an ADAS ECU.
In 2024, Veoneer will begin delivering and integrating the Active Safety system, designed for collaborative driving on level 2+. The system's high level of safety functionality is maintained in bad weather and darkness, and it includes features that can trigger alarms, take control of the vehicle to avoid a crash or lessen an accident's severity as well as automatically brake the car.
The award includes Veoneer's 4th generation monovision system. The hardware is integrated with software which is using deep learning technologies that identify objects such as cars, road markings and road signs.
Veoneer's corner and forward-looking radars provide reliable and accurate situational awareness of objects surrounding the vehicle and are critical for autonomous driving systems. Radar systems warn for potential collisions, are used for avoiding blind spot crashes, and can control stop-and-go functions in queues, among other functionalities.
The award also includes Veoneer's ADAS ECU, the multi-lingual brain that monitors the surroundings and fuses data from cameras, radars and sensors, interprets the situation and makes informed decisions to support the driver and triggers features such as emergency braking or automatic lane changes.
That roadmap may have spelt out Veoneer's plans to use "SoC Camera" or "Smart Camera" but I think we all know it better as "Magna Mirror". Back in 2021 they put it on their road map and the next step was DMS controlling RCS where they make much of their money.
When Paul said the future Tier 1 winners are not what we expect - he was telling us Veoneer won't win DMS because it will be a Magna win even if the purchase fell through, it was on the roadmap published in mid 2021
This link may be more insightful https://www.linkedin.com/posts/veoneer_press-releases-activity-7009440592954830848-K_vY?utm_source=share&utm_medium=member_ios given several SEE names have liked it.
I thought at the time it could be related to OMS providing the positions and estimated weight of the occupants. So I have done some digging.
https://showroom.veoneer.com/#/show/productshowcase/product_02/description
This is from mid 2021. You will see in the text
OEM1 = Ford
OEM2 possibly Fisker?
2023 Ford using "DMS3" - so that is a new vehicle or refresh we can expect to see soon .
The code names on the parts CX727 and P702 are Ford's internal project codes for Mach-E and F-150
The picture is clearly a Mach-E steering wheel.
The video is a pre-production version of a Lincoln (posh Ford brand) - the Mach-E sensor is the wrong shade of plastic
Next slide is the product history for DMS:
Gen 2 SOP is 2020 (Mach-E and F-150)
Gen 3 SOP is mid 2023 (we know this is OEM1 = Ford)
2025 has two entries
- Smart Camera (no, not Smart Eye, don't worry, have patience)
- Body Tracking Pre-Airbag deployment features)
Final slide - DMS Roadmap
- DMS 2 2020
- DMS 3 2023
- SOC Camera 2024 (This is "smart camera", System On Chip. I think this is possibly OmniVision's OAX4600 chip in the same package as the Camera?)
- Smart RCS - so this is the DMS/OMS tracking the bodies to control how the Airbags are deployed
Now the Veoneer win is for 2026, so that fits with the timeline (as it was in Mid 2021)
https://www.veoneer.com/en/press-releases?page=/press/perma/2079465
Ignore the heading. STD got shot down for posting this which is for a Reaction Control System (RCS) win by Veoneer.
See my next post for reasons (and a fresh Subject)
Paul told us at the town hall that certain companies attitude to pricing was to demand more and more discount:
1 box of Pencils for $x dollars, and if they ordered 2 boxes they wanted them half price.
Seeing Machines are working with fewer Tier1s and they are working far more deeply with those companies to build a compelling business proposition.
Now, if we have walked away from a Tier 1 that doesn't mean they can't win. DMS is not the only thing that they are selling, normally the OEM has specified a stack of requirements including ADAS and a total budget for the features. So it is possible that a cheaper Bosch of Pencils has been specified for the T1 to get the business.
"Up to 53 models" strange wording. Either the OEM isn't sure that 53 models will make it to the market, or they have sold a similar deal to another T1 and they will both start with some confirmed models and play it out to see who gets the lions hare based on how the whole packages actually work and how they are received by the market.
Let's see how this goes...