Member Info for johnchucka

This SEE Patent refers to driver gaze and road scene object / driver facing and forward facing camera calibration (See Description points 70/71/72/73/74/75 for detail) and imo look to share common themes of technical progress of the possible parallel developments that Toyota and GM patents have recently described and perhaps are related to the Data Acquisition Specialist role advertised yesterday.
SYSTEM AND METHOD FOR IDENTIFYING A CAMERA POSE OF A FORWARD FACING CAMERA IN A VEHICLE
https://worldwide.espacenet.com/publicationDetails/description?CC=US&NR=2019266751A1&KC=A1&FT=D&ND=3&date=20190829&DB=&locale=en_EP#

TLS,
This sentence I believe supports my view that it's external objects they will be 'labelling' .
'a current driver's license and confidence to drive on public roads in order to assist in data collection activities'.

Seeingtom, I get the impression that the job is a little more involved than you suggest;)

About the opportunity
The Seeing Machines Data Acquisition team works alongside key business units, marking up video datasets and preparing the data for machine learning algorithm validation and training.
As a Data Acquisition Specialist, you will:
gather data collection requirements through consultation with algorithm scientists and algorithm performance engineers
develop data collection protocols by soliciting and sharing technical feedback from all relevant stakeholders
prepare, execute and supervise data collection, data annotation and dataset curation activities in accordance with the approved data collection protocols and relevant guidelines
monitor progress against milestones and administer review processes to ensure quality benchmarks are met
support the Data Acquisition Supervisor by providing performance feedback on data acquisition assistants, and promote a continuous learning culture within the team
About you
As a Data Acquisition Specialist, you will provide critical support to the R&D team - as they apply machine learning technology to continued development of our class leading algorithms and technology.
We are seeking people with the following attributes:
strong organisational and management skills, with an ability to operate in a rapidly changing environment whilst maintaining high work quality and attention to detail
good communication and interpersonal skills, and the ability to hold effective technical discussions with engineers
ability to work with incomplete technical information and possess good problem-solving, analytical and critical thinking skills
experience in writing technical documents and reports
strong computer literacy, and experience with Microsoft Office
experience in developing scripts (e.g. in Python) to process data
experience in a technical supervisory role, especially in video image data collecting, processing and annotation (labeling) is highly desirable.
a current driver's license and confidence to drive on public roads in order to assist in data collection activities.

Adrein Gaiden explains what 'labelling' means to Toyota here @5.50m
Interview with Adrien Gaidon, Machine Learning Engineer, Toyota Research
https://www.youtube.com/watch?v=jBIYRoAQJuo

TLC,
Could be but interior objects are not going to take that long I would imagine, it's pretty limited, not exactly a career move, where as Objects in the wild is the sort of thing Simon Stent of TRI could talk for days about.
https://arxiv.org/abs/1910.10088

street scene objects perhaps ??

Data Acquisition Specialist.
About you
As a Data Acquisition Specialist, you will provide critical support to the R&D team - as they apply machine learning technology to continued development of our class leading algorithms and technology.
'experience in a technical supervisory role, especially in video image data collecting, processing and annotation (labeling) is highly desirable'.
https://seeingmachines.springboard.com.au/jobtools/jncustomsearch.searchResults?in_organid=18900&in_jobDate=All

Semicast Research@semicast_res
For vehicles produced in Japan, US and EU, Toyota declare Safety Sense install rate went from zero to 90% in 3.5 years. EU mandate timings look very sensible and well planned.

Thanks Jazz, That sounds about right. Glad one of us was paying attention;)

Of interest also maybe that Magna make reference to a Lyft / AVCC article which talks again of 'unprecedented collaboration' in the auto industry. It's unlikely that SEE is not in the AVCC mix imo but we have had no confirmation of that yet.
https://medium.com/lyftlevel5/self-driving-requires-unprecedented-collaboration-c2f7762f3c70

Morning TLC, thanks for the feed back,
A couple of observations which maybe relevant.
Colin Semicast talked about this and his CMD comments as reported by Faz (thankyou) were
'CB then talked through his approach to saving on costs. Front and Interior facing cameras should both be in the rear view mirror assembly. All the vision processing should take place in the rear view mirror assembly to avoid having to transport data/video around the vehicle. Integrate all processing onto a single chip'.

Also the Continental and Magna article below may suggest that Continental and Magna may have a working relationship.
https://www.wardsauto.com/technology/continental-magna-go-driverless-mbs

I also felt from a cursory scan of both the patents (Continental and Magna) I posted yesterday that the Conti patent could technologically related as they both related with differing sophistication to in-car driver navigation of the various displays. The more recent Conti patent I wondered maybe a enhanced version of the Magna Patent. I also recall posting regarding Magna in relation to SEE a good while ago and Lewbo also makes possible reference in his chart so could be related.

'However, the EU mandate includes all new Trucks and Buses also, how do the truck OEMs (of which Daimler is the largest ) comply, I have heard no mention of a Truck or Bus OEM tender or do they all fit Guardian to new trucks and buses ?'

S2020.. Alastair Protheroe talked about this with Jazz and I at the CMD. I seem to remember him saying that due to various parts of the trucks/buses being made by different companies for other companies that it was not practical to put DMS in as a factory fitted box so likely as you suggest Guardian would be fitted as a after market.

Jazz is that your recollection ?

Continental and 3M partner to develop infrastructure-to-vehicle (I2V) interfaces
15 October 2019
Continental entered into a collaborative evaluation partnership with 3M to assess the infrastructure-to-vehicle (I2V) interface between infrastructure-related objects and technologies (including pavement markings, conspicuity film, signs and other landmarks on or near the roadway environment), and vehicle-related technologies.
Continental has a long history in automotive safety systems. This extensive know-how, combined with 3M’s innovative expertise and leadership in roadway technologies, enables the two companies to collaborate on improving the current transportation infrastructure and work toward the goal of a more balanced and intelligent infrastructure optimized future solution. The short-term goal is to help increase road safety while continuing to work toward automated driving solutions.
Continental and 3M will collaboratively evaluate the infrastructure and vehicle interface in the following ways: localization and correction services using clear environmental landmarks for locating a vehicle on a high-definition map; identification and classification of objects in urban roadway environments including urban cross-walks, signalized intersections and other stationary and moving objects to improve safety; improved awareness of objects in and around work zones and the status of the work zones; and enhanced object detection using sensor fusion detectable infrastructure-related objects.
https://www.greencarcongress.com/2019/10/20191015-conti.html

[0039] The present invention thus provides a display at a windshield area of the vehicle, with the display being at least partially transparent or non-discernible in its transmissive or transparent state (such as when it is deactivated). When activated (or changed to its display state), the display unit may display images captured by one or more cameras of the vehicle, such as a rearward viewing camera and/or such as multiple cameras of a surround view system of the vehicle. The display unit may be activated in response to a determination that the driver of the vehicle is looking at the display area, so that the displayed images are only displayed when the driver wants to see them. When driving the vehicle forwardly, the driver may only occasionally want to view rearward of the vehicle, so the display unit is deactivated except when a determination is made that the driver is looking at or towards the display area. Optionally, the display may comprise a reflective element that, when in its active or non-transparent state, provides a reflector to allow the driver to view rearwardly via reflection off of the reflector of the display.

[0040] Optionally, the transparent display and focal/gaze detection system of the present invention may be used for a navigation screen, whereby the navigation screen may be activated and visible or viewable when the system determines that the driver is looking at the navigation screen display location but otherwise would not be activated and viewable. Optionally, the navigation screen (even when not being viewed and thus not viewable) would automatically activate when the vehicle is approaching a turn or exit or the like on the programmed route. Optionally, the system may include two transparent displays, with a rearview display (functioning like a rearview mirror and operable to display exterior images when viewed by the driver) above a navigation display (functioning as a navigation screen to show a map or driving directions or the like when that area is viewed by the driver), with the system determining which location the driver is looking at and activating the appropriate display according to the determined driver's gaze.

[0041] Thus, during normal driving when the driver is viewing ahead of the vehicle while driving the vehicle in a forward direction along a road, neither screen may be activated and the driver can view through the entire windshield. When the driver wants to view rearward (such as during a reversing maneuver or when making a lane change or the like), the driver may gaze at the “rearview mirror” location and the first display may be activated to display the rearward view or the like (optionally, and desirably, the system may automatically change the display to display the rearward view during a reversing maneuver of the vehicle). Also, when the driver wants to view a map or driving instruction or other navigation information, the driver may gaze at a “navigation screen” location (which may b

[0026] Optionally, the eye gaze system may include the ability to determine whether the driver's eyes are focused at a near distance (such as the distance to the windshield) or a far distance (such as to nearly infinite such as when looking at the scene outside the vehicle). The system may engage or activate the display responsive to the focal range and to the eye gaze, and may engage or be activated when the system determines that the driver is focusing his or her gaze at the windshield plane into the area he or she expects the rear view image to appear. Typically, eye gaze systems determine the general viewing direction. To determine the focal depths of a viewer's (driver's) eye pair there are two methods, which may come into use independently or in combination. One is to determine the difference of the eyes' general viewing angle or angles to a common center. The closer the viewing point is to the driver, the higher the difference or angle is of the eyes (FIG. 3).

[0027] Another method is to project an (use-) image (which may be visible) or a test pattern (which may be in an invisible wave length) via a retinal display or a similar projection unit to the eye ball's back plane (the retina). Due to the light refraction being different when the human eye's flexible lens is relaxed to when the lens (its muscles) are strained, the image or test pattern on the retina may appear larger or smaller or sharper or less sharp to a camera looking into the eye. The camera may be similar or identical to the eye gaze camera. In practice, the test pattern may be reduced to just two infrared (IR) light rays, projecting two test points which distance may be detectable. The focus detection system may be incorporated to the above retina projector and may be similar to or the same as the sharpness (focus) control.

[0028] Thus, the system may be able to determine not only the direction of the driver's gaze, but the focal distance, in order to determine when the driver is actually focusing on the display area at the window (so as to want to view the displayed information) or on or near a point outside of the vehicle that happens to have a line of sight from the driver through the display area of the window. When the driver is viewing through the windshield to a point (such as scenery or a street sign or traffic light or the like) outside of the vehicle, it would be undesirable to have the display system activate the display, since the display would then block the driver's view. FIG. 4 shows an in-cabin scene as the driver (inside 23 the vehicle) is driving the vehicle along a road (to be seen in front of the vehicle). When the driver looks and focuses at the display area at the windshield 19, the system (such as via the eye tracking system 16) determines that the driver is focusing at the display area and activates the OLED display 17 at the windshield, whereby the rearview image is displayed and the back light blocking blackening or darkening LC is turned on

METHOD FOR DISPLAYING INFORMATION TO VEHICLE DRIVER 2018-11-01
A method for displaying information to a driver of a vehicle includes disposing a camera at a vehicle and disposing a bimodal display at a display region at a windshield of the vehicle. Responsive at least in part to determination that gaze of the driver is not at the display area or gaze of the driver is at the display region and the driver is not focusing at the display region for the threshold period of time, the bimodal display operates in a first mode, where the bimodal display is transmissive of visible light therethrough. Responsive at least in part to determination that gaze of the driver is at the display region and the driver is focusing at the display region for the threshold period of time, the bimodal display operates in the second mode, where images derived from captured image data are displayed at the display region.

[0018] Referring now to the drawings and the illustrative embodiments depicted therein, a vehicle 10 includes an imaging system or vision system 12 that includes at least one exterior facing imaging sensor or camera, such as a rearward facing imaging sensor or camera 14a (and the system may optionally include multiple exterior facing imaging sensors or cameras, such as a forwardly facing camera 14b at the front (or at the windshield) of the vehicle, and a sidewardly/rearwardly facing camera 14c, 14d at respective sides of the vehicle), which captures images exterior of the vehicle, with the camera having a lens for focusing images at or onto an imaging array or imaging plane or imager of the camera (FIG. 1). The vision system 12 includes a control or electronic control unit (ECU) or processor 18 that is operable to process image data captured by the cameras and may provide displayed images at a display device or unit 17 for viewing by the driver of the vehicle (although shown in FIG. 1 as being at a windshield 19 of the vehicle, the control and/or the display device may be disposed elsewhere at or in the vehicle). The data transfer or signal communication from the camera to the ECU may comprise any suitable data or communication link, such as a vehicle network bus or the like of the equipped vehicle. The display system may be responsive to or may include a driver gaze detection system or driver monitoring system 16, which is operable to determine the driver's gaze direction to determine when the driver is looking at the display area.
https://worldwide.espacenet.com/publicationDetails/description?CC=US&NR=2018312111A1&KC=A1&FT=D&ND=3&date=20181101&DB=&locale=en_EP#

DISPLAY SYSTEM IN A VEHICLE
CONTINENTAL AUTOMOTIVE GMBH [DE]; CONTINENTAL AUTOMOTIVE SINGAPORE PTE LTD [SG] +
The invention relates to a display system in a vehicle, the system comprising:one or more displays;at least one sensor configured to determine a user's line of sight;a controller configured to determine an active zone and a non-active zone of the one or more displays based on the user's line of sight,wherein the one or more displays are configured to operate the active zone at an enhanced level as compared to the non-active zone.There is further provided a controller configured to rank content displayed on one or more displays in a vehicle according to an attention score determined by at least one sensor configured to measure gaze of a user based on the user's line of sight; and a display system comprising the controller,wherein one of the displays is selected to display the highest ranked content.There is also provided a method of operating one or more displays in a vehicle using the disclosed display systems.

Interior cameras or optical sensors gather raw optical data, using for example colour CMOS sensor, time of flight sensor, etc, to track the user's eyes. The tracking and prediction of user's line of sight, also known as eye-gaze detection, are done with known methods such as pattern recognition, corneal reflection points, dark and bright pupil effect or combinations thereof. Where the user' s eyes are partially or fully obscured, the head or body movement sensor can assist to determine the direction of user' s view and estimate the line of sight . Head movement sensors are capable of sensing the head's lateral movements and rotation movements (e.g. roll, yaw and pitch) . Body movement sensors are capable of sensing movement of a subject's body in a plurality of directions. Therefore, such head or body movement sensors are capable of detecting gestures from the user. In some instances, such optical sensors or head or body movement sensors may be individual devices on its own. In other instances, the sensors may be integrated as one device; for example, the head or body movement sensors may be embedded or integrated into the interior camera .

The sensor (s) may constantly monitor user's eyes, head and/or body movement and predict the user's line of sight. The user's line of sight is therefore used to determine which display or displays and/or which content is required or desired by the user. The display (s) and/or content are then grouped into areas by the controller. That is, the controller takes in input from the sensor (s) and maps the input into areas or zones within the group of displays and/or within a single display. The area in which the user is gazing at or focussing on is termed the "active zone". Consequently, all other display areas will be considered the "non-active zone".

Methods and Systems for Head Up Display (HUD) of Aerial Refueling Operation Status and Signaling.
Example methods and systems for providing information to a pilot of a receiver aircraft during a refueling operation are described. The refueling operation includes a tanker aircraft with a hose for supplying fuel to the receiver aircraft and a refueling coupler coupled to an end of the hose. One example method includes, receiving at a first transceiver located at the receiver aircraft refueling information from a second transceiver located at the tanker aircraft, receiving from a vision system located at the receiver aircraft image information indicative of the hose and the refueling coupler, and displaying, on a head up display (HUD) of the pilot of the receiver aircraft a visual representation of one or more of the refueling information and the image information.
[0057] The vision system 134 may also include other types of sensors such as Infrared, Visual, Multi-Spectral, Electro-Optical, or other sensors useful to provide data for object tracking, for example.
[0079] The visual representation 172 may be overlaid onto the refueling coupler 130 as seen from the viewpoint of the pilot by tracking the position of the refueling coupler 130, tracking the position of the head/field of view of the pilot, and mapping the visual representation 172 to a corresponding position overlaid onto the position of the refueling coupler 130. To track the position of the head of the pilot, a helmet worn by the pilot may have a helmet mounted global positioning system (GPS) or inertial measurement unit (IMU) system to establish a helmet view angle. Helmet tracking can also be performed using a magnetic mapping within the ****pit 135 of the receiver aircraft 112 between the helmet and fixed locations to establish a helmet view angle. In further examples, a ****pit-mounted video vision system can be used with helmet mounted targets to establish a helmet view angle. In another example, a close range wide-field LIDAR can be used in the ****pit 135 to establish a helmet view angle.
https://worldwide.espacenet.com/publicationDetails/biblio?CC=US&NR=2019272660A1&KC=A1&FT=D&ND=3&date=20190905&DB=&locale=en_EP#
https://www.bing.com/profile/history?FORM=EDGEHS

I agree S2020. I also believe 60-70%++ penetration because industry decision makers are and need to look beyond DMS in the car primarily aimed at Distraction and impairment and whatever the current wave of regulations stipulate. They must I believe in the decisions they have and are making about which DMS supplier to use be looking towards the horizon that the AVCC describe where the future is 'shared platforms' , 'Common computing platforms' and where 'unprecedented industry collaboration' is key to company survival.

In this environment how do the second and third best companies fare without naturalistic driving Data that Fleet provides and a depth of IP that allows the best and most thermally efficient way of internal and external data to be processed ,in an environment where ADAS data and driver object perception data fusion of the outside of the vehicle environment are so important for systems/ platform development.

Personally I understand why Paul is telling us that the company is focused on the execution phase and not so hot as it historically has been on innovation and R&D, but sorry Paul I'm not buying your pendulum swing narrative at all as I'm convinced it would be commercially unwise to take the foot of the 'innovation' pedal at this time especially when one considers the AVCC comments and why the industry feels it needs the AVCC to develop future generations of vehicles.

'As the automotive industry pushes forward to meet the needs of an increasingly connected environment, vehicles are evolving to become more intelligent, more ecologically friendly, and more affordable. Looking ahead, future autonomous vehicles will be operated with even more complicated and large-scale software driven by artificial intelligence. Vehicle edge computing platforms will play a key role in achieving the required ultra-high-performance computing within the power, thermal, size, security, and safety constraints. A standardized technical framework is also critical in order to share, re-use and incrementally improve developments resulting from this widescale automotive ecosystem'.

https://finance.yahoo.com/news/renesas-electronics-joins-autonomous-vehicle-120000252.html
https://www.iotglobalnetwork.com/iotdir/2019/10/10/new-consortium-to-develop-a-common-computing-platform-for-autonomous-vehicles-21617/

Thanks TLS, fair point.
Do you see that changing at all with the object detection/ identification / sensor fusion / outward and interior facing camera developments involved with vehicle to vehicle communication or will all that Data be owned by the car companies?

Interesting observation from Christian92 and the read across to SEE unique data.
Chris@Christiano92
Mobileye gets 1.24 million miles of data a day (186 million miles total) and will have all of Europe mapped by Q1 2020 and all of US mapped by end of 2020 (highway and surface streets) @Mobileye "Nobody other than Mobileye has this ability"...yes not even @Tesla or @Waymo
https://twitter.com/Christiano92/status/1191715679969386500