Member Info for johnchucka

[0082] In some implementations, the first portion 108 may be identified using eye position information, such as the eye position information 504. The method 800 may include receiving the eye position information 504 (e.g., from the camera 550 of FIG. 5) and identifying the first portion 108 further based on the eye position information 504.

[0091] The electronic device 900 may further include a camera 950 (e.g., the camera 550 of FIG. 5). The camera 950 may be configured to generate the eye position information 504 of FIG. 5, such as by capturing one or more images of a user that indicate the eye position information 504. The fovea estimation engine 104 may be configured to identify an area of interest based on the eye position information 504, as described with reference to FIG. 5.

[0061] In an illustrative example, the eye position analyzer 510 may be configured to receive the eye position information 504 from the camera 550 (e.g., a sensor device) that tracks eye position of a user. For example, in some applications, a headset may be worn by a user, and the headset may include the camera 550. The eye position analyzer 510 may be configured to determine an actual eye position 520 based on the eye position information 504.

[0062] The eye pose estimator 512 may be configured to determine an estimated eye position 516 of the user. For example, the eye pose estimator 512 may be configured to determine the estimated eye position 516 based on information provided by one or more of the graphics metric analyzer 506, the API calls analyzer 508, or the eye position analyzer 510. The first portion 108 may be identified using (or may correspond to) the estimated eye position 516. In some implementations, the indication 106 may identify the estimated eye position 516.

[0063] The fovea estimation engine 104 may be configured to determine an estimation error 522 (e.g., a difference between the estimated eye position 516 and the actual eye position 520). The fovea estimation engine 104 may be configured to use the estimation error 522 in connection with one or more subsequent operations. For example, the fovea estimation engine 104 may be configured to use the estimation error 522 to “predict” subsequent eye positions of a user.

[0068] The fovea estimation engine 104 may be configured to rank bins associated with the first frame 114 based on the scene information 110 of FIGS. 1-5. In the illustrative example of FIG. 6, a ranking may correspond to a value selected from the integers 1, 2, . . . 10. To illustrate, the fovea estimation engine 104 may be configured to assign a ranking of 6 to the first bin 202 and a ranking 2 to the second bin 204. In this example, a ranking of 6 may indicate that a frame portion within the first bin 202 is more likely to draw the gaze of a user than a frame portion within the second bin 204.

[0069] Although the binning and ranking process 600 of FIG. 6 is described with reference to two portions 108, 118, it should be appreciated that the binning and ranking process 600 may be performed using a different number of areas of the graphics content 102 (e.g., using three or more areas). Further, different areas may be rendered using different amounts of foveating. For example, a “spectrum” of foveating may be applied to the graphics content 102 so that differently ranked areas are foveated based on different amounts. In some cases, the binning and ranking process 600 may enable foveating of multiple areas of interest. For example, although user gaze may be focused on the soccer ball illustrated in FIG. 6, in some cases both the soccer ball and the soccer player of FIG. 6 may be foveated.

[0005] An electronic device may track gaze of a user (e.g., using a sensor) to identify a graphics portion to be foveated. For example, if the electronic device determines that eye movement of the user follows a particular object presented at a display, the electronic device may foveate the object. In some cases, tracking error or latency may cause the electronic device to incorrectly track the gaze of the user. As a result, foveated rendering of graphics may be inaccurate in some circumstances (e.g., due to tracking error, latency, or both).

[0021] By foveating graphics content based on scene information, use of a sensor to track eye position of a user may be reduced or avoided. As a result, performance may be improved, such as by avoiding latency associated with tracking of eye position. In addition, multiple areas of graphics content may be foveated concurrently, which may improve user experience in some cases.

[0022] FIG. 1 depicts an illustrative example of a device 100. The device 100 may be included in a graphics processing unit (GPU) that is configured to perform a graphics processing operation using graphics content 102, as an illustrative example. In some examples, the graphics content 102 may correspond to an augmented reality (AR) application or a virtual reality (VR) application.

[0032] The rendering engine 112 is configured to render the graphics content 102 using predictive adjustment to reduce latency associated with foveating an area of interest, such as the first portion 108. For example, by rendering the graphics content 102 based on the indication 106, the rendering engine 112 predictively adjusts (e.g., predictively foveates) the graphics content 102 (e.g., instead of “waiting” for eye tracking information from a sensor device, such as a camera, that indicates eye position of a user). As used herein, “predictive adjustment” may refer to foveating the area of interest based on graphics processing operations performed prior to presenting the area of interest at the display 150 (alternatively or in addition to foveating the area of interest based on eye tracking information). Predictively adjusting the graphics content 102 reduces latency as compared to “waiting” for eye tracking information (e.g., after presenting the graphics content 102 at a display).

[0033] By foveating the graphics content 102 based on the scene information 110, use of a sensor to track eye position of a user may be reduced or avoided. As a result, device performance may be improved, such as by avoiding latency associated with tracking eye position.

0058] FIG. 5 illustrates a particular illustrative example of a system 500 that includes the fovea estimation engine 104 and a camera 550. In the example of FIG. 5, the fovea estimation engine 104 includes a graphics metric analyzer 506, an application program interface (API) calls analyzer 508, and an eye position analyzer 510. The fovea estimation engine 104 may further include an eye pose

SCENE-BASED FOVEATED RENDERING OF GRAPHICS CONTENT
An apparatus is configured to render graphics content to reduce latency of the graphics content. The apparatus includes a display configured to present graphics content including a first portion corresponding to an area of interest and further including a second portion. The apparatus further includes a fovea estimation engine configured to generate an indication of the area of interest based on scene information related to the graphics content. The apparatus further includes a rendering engine responsive to the fovea estimation engine. The rendering engine is configured to perform a comparison of a first result of an evaluation metric on part of the area of interest with a second result of the evaluation metric with another part of the area of interest. The rendering engine is further configured to render the graphics content using predictive adjustment to reduce latency based on the comparison.
https://worldwide.espacenet.com/publicationDetails/description?CC=US&NR=2020066024A1&KC=A1&FT=D&ND=3&date=20200227&DB=&locale=en_EP#
Drawings;
http://www.freepatentsonline.com/20200066024.pdf

MIXED REALITY OBJECTS
https://worldwide.espacenet.com/publicationDetails/description?CC=US&NR=2020066059A1&KC=A1&FT=D&ND=3&date=20200227&DB=&locale=en_EP#
http://www.freepatentsonline.com/20200066059.pdf

[0029] HMD device 104 may indicate user interactions with third-party objects in various manners and with any suitable level of granularity. For example, HMD device 104 may provide indications that differentiate among different paradigms of user input—e.g., the HMD device may separately indicate a gesture input, a gaze input, a voice input, input provided via a peripheral input device coupled to the HMD device, biometrics of user 110, and/or other input types. More granular data may accompany indications, such as data indicating a gaze pattern (e.g., a time series of gaze positions), a time series of two or three-dimensional coordinates corresponding to hand 120, etc. Regardless of their form, indications reported by HMD device 104 may be used to determine positive, negative, and/or other levels of engagement with third-party objects. As examples, an indication of relatively sustained gaze on a third-party object (e.g., a minimum duration of gaze within an area that allows for saccadic movements), or relatively sustained gaze combined with a hand gesture or other input directed to an object, may be considered positive engagement with the object. Conversely, a relative lack of sustained gaze, or a relative sustained gaze that is not followed by a directed input, may be considered negative engagement with the object. In some examples, MR application 112 may support user inputs that explicitly indicate negative engagement—for example, user 110 may instruct HMD device 104 via a hand gesture and/or voice command to remove a third-party object from MR environment 100.

[0060] The computing system 600 may include various sensors and related systems to provide information to the on-board computer 604. Such sensors may include, but are not limited to, one or more inward facing image sensors 610A and 610B, one or more outward facing image sensors 612A and 612B, an inertial measurement unit (IMU) 614, and one or more microphones 616. The one or more inward facing image sensors 610A, 610B may be configured to acquire gaze tracking information from a wearer's eyes (e.g., sensor 610A may acquire image data for one of the wearer's eye and sensor 610B may acquire image data for the other of the wearer's eye). One or more such sensors may be used to implement a sensor system of HMD device 104, for example.

[0061] Where gaze-tracking sensors are included, the on-board computer 604 may be configured to determine gaze directions of each of a wearer's eyes in any suitable manner based on the information received from the image sensors 610A, 610B. The one or more inward facing image sensors 610A, 610B, and the on-board computer 604 may collectively represent a gaze detection machine configured to determine a wearer's gaze target on the near-eye display

SELF-DRIVING MOBILE ROBOTS USING HUMAN-ROBOT INTERACTIONS
Systems, methods, and computer-readable media are disclosed for enhanced human-robot interactions. A device such as a robot may send one or more pulses. The device may identify one or more reflections associated with the one or more pulses. The device may determine, based at least in part on the one or more reflections, a cluster. The device may associate the cluster with an object identified in an image. The device may determine, based at least in part on an image analysis of the image, a gesture associated with the object. The device may determine, based at least in part on the gesture, a command associated with an action. The device may to perform the action.

Successful operation of autonomous systems, such as self-driving cars and self-delivering robotic platforms, may become increasingly dependent on their ability to interact with humans, especially in environments mixed with humans, such as densely populated city centers or manufacturing assembly plants. For example, a robot may include one or more computing devices which may detect people and objects in a nearby area, and may respond to commands provided by nearby users and identified by the robot. However, when the nearby area has multiple people and/or objects, the robot may struggle to differentiate between different people and objects, and to recognize which users may be gesturing to the robot with the intent to cause the robot to execute a command associated with the gesture. Likewise, a noisy environment may result in difficulties for the robot to detect voice gestures.

Some human-robot interactions address humans interacting with autonomous vehicles such as robots and self-driving cars. In some vehicles and computer infrastructures, human-to-human interactions may be used between drivers and road users to accomplish effective driving, which may be useful for the self-driving cars to be effective. In addition to self-driving cars, human-robot interaction (HRI) may be valuable in a variety of autonomous systems, ranging from manufacturing to service robots. Such systems may interact with humans collaboratively, coordinately, and in socially acceptable ways to achieve the desired effectiveness, efficiency, and automation performance.
https://worldwide.espacenet.com/publicationDetails/biblio?II=11&ND=3&adjacent=true&locale=en_EP&FT=D&date=20200227&CC=US&NR=2020064827A1&KC=A1#
https://patents.justia.com/patent/20200064827
http://www.freepatentsonline.com/y2020/0064827.html

Drawings
http://www.freepatentsonline.com/20200064827.pdf

I get the impression they'll be plenty of 'incentivised' activity for drivers and passengers to par take in for revenue advertising purposes but also to use ware able devices for complimentary personal, object and environmental data collection purposes. I have posted a couple of Toyota patents previously where this is also described.

High Precision Low Bit Convolutional Neural Network
Described herein are systems, methods, and computer-readable media for generating and training a high precision low bit convolutional neural network (CNN). A filter of each convolutional layer of the CNN is approximated using one or more binary filters and a real-valued activation function is approximated using a linear combination of binary activations. More specifically, a non-1×1 filter (e.g., a k×k filter, where k>1) is approximated using a scaled binary filter and a 1×1 filter is approximated using a linear combination of binary filters. Thus, a different strategy is employed for approximating different weights (e.g., 1×1 filter vs. a non-1×1 filter). In this manner, convolutions performed in convolutional layer(s) of the high precision low bit CNN become binary convolutions that yield a lower computational cost while still maintaining a high performance (e.g., a high accuracy).
https://worldwide.espacenet.com/publicationDetails/biblio?II=0&ND=3&adjacent=true&locale=en_EP&FT=D&date=20200227&CC=US&NR=2020065661A1&KC=A1#

FOVEATED COLOR CORRECTION TO IMPROVE COLOR UNIFORMITY OF HEAD-MOUNTED DISPLAYS
A color correction mask programmatically generated in software on an HMD device is applied to a gaze region on a display field of view (FOV) on a head-mounted display (HMD) device to optimize system resources while rendering a display. Eye monitoring sensors are utilized to track movement of a user's eyes while the user operates the HMD device to determine a gaze position of the user's eyes on the display FOV. Using the determined gaze position, a dynamic foveal gaze region is sized around the gaze position so that the foveal portion of the display FOV is color-corrected, that is, color non-uniformities are reduced or eliminated. In other implementations, a gaze-based weighting mode is implemented in which measurements of the user's full eye or eye orbit are utilized to color correct a larger surface area of the display FOV relative to the foveal color correction mode.
https://worldwide.espacenet.com/publicationDetails/biblio?II=12&ND=3&adjacent=true&locale=en_EP&FT=D&date=20200227&CC=US&NR=2020064631A1&KC=A1#
https://worldwide.espacenet.com/searchResults?ST=singleline&locale=en_EP&submitted=true&DB=&query=gaze+
http://www.freepatentsonline.com/y2020/0064631.html
Drawings
http://www.freepatentsonline.com/20200064631.pdf

APPARATUS FOR CONTROLLING DISPLAY OF VEHICLE, SYSTEM HAVING THE SAME, AND METHOD THEREOF
A vehicle display control apparatus for controlling at least one display of a vehicle may include: a processor configured to: calculate a display control resource distribution ratio of the at least one display based on at least one of: an autonomous driving state of the vehicle, a gaze direction of a user, and a characteristic of content displayed by the at least one display, and control operation of the at least one display by allocating a control resource to each of the at least one display based on the calculated display control resource distribution ratio; and a storage configured to store the calculated display control resource distribution ratio.
https://worldwide.espacenet.com/publicationDetails/biblio?II=11&ND=3&adjacent=true&locale=en_EP&FT=D&date=20200227&CC=US&NR=2020065042A1&KC=A1#
https://worldwide.espacenet.com/searchResults?ST=singleline&locale=en_EP&submitted=true&DB=&query=gaze+
http://www.freepatentsonline.com/y2020/0065042.html
Drawings
http://www.freepatentsonline.com/20200065042.pdf

CONTROL METHOD OF AUTONOMOUS VEHICLE
A method of controlling of an autonomous vehicle according to an embodiment of the present disclosure, the method comprising the steps of: acquiring state information of a driver from a sensor mounted inside the vehicle; determining a glare state of the driver based on state information of the driver; operating a primary light source blocking when recognizing a glare state of the driver; operating, by the primary light source blocking, a light source blocking device mounted on the vehicle at the moment of recognizing the glare state of the driver, and tracking a gaze direction of the driver through a first image to acquire the gaze direction of the driver; and operating secondary light source blocking, when the acquired gaze direction of the driver is outside a predetermined range. The autonomous vehicle according to the present disclosure may be associated with an artificial intelligence module, a drone (UAV), a robot, an AR device, a VR device, a device related to 5G service, etc.
https://worldwide.espacenet.com/publicationDetails/biblio?II=6&ND=3&adjacent=true&locale=en_EP&FT=D&date=20200227&CC=US&NR=2020065596A1&KC=A1#
http://www.freepatentsonline.com/y2020/0065596.html
Drawings
http://www.freepatentsonline.com/20200065596.pdf

EYE GAZE TRACKING OF A VEHICLE PASSENGER
Systems, methods, and devices for determining an object identification hit for an object within a user's approximate field of vision during a vehicle trip are provided. A method of the disclosure includes determining eye tracking data associated with a user of a vehicle from a vehicle sensor and determining a field of vision of the user based on the eye tracking data. The method includes determining object data associated with an object within the field of vision and identifying the object within the field of vision based on the object data to produce an object identification hit. The method includes storing the object identification hit in memory.

DETAILED DESCRIPTION
Applicant recognizes that in a competitive ride sharing market, there is a need to offer variable pricing or reduced pricing to ride sharing customers. In such an environment, a customer may elect to provide, for example, eye tracking data through the duration of the ride in exchange for a reduced fare on the ride share. The ride sharing provider may maintain a profit by selling data received to, for example, advertisers, researchers, and other marketers. In such an embodiment, the ride sharing provider may sell data indicating, for example, what billboards or other advertisements the user viewed during the ride, what route the user traversed, and the user's attention span for various advertisements.

Applicant presents methods, systems, and devices for identifying objects within a user's field of vision during a ride in a vehicle. Such methods, systems, and devices may provide increased revenue for ride sharing providers by utilizing eye tracking technology and vehicle localization technology to determine the objects of focus for one or more users in a vehicle. Such objects of focus may register as object identification hits or “hits” that can be accumulated by the vehicle controller, stored in a cloud-based server, and provided to interested parties at a cost. Such hits can apply to various objects, including advertising signs, buildings, roadways, in-vehicle advertisements, mobile phone advertisements, and so forth. In various embodiments, such methods, systems, and devices may be configured to collect eye tracking data when a ride sharing user provides positive affirmation that he wishes to participate in the data collection.
https://worldwide.espacenet.com/publicationDetails/biblio?II=4&ND=3&adjacent=true&locale=en_EP&FT=D&date=20200227&CC=US&NR=2020064912A1&KC=A1#
https://patents.justia.com/patent/20200064912

Drawings
http://www.freepatentsonline.com/20200064912.pdf

SYSTEM AND METHOD FOR FOVEATED SIMULATION
A system includes a head mounted display (HMD), a processor, and a memory. The HMD is configured to generate sensor data indicative of a gaze vector of a user and a position of the user. The processor is configured to communicate with the HMD. The memory is coupled to the processor and stores instructions that, when executed by the processor, cause the processor to obtain data indicating the gaze vector of the user and the position of the user. The instructions also cause the processor to determine a component vector for a virtual component of a virtual reality simulator based on the position of the user and a position of the virtual component. The instructions further cause the processor to calculate an alignment value based on the gaze vector and the component vector, and adjust a simulation setting of the virtual component based on the alignment value.
https://worldwide.espacenet.com/publicationDetails/biblio?II=1&ND=3&adjacent=true&locale=en_EP&FT=D&date=20200227&CC=US&NR=2020068190A1&KC=A1#
https://patents.justia.com/patent/20200068190

Semicast Research@semicast_res
This is why Nvidia was never the solution.

E.W. Niedermeyer@Tweetermeyer
"Autonomy is an AI problem"
Yeah... but only if you ignore the challenge of ensuring safety-critical reliability and robustness using probabilistic machine learning in domains characterized by unbounded complexity. Otherwise, it seems like more of a systems engineering problem.

Tell that to SYM
https://www.lse.co.uk/SharePrice.asp?shareprice=SYM&share=Symphony-Env

A better view of Super Cruise in a 2021 Escalade
https://www.linkedin.com/posts/jeffrey-evangelist_teamgm-ugcPost-6637058044998008833-CP8n

Semicast Research@semicast_res
Did research on tech & small-cap insti investment analysts in London in December. Most common 1st degree: History! Am an elec.eng. & find DMS v. hard to understand. Few appreciate costs & timescales associated w/ developing mission-critical software. This from the briefing deck:
https://twitter.com/semicast_res/status/1232652625465745415/photo/1

Semicast Research@semicast_res

After the NTSB fun, a sober reminder that any car w/ L2+/hands-off/highway assist features going for type-approval in EU needs adv. distraction warning from June 22, or 24mos ahead of all others. The most advanced vehicles go first & thus will set the scene for eventual winners.

Semicast Research@semicast_res
IAA19 in September suggested a clear pivot towards safety DMS. That is likely contributor to the delays. Make up your own mind which T2s can do safety. We’ll know for sure soon enough.

S2020,
'DMS for SEE’s customer GM, which is something we have not considered before'.

If only we all could have that special relationship with reality, often drug induced psychotropic perspicacity and thinking outside the Box genius that we could imagine SEE's GM nurtured relationship with Qualcomm being displaced by SE.
What is he on FFS? I want some.

https://usawatchdog.com/covid19-real-beast-causing-dangerous-pandemic-chris-martenson/