London South East | The UK's favourite investor portal.

TLS, maybe we will see this technology mix in the Lyiriq.

Systems and methods for in-vehicle augmented virtual reality system / GM
http://translationportal.epo.org/emtp/translate/?ACTION=description-retrieval&COUNTRY=CN&ENGINE=google&FORMAT=docdb&KIND=A&LOCALE=en_EP&NUMBER=110717991&OPS=ops.epo.org/3.2&SRCLANG=zh&TRGLANG=en

CIRCULAR LIGHT ELEMENT FOR ILLUMINATION OF CORNEA IN HEAD MOUNTED EYE-TRACKING / SEE
https://worldwide.espacenet.com/publicationDetails/description?CC=US&NR=2016363995A1&KC=A1&FT=D&ND=3&date=20161215&DB=&locale=en_EP

NAVIGATION BASED ON SENSED LOOKING DIRECTION OF A PEDESTRIAN
The present disclosure relates to systems and methods for host vehicle navigation. Disclosed systems and methods may receive, from a camera, a plurality of images representative of an environment of the host vehicle; analyze the plurality of images to identify at least one pedestrian in the environment of the host vehicle; identify eyes of the at least one pedestrian represented in at least one of the plurality of images; and determine, based on analysis of the at least one of the plurality of images and based on the identification of the eyes of the at least one pedestrian .

0401] Camera 2007a may include any type of device suitable for capturing at least one image from an environment. Camera 2007b, similar to camera 2007a, may include any type of device suitable for capturing at least one image from an environment. Camera 2007b may be associated with an interior of system 2000 (e.g., to capture images of occupants thereof) while camera 2007a may be associated with an exterior of system 2000 (e.g., to capture images of an environment thereof). Moreover, although depicted in FIG. 20 as a single camera associated with the interior and a single camera associated with the exterior, any number of image capture devices may be used to acquire images for input to the image processor. Some embodiments may include only a single image capture device for the interior and exterior, respectively, while other embodiments may include two, three, or even four or more image capture devices for the interior and exterior. Some embodiments may include more cameras associated with the exterior than the interior, and other embodiments may include more cameras associated with the interior than the exterior.

0402] Eye tracking system 2009 may include any type of device suitable for tracking eye movements of one or more occupants of system 2000. For example, eye tracking system 2009 may comprise one or more cameras configured to identify pupils, irises, corneas, and other features associated with eyes in captured images and track the identified features across frames. Accordingly, eye tracking system 2009 may include a processing device configured to identify a looking direction of an identified eye across frames. Additionally or alternatively, eye tracking system 2009 may transmit light (e.g., infrared red or the like) for reflection off one or more portions of an eye (such as a cornea or lens). Based on the resulting Purkinje image(s), eye tracking system 2009 may identify a looking direction of an identified eye across frames. Other biofeedback devices may be used. For example, a heart rate monitor may be used to sense the heart rate of an occupant of the vehicle and deduce a state of the occupant from the occupant's heart rate One or more sensors embedded in the car seats may determine how many occupants are in the car and their sitting locations.
https://tinyurl.com/y2bcmthj

This patent got my attention mainly due to the school association with CAT and the medical application it describes. This along with the wider possibilities the application of the technology being developed here may have for CAT for remote robotic work if they are involved. It also shows how gaze tracking is evolving and being developed for a range of commercial uses.

Autonomous robotic laparoscope based on eye tracking.
The presently disclosed devices, methods, and systems involve direct and intuitive visualization using gaze-control in robotic laparoscopy. For example, when the system detects a new visual interest, the robot may guide the scope to approach or move toward the target view. In order to achieve the disclosed control, a system coordinate transformation is developed. The disclosed devices, methods, and systems may translates gaze positions on image (in pixels) to relative rotation angles and/or translation distances of the laparoscope. In most cases, this relationship, which may be built, in part, on the parameters of the laparoscope and the inserted laparoscope length in the patient's body, may allow the robot to directly follow the surgeon's gaze position on the monitor. The disclosed devices, methods, and systems may help to reduce cognitive and physical burdens on a laparoscopic surgeon.

Turning now to FIG. 2, is a circuit diagram for the gaze detection controller 200 for the robotic device and system of FIG. 1. The gaze detection controller 200 may have a CPU or processing element 202 for controlling various functions of the controller 200 and may be in electrical communication with other elements of the controller 200. A storage device 204 is provided which may have operation instructions, calibration data, applications for controlling various modules, and the like. An image processing module 210 may input image information from the wide and narrow FOV cameras 250, 256 in order to extract features of the image. The wide camera 250 may be used to detect a user's face (see FIG. 4), while the narrow camera 256 may be used to determine the visual axis of a user's eyes for use with the eye tracking module. Narrow cameras 256 may each be provided with panning and tilting capability, and the gaze detection controller may operate the pan and tilt motors 254 to ensure the narrow cameras 256 are focused on each of the user's eyes, respectively. The gaze detection controller 200 may further control infrared (IR) illumination sources 252 to shine IR light onto the user's eyes. In some examples, more than one IR source is provided, thus projecting a predetermined pattern of reflections or “glints” on the eye.
https://tinyurl.com/y4trdbm6

https://www.caterpillar.com/en/careers/careersnews/coloradoschoolofmines.html.html
https://mechanical.mines.edu/project/zhang-xiaoli/
https://minesmagazine.com/9699/

VEHICLE EMERGENCY V2X NOTIFICATION BASED ON SENSOR FUSION
Methods, systems, computer-readable media, and apparatuses for vehicle emergency vehicle-to-anything (V2X) notification based on sensor fusion are presented. In some embodiments, a vehicle may receive an indication of an issue or impairment of the vehicle or an occupant of the vehicle, including the driver. Using information from sensors within the vehicle, an on-board processor may determine the nature of the impairment. Information about the nature of the impairment may then be transmitted, using V2X notification, to neighboring vehicles and/or an infrastructure entity (e.g., a base station, a roadside unit, a radio tower, and/or a central application server). The neighboring vehicles may then negotiate using V2X communication with the impaired vehicle and the other neighboring vehicles and/or be directed by the infrastructure entity to proceed using a negotiated course of action that accounts for the nature of the impairment in the impaired vehicle.

[0032] Impaired vehicle 120 may include V2X notification with sensor fusion system 200, wireless communication antenna(s) 235, and sensors 230. The sensors 230 may include any suitable sensors within impaired vehicle 120. For example, sensors 230 may include one or more vehicle malfunction sensors such as a tire pressure sensor, an oil pressure sensor, an engine temperature sensor, a collision sensor, and/or the like. Sensors 230 may also include occupant sensors that may determine the status of one or more occupants of the impaired vehicle 120 including both the driver and/or passengers. For example, sensors 230 may include a biometric sensor to determine whether an occupant has a health impairment, a camera or other visual detection sensor that detects fainting, sleeping, and/or other gesture recognition of the driver and/or passengers in the impaired vehicle

[0036] Impairment determination subsystem 215 may receive the signal from signal processing subsystem and/or information extracted from the signal by signal processing subsystem 205 as well as information from sensor polling subsystem 210. Using the information obtained from signal processing subsystem 205 and/or sensor polling subsystem 210, impairment determination subsystem 215 may determine the nature of the impairment. For example, biometric sensors that indicate for example, that driver eyelid position is low through visual identification of the eyelid position in addition to swerving sensors that may indicate the impaired vehicle 120 is not travelling steadily may indicate that the driver is fatigued or substance use impaired. Impairment determination subsystem 215 may include various algorithms and machine learning algorithms that may analyze the sensor indicators and combine the information to determine the nature of the impairment and/or if there are more than one impairment.
https://tinyurl.com/y6yshqoa

[0028] Aspects of the present disclosure include lighting systems and associated controls that can provide active control of lighting device settings, such as on/off, color, intensity, focal length, beam location, beam size and beam shape. In some examples, control systems can provide lighting system control in a sterile manner, which can be advantageous for surgical lighting applications. In some examples, lighting systems may include eye tracking technology and sensor feedback for one or more lighting device settings and may be configured with depth perception and cavity or incision recognition capability through image or video processing. Surgical lighting systems of the present disclosure may also include surgical lighting presets, in which a lighting device such as a surgical headlamp can be tailored and optimized for particular surgeries. For example, beam size, beam shape, focal length, intensity, etc., may be preset ahead of time, reducing a surgeon's setup time in the operating room. Aspects of the present disclosure also include machine learning algorithms configured to optimize one or more lighting device settings, for example, based on a user's movements, head position, duration of use, etc. Aspects of the present disclosure also include calibration procedures for calibrating an eye position sensor with an adjustable lighting device for a particular user.

0029] Intelligent lighting devices with automatic lighting adjustment made in accordance with the present disclosure may be used in surgical theaters to improve sterility in a surgical environment and provide a sterile means of control and automation of surgical lighting, which can have a positive influence on a surgeon's ability to perform his or her tasks. Lighting devices disclosed herein can help reduce the dependence of a surgeon on other people within the operating room to make changes to surgical lighting device settings. Benefits may also include glare reduction in the surgical field and operating room. Surgical rooms typically include a large number of metallic items that cause specular reflections and unwanted glare. By tailoring the settings of surgical lighting and enabling improved lighting control during surgery, glare can be reduced and surgical performance improved. For example, incision size can be reduced and tissue trauma can also be reduced from pulling occluding tissue out of the way. These improvements can directly lead to improved clinical outcomes, as reducing incision size and tissue trauma may result in mitigating tissue trauma and potentially shortening recovery times.

Lighting Devices with Automatic Lighting Adjustment

[0006] In one implementation, the present disclosure is directed to a method of controlling a lighting device. The method includes receiving, at a processor, position sensor data, the position sensor data representing a position of at least one of a user's head and a user's eyes, determining, by the processor, whether the user is looking at a task area based on the position sensor data, and controlling, by the processor, an intensity of light emitted by the lighting device or turning the light on or off in response to determining whether the user is looking at the task area.

[0007] In some embodiments, the position sensor data is head position sensor data indicating a position of a user's head, and determining whether the user is looking at the task area includes calculating, by the processor, an angle of the user's head from the head position sensor data, and determining, by the processor, whether the calculated angle is within a first range of angles associated with the user looking at the task area. In some embodiments, the position sensor data is eye position sensor data indicating a position of a user's eyes, and determining whether the user is looking at the task area includes determining, by the processor, whether the eye position sensor data is within a first range of positions associated with the user looking at the task area. In some embodiments, the first range of positions are associated with the user looking through loupes. In some embodiments, the method further includes determining, by the processor, a location where the user is looking based on the eye position sensor data, and controlling, by the processor, a location of a beam of light emitted by the lighting device to be substantially coincident with the location where the user is looking. In some embodiments, the eye position sensor data is collected by an eye position sensor, and the method further includes pointing the lighting device at a first one of a plurality of targets, recording sensor data from the eye position sensor while a user is looking at the first one of the plurality of targets, and determining a set of calibrating parameters for translating a coordinate of the sensor data to a lighting device coordinate. In some embodiments, the set of calibration parameters are parameters of a two-dimensional linear approximation. In some embodiments, the set of calibration parameters represent a translation and a rotation of a coordinate system of the eye position sensor to a coordinate system of the lighting device.
https://worldwide.espacenet.com/publicationDetails/biblio?CC=US&NR=2020205268A1&KC=A1&FT=D&ND=3&date=20200625&DB=&locale=en_EP
Drawings
http://www.freepatentsonline.com/20200205268.pdf

'Bosch VW link very interesting as Bosch have a close alignment to SM'

SYSTEM AND METHOD FOR IDENTIFYING A CAMERA POSE OF A FORWARD FACING CAMERA IN A VEHICLE/ SEE
A method includes capturing images of a vehicle driver's face from a driver facing camera and images of a forward road scene from a forward facing camera. The images are analyzed to derive gaze direction data in a vehicle frame of reference. The gaze direction data are statistically collated to determine a frequency distribution of gaze angles. One or more peaks in the frequency distribution are identified and associated with corresponding reference points in the images to determine one or more reference gaze positions in the vehicle frame of reference. The one or more reference gaze positions are correlated with a position of the reference points in a forward facing camera frame of reference to to determine a camera pose of a forward facing camera in the vehicle frame of reference.
https://tinyurl.com/yyta33hh

CAMERA REGISTRATION IN A MULTI-CAMERA SYSTEM / SEE
Described herein is a method of registering a position and an orientation of one or more cameras in a camera imaging system. The method includes: receiving, from a depth imaging device, data indicative of a three dimensional image of the scene. The three dimensional image is calibrated with a reference frame relative to the scene. The reference frame includes a reference position and reference orientation. A three dimensional position of each of the cameras within the three dimensional image is determined relative to the reference frame. An orientation of each camera is determined to relative to the reference frame. The position and orientation of each camera is combined to determine a camera pose relative to the reference frame.
https://tinyurl.com/y2q7o45c

SEMANTIC SEGMENTATION USING DRIVER ATTENTION INFORMATION /Bosch
[0007] Another embodiment provides a method for creating a trained semantic segmentation neural network model and a trained attention neural network model to operate a vehicle. The method includes receiving, via one or more electronic controllers, an image from an outside view camera and an image of a driver's eye movements from a driver-facing camera. The method also includes calibrating, via the one or more electronic controllers, the image of the driver's eye movements with the image from the outside view camera to create calibrated images used to create a pixel weighted heat map of the calibrated images. The method also includes creating, via the one or more electronic controllers, the trained semantic segmentation neural network model using the image from the outside view camera and creating the trained attention neural network model using the pixel weighted heat map. The method also includes operating, via the one or more electronic controllers, the vehicle using the trained semantic segmentation neural network model and the trained attention neural network model.
https://tinyurl.com/y

APPARATUS AND METHOD FOR DEFINING AND INTERACTING WITH REGIONS OF AN OPERATIONAL AREA
A display apparatus and method for displaying an operational area to an operative of a host platform, said operational area being defined within an external real-world environment relative to said host platform, the apparatus comprising a viewing device (12) configured to provide to said operative, in use, a three-dimensional view of said external real-world environment; a display generating device for creating images at the viewing device, a user input (33) configured to receive user input data (35) representative of a specified target or region in respect of which an operation is to be performed, and thereby defining an initial geometric volume for the operational area, said user input data including data representative of the location within said external real-world environment of said specified target or region and data representative of said operation to be performed in respect thereof; and a processor (32) configured to: use said user input data to generate or obtain three-dimensional image data representative of an adjusted geometric volume based, at least, on said initial geometric volume and on said operation to be performed, and display one or more images depicting said adjusted geometric volume and created using said three-dimensional image data, on said display generating device, the apparatus being configured to project or blend said one or more images displayed on said display generating device into said view of said external real-world environment at the relative location therein of the specified target or region.

[0011] The apparatus may further comprise one or more input devices configured to enable an operative to interact with the one or more images on the viewing device to move it/them to a desired location within their view of the external real-world environment and/or manually adjust the shape, size and/or orientation of the one or more images relative thereto. The input device(s) may comprise one or more of a plurality of controls, such as a conventional cursor with joystick, mouse or similar manual control device, touch controls via a touchscreen device, arm, hand or finger gestures, head movement, eye tracking technology, etc. A range of multi modal control means for manipulating image data on a touchscreen will be known to a person skilled in the art, and the present invention is not necessarily intended to be limited in this regard.
https://worldwide.espacenet.com/publicationDetails/biblio?CC=US&NR=2020219314A1&KC=A1&FT=D&ND=3&date=20200709&DB=&locale=en_EP

BOEING CO 2020-07-09
SYSTEM AND METHOD FOR FOVEATED SIMULATION
A system includes a head mounted display (HMD), a processor, and a memory. The HMD is configured to generate sensor data indicative of a gaze vector of a user and a position of the user. The processor is configured to communicate with the HMD. The memory is coupled to the processor and stores instructions that, when executed by the processor, cause the processor to obtain data indicating the gaze vector of the user and the position of the user. The instructions also cause the processor to determine a component vector for a virtual component of a virtual reality simulator based on the position of the user and a position of the virtual component. The instructions further cause the processor to calculate an alignment value based on the gaze vector and the component vector, and adjust a simulation setting of the virtual component based on the alignment value.

BAE SYSTEMS PLC 2020-07-09
HEAD-MOUNTED DISPLAY AND CONTROL APPARATUS AND METHOD
An apparatus and method for displaying an operational area, the apparatus comprising a headset (10) for placing over a user's eyes, the headset including a viewing device (12) configured to provide to said user, in use, a view of a real-world environment, a display generating device for depicting an operational area, said operational area being defined within said real-world environment and comprising a plurality of functional regions each defining a different one or more selectable functions or operations that can be performed in respect of said operational area, the apparatus being configured to transfer image data from said display generating device into said user's view of said real-world environment at said viewing device to generate an augmented reality environment, the apparatus further comprising a control module (26) including a control device (28) configured to be selectively communicably coupled to all of said functional regions to enable a user to selectively perform the respective one or more functions or operations associated therewith, wherein the control device (28) is communicably coupled to a selected functional region only in response to a respective actuation signal, and an eye tracker module (24) configured to monitor said user's gaze relative to said augmented reality environment, in use, and, when said user's gaze is directed at a selected functional region displayed therein, generate and transmit to said control module (26) a said actuation signal and when said user's gaze is no longer directed at said selected functional region, cause said control device (28) to be decoupled therefrom.

Latest Share Chat

Member Info for johnchucka

Followers

Following

Posts (Last 30 Days)

Posts (All Time)

Last Post

Member Since

Share Chat Post Distribution (Last 30 Days)

Activity Log (Last 30 Days)

Latest Share Chat

Member Info for johnchucka

Followers

Following

Posts (Last 30 Days)

Posts (All Time)

Last Post

Member Since

Share Chat Post Distribution (Last 30 Days)

Activity Log (Last 30 Days)

Please enter at least 2 characters

No results found

Search results

Recently viewed

Trending