Member Info for johnchucka

EAD-MOUNTED DISPLAY AND CONTROL APPARATUS AND METHOD
An apparatus and method for displaying an operational area, the apparatus comprising a headset (10) for placing over a user's eyes, the headset including a viewing device (12) configured to provide to said user, in use, a view of a real-world environment, a display generating device for depicting an operational area, said operational area being defined within said real-world environment and comprising a plurality of functional regions each defining a different one or more selectable functions or operations that can be performed in respect of said operational area, the apparatus being configured to transfer image data from said display generating device into said user's view of said real-world environment at said viewing device to generate an augmented reality environment, the apparatus further comprising a control module (26) including a control device (28) configured to be selectively communicably coupled to all of said functional regions to enable a user to selectively perform the respective one or more functions or operations associated therewith, wherein the control device (28) is communicably coupled to a selected functional region only in response to a respective actuation signal, and an eye tracker module (24) configured to monitor said user's gaze relative to said augmented reality environment, in use, and, when said user's gaze is directed at a selected functional region displayed therein, generate and transmit to said control module (26) a said actuation signal and when said user's gaze is no longer directed at said selected functional region, cause said control device (28) to be decoupled therefrom.
https://worldwide.espacenet.com/publicationDetails/mosaics?CC=US&NR=2020218066A1&KC=A1&FT=D&ND=3&date=20200709&DB=&locale=en_EP

PERSONALIZED EYE OPENNESS ESTIMATION
Methods, systems, and devices for personalized (e.g., user specific) eye openness estimation are described. A network model (e.g., a convolutional neural network) may be trained using a set of synthetic eye openness image data (e.g., synthetic face images with known degrees or percentages of eye openness) and a set of real eye openness image data (e.g., facial images of real persons that are annotated as either open eyed or closed eyed). A device may estimate, using the network model, a multi-stage eye openness level (e.g., a percentage or degree to which an eye is open) of a user based on captured real time eye openness image data. The degree of eye openness estimated by the network model may then be compared to an eye size of the user (e.g., a user specific maximum eye size), and a user specific eye openness level may be estimated based on the comparison.
https://worldwide.espacenet.com/publicationDetails/biblio?II=54&ND=3&adjacent=true&locale=en_EP&FT=D&date=20200709&CC=US&NR=2020218878A1&KC=A1
http://www.freepatentsonline.com/20200218878.pdf

METHOD AND SYSTEM FOR FACILITATING SAFELY CONTROLLING VEHICLE OPERATION / Scania / 2020-04-23
The present invention relates to a method for facilitating safely controlling vehicle operation of a vehicle. Said vehicle operation comprises different drive operation modes. Said drive operation modes comprises a manual drive operation mode and an assisted drive operation mode and/or an autonomous drive operation mode. The method comprises the steps of: determining (S1) current drive operation mode; determining (S2) if certain predetermined requirements for said current drive operation mode are fulfilled; taking action (S3A) for preventing unsafe vehicle operation if said certain predetermined requirements for a current drive operation mode are not fulfilled; and allowing continued drive (S3B) in said current drive operation mode if said certain predetermined requirements for a current drive operation mode are fulfilled. The present invention also relates to a system for facilitating safely controlling vehicle operation. The present invention also relates to a vehicle.
Thus, no action is taken and the vehicle may continue driving in current drive mode if there is no indication that certain predetermined requirements for the current drive operation mode are not fulfilled. According to an embodiment of the method the step of taking action for preventing unsafe vehicle operation if said certain predetermined requirements for a current drive operation mode are determined not to be fulfilled comprises the step of warning the operator of the vehicle of said certain predetermined requirements for a current drive operation mode not being fulfilled. By thus warning the operator of the vehicle of said certain predetermined requirements for the current operation not being fulfilled safety may be further improved.

The step of warning the operator of the vehicle of said certain predetermined requirements for a current drive operation mode not being fulfilled is performed by means of warning means. Said warning means may comprise visual warning means comprising e.g. any suitable display unit in the vehicle and/or projection outside the vehicle, e.g. on the road in front of the vehicle. Said warning means may comprise sound means for audible warning such as a loud speaker on board the vehicle. Said warning means may comprise tactile warning means, such as vibrations in the seat, pedals, steering wheel or the like.

Said certain predetermined requirements to be fulfilled for a current drive operation mode may comprise state of driver. The step of determining state of driver may comprise any suitable sensor for detecting awareness of driver, whether driver is sleeping or the like.
https://worldwide.espacenet.com/publicationDetails/biblio?II=0&ND=3&adjacent=true&locale=en_EP&FT=D&date=20200423&CC=WO&NR=2020080981A1&KC=A1

Method and control arrangement for transition between modes of a vehicle / SCANIA CV AB/ 2020-02-03
The sensors 110 may comprise e.g. a camera, a stereo camera, an infrared camera, a video camera, a radar, a lidar, an ultrasound device, a Passive Infrared (PIR) sensor, a time-offlight camera, or similar device, in different embodiments. The vehicle 100 may comprise several sensors 110 directed in distinct directions around the vehicle 100. However, many of the sensors 110 may be directed in the driving direction. The sensors 110 may be of the same, or different type. An advantage with using different types of sensors is that different sensor types have different capabilities; i.e. a camera may function well in daylight while a laser sensor may be advantageous in darkness, etc.

The control arrangement 120 may determine to switch into autonomous driving mode e.g. when the driver is detected to fall asleep, to be non-focused on the environmental traffic situation, to be unconscious, to be affected by alcohol or drugs, etc., which may be detected by one or several sensors directed towards the driver such as e.g. a driver monitoring camera, video camera, a heart rate monitor, a blood pressure monitor, a respiration meter, etc., and / or a vehicle displacement detector.
https://worldwide.espacenet.com/publicationDetails/biblio?CC=SE&NR=1950530A1&KC=A1&FT=D&ND=3&date=20200203&DB=&locale=en_EP

Probably posted before.

Thales has secured the first customer for its touchscreen FlytX avionics suite, namely the French defense procurement agency (DGA) jointly with Airbus, which chose FlytX for the future H160M Joint Light Helicopter. Thales is also studying the possible application of FlytX in the future Tiger Mk.3.

The company launched the commercial version of FlytX this week at the Paris Air Show and the system is on track for certification in 2022, according to Guillaume Lapeyronnie, marketing director for the Thales Flight Avionics Business Line. Following the H160M installation, Thales will target military then air transport and business airplanes.
https://www.ainonline.com/aviation-news/air-transport/2019-06-18/airbus-picks-thales-flytx-suite-h160m

“FlytX [is] an avionics solution that is already under development for helicopters. We’re now preparing this for the business jet market," Boussiron told Avionics International. "In business jets, we’re already onboard several platforms in terms of electrical flight controls, but this would be our first full fledged avionics ****pit solution, to be in competition with Honeywell and Collins."
https://www.aviationtoday.com/2020/01/28/thales-showcasing-flytxs-new-way-flying-helicopters-heli-expo/
https://www.aviationtoday.com/2019/12/06/flytx-virtual-ai-assistant-development-future-business-jet-****pits/
https://www.aviationtoday.com/2019/11/26/thales-explains-connected-****pit-approach-behind-2020s-fms-pureflyt/

This patent SURGEON HEAD-MOUNTED DISPLAY APPARATUSES got my attention due to the number of Thales employees on the author list.
GORGES SEBASTIEN
JAMES YANNICK
LAGUARDA JOACHIM
BRUZY CHRISTOPHE
CALLOWAY THOMAS
POPOOLAPADE JOHN has a L 3 COMMUNICATIONS relationship.

Globus Medical, Inc. Overview.
Enabling Technologies
Enabling Technologies are advanced computer-assisted intelligent systems designed to enhance a surgeon’s capabilities and streamline surgical procedures to be safer, less invasive, more accurate, and more reproducible, to ultimately improve patient care and reduce radiation exposure for all involved. Our current enabling technologies are comprised of imaging, navigation and robotic (“INR”) assisted surgery solutions. This includes the
ExcelsiusGPS® platform, a robotic guidance and navigation system that supports minimally invasive and open procedures by improving visualization of
patient anatomy using patient images and by guiding instruments and implants to the specified trajectory using a robotic arm, to ultimately enhance the
surgeon’s decision making process. The ExcelsiusGPS® platform has a modular design that can be used for a variety of screw placement applications, and we
expect that it will serve as a foundation for future clinical applications using artificial intelligence and augmented reality. Surgimap® pre-planning software is used for effortless and convenient surgical planning at any time.

in2018, we acquired Nemaris Inc., a privately held company that markets and develops Surgimap®, a leading surgical planning software platform, to further bolster our efforts to advance surgical procedures through Enabling Technologies. In 2019, we acquired substantially all of the assets of StelKast, Inc., a privately held company that designs, manufactures and distributes orthopedic implants for knee and hip replacement surgeries. We intend to selectively pursue acquisitions and alliances that complement our strategic plan and provide innovative technologies, personnel with significant relevant experience, or increased market penetration. We are currently evaluating possible acquisitions and strategic
relationships and believe that our resources and experience make us an attractive acquirer or part
http://www.investors.globusmedical.com/static-files/7068e451-8960-4b81-aa3f-0bb8ca8f9ee3

SURGEON HEAD-MOUNTED DISPLAY APPARATUSES
An augmented reality surgical system includes a head mounted display (HMD) with a see-through display screen, a motion sensor, a camera, and computer equipment. The motion sensor outputs a head motion signal indicating measured movement of the HMD. The computer equipment computes the relative location and orientation of reference markers connected to the HMD and to the patient based on processing a video signal from the camera. The computer equipment generates a three dimensional anatomical model using patient data created by medical imaging equipment, and rotates and scales at least a portion of the three dimensional anatomical model based on the relative location and orientation of the reference markers, and further rotate at least a portion of the three dimensional anatomical model based on the head motion signal to track measured movement of the HMD. The rotated and scaled three dimensional anatomical model is displayed on the display screen.

[0053] The gesture sensor 602 may include any type of sensor that can sense a gesture made by a user. In a surgical environment, use of a gesture sensor 602 to receive a gesture-based command from a surgeon or other OR personnel can be advantageous because it avoids a need for the user to touch a non-sterile surface of the HMD 600 or other device. The gesture sensor 602 may include the camera 610 which outputs video (e.g., RGB-D video) displaying movement of a user's hand, fingers, arms or other objects moved by the user along a pathway that the user knows will define a command identifiable by an operational surgical program (OSP) 632 and/or another component of the system. The camera 610 or another camera may be directed toward one of the user's eyes to identify a dwell time of the eye, blink timing, and/or movement of the eye to generate a command from the user to control what is displayed on the display screen 608.

[0073] In the example of FIG. 10, the surgeon's head is tilted downward below the first pitch threshold so that the surgeon's line-of-sight 702 is toward the operation site while looking through the TTL loupe. The surgeon may shift eye position upward so that the surgeon's line-of-sight 700 now looks through the display screen 752. The computer equipment 620 responds to the surgeon's head tilted below the first pitch threshold by operating in the full-screen operational mode to select a defined one of the video streams which it displays on the display screen 752 to generate the primary virtual display panel 710. Accordingly, the primary virtual display panel 710 can be positioned by the HMD 750 to appear to the surgeon, while maintaining the line-of-sight 700, to hover in space above and adjacent to the location of the operation site. None of the three other video streams mapped to the three secondary virtual display panels 720, 722, and 724 are presently displayed on the display screen 752.
https://tinyurl.com/yd7p2r4z

Qualcomm's Snapdragon 865 Plus is here! Here's what's new.
https://www.youtube.com/watch?time_continue=63&v=wL2H7G8JoKM&feature=emb_logo

Pushing the envelope with the Snapdragon 865+ 5G Mobile Platform
https://www.youtube.com/watch?v=hgQd_8qVGzI&feature=youtu.be

Seeingtom,
I agree with the points made but one observation that I have made that makes me question the case that Toyota are developing a in house model is how often there technological steps in patent terms look to work in lock step with GM/Hyundai/Bosch/Conti/ etc.
Obviously I'm conformationally biased here and looking for patterns and it could all me coincidence but the IP progress looks to me to have a symmetry that may challenge the case for Toyota developing inhouse DMS.

SYSTEM AND METHOD FOR ADJUSTING REACTION TIME OF A DRIVER
The present invention provides a method and system for monitoring driver inattentiveness using plurality of physiological factors of the driver. In this method, captured images and/or short videos are used to determine the physiological factors of the driver. The first physiological factor from the plurality of physiological factors is used to determine the level of drowsiness and/or inattentiveness of the driver, which further supported by the second physiological factor. The data generated from the analysis of first and second physiological factors is further utilized to generate a predictive warning to the driver. The intensity level of the warning is varied based on the analyzed level of inattentiveness of the driver. The warning may be an audio warning, a visual warning, an audio-visual warning, haptic warning like vibration, etc.

[0025] Driver monitoring module 122 collects data about driver during an active autonomous drive mode, semiautonomous mode and manual mode. It collects data about driver like expressions, eye gaze, emotions, facial identity etc. Data about driver may be collected using various cameras facing into a cabin of the vehicle 100.

[0026] Drive monitoring module 124 collects data about drive of the vehicle 100. The drive may be autonomous drive or manual drive. Data collected may be like drive behavior in various situations, various conditions, confidence level, stress induced mistakes etc. Drive monitoring module 124 may help in ascertaining drive behavior during the drive that may be kept for records and utilized for improving future drive interactions, and mistakes while driving the vehicle 100. Furthermore, collected data, such as the deviation of behavioral trends, spoken words, gaze monitoring and / or environmental conditions within the vehicle are used to provide useful data regarding the state of the driver.
https://worldwide.espacenet.com/publicationDetails/description?CC=US&NR=2020211354A1&KC=A1&FT=D&ND=3&date=20200702&DB=&locale=en_EP

INATTENTIVENESS DETERMINATION DEVICE, INATTENTIVENESS DETERMINATION SYSTEM, INATTENTIVENESS DETERMINATION METHOD, AND STORAGE MEDIUM FOR STORING PROGRAM
This inattentiveness determination device is provided with: a photographic image acquisition means which acquires a photographic image of a driver driving a moving body; a first inattentiveness determination means which detects the gaze direction of the driver on the basis of the photographic image and which, on the basis of the gaze direction and a prescribed gaze range determined with the traveling direction of the moving body set as the reference, determines whether or not the driver is in a state of inattentiveness; and a second inattentiveness determination means which, in the case when no gaze direction can be detected, determines, on the basis of the driver's face orientation and a prescribed face-orientation range determined with the traveling direction of the moving body set as the reference, whether or not the driver is in a state of inattentiveness.
https://worldwide.espacenet.com/publicationDetails/biblio?II=0&ND=3&adjacent=true&locale=en_EP&FT=D&date=20200702&CC=WO&NR=2020138088A1&KC=A1

System and method for dialog session management
A robot or other conversational agent determines whether a human interlocutor has ceded the conversational floor by detecting the absence of a particular speech characteristic such as the introduction of filler speech at the end of an utterance, and the presence of a characteristic physical behaviour, such as the interlocutor returning their gaze to a focal point of the conversion, such as the robot's face, during a specified time window after the end of the utterance. Where this combination of characteristics is detected, the robot can determine that the conversational floor is open and that a response may be appropriate. This may trigger a conversational response for example via voice or text.
https://worldwide.espacenet.com/publicationDetails/biblio?II=15&ND=3&adjacent=true&locale=en_EP&FT=D&date=20200702&CC=AU&NR=2018367187A1&KC=A1

SYSTEM AND METHOD FOR A DYNAMIC HUMAN MACHINE INTERFACE FOR VIDEO CONFERENCING IN A VEHICLE
A system in a vehicle comprising one or more sensors configured to obtain cognitive-load data indicating a cognitive load of an occupant of the vehicle, a controller in communication with the one or more sensors, wherein the controller is configured to determine a cognitive load of the occupant utilizing at least the cognitive-load data in response to an initiation of a video conference session, and output on a vehicle display a first user interface in connection with the video conference session when the cognitive load exceeds a threshold and a second user interface in connection with the video conference session when the cognitive load is below a threshold, wherein the second user interface includes video of the video conference session and the first user interface excludes video of the video conference session.
https://worldwide.espacenet.com/publicationDetails/biblio?II=14&ND=3&adjacent=true&locale=en_EP&FT=D&date=20200702&CC=US&NR=2020213560A1&KC=A1
Drawings
http://www.freepatentsonline.com/20200213560.pdf

Aircraft and method of controlling .
A method 140 of controlling an aircraft 10 comprises tracking 141, using at least one sensor, one of an input by a pilot or a biometric parameter of a pilot during flight of the aircraft. The method includes determining 142 a lack of attention by the pilot or a depth of sleep of the pilot based on the sensor data. A predetermined amount of lack of attention, or a controlled rest, is allowed 143 before a system of the aircraft is controlled 144, or access to a system is modified (154, Fig. 5). The method may include determining a portion of the flight plan during which a period of controlled rest is allowable e.g. when cruising at altitude. An indication may be provided in the ****pit that controlled rest is allowable, but only if a second pilot is present and not sleeping. A timer may monitor a period of controlled rest, and trigger a wake-up procedure e.g. audible warning, vibrating seat. If unacceptable absence or incapacitation of the pilot is determined then a recovery procedure begins e.g. unlock ****pit door, automatic landing, distress tracker.

[0005] Aspects of the present disclosure are directed to a method of controlling an aircraft. During operation of an aircraft, portions of a flight plan such as takeoff or landing can require significant operator attention or alertness for operation. Other portions of a flight plan, such as cruise during long flights, may allow for minimal pilot attention or alertness for operation of the aircraft. It can be beneficial to allow a pilot to rest during appropriate portions of flight in the event of pilot fatigue, as well as to detect fatigue or incapacitation of a pilot during flight.

[0006] Pilot incapacitation can occur due to events such as decompression or due to illness or fatigue of the pilot. In an example where a pilot is fatigued, pilot incapacitation can take the form of an uncontrolled deep sleep or the form of "microsleep" wherein a pilot appearing to be awake is actually experiencing brief (approximately 1-10 seconds) periods of sleep or loss of attention. It can be beneficial to track an attention level of a pilot or to detect that a pilot is incapacitated and act accordingly.

[0007] During long flights, or during particular portions of a given flight, pilots may be allowed a controlled rest (also referred to as "controlled sleep" or "in-seat rest") to reduce operator fatigue. Typically, a co-pilot will take over control of the aircraft while the pilot briefly sleeps. A member of the cabin crew can also periodically call the co-pilot during this time to ensure alertness of the co-pilot.
https://worldwide.espacenet.com/publicationDetails/description?CC=GB&NR=2579655A&KC=A&FT=D&ND=3&date=20200701&DB=&locale=en_EP

Drawings
http://www.freepatentsonline.com/20200210733.pdf

ADAPTIVE WARNINGS AND EMERGENCY BRAKING FOR DISTRACTED DRIVERS
System and methods for a vehicle for determining driver distraction and adapting the operation of driving assistance systems. One example system includes a subsystem for sensing driver interaction with electrical devices located within a passenger compartment of the vehicle, a first sensor configured to sense conditions outside of the vehicle, a second sensor configured to sense conditions outside of the vehicle, and an electronic controller. The electronic controller is configured to determine a state of driver attentiveness based on driver interactions, provide the driver attentiveness state to a driver assistance system, determine a threshold of the driver assistance system based on the state of driver attentiveness by adjusting at least one of a sensor fusion confidence threshold, an object classification confidence threshold, a lane association threshold, blind spot detection threshold, or lane detection warning threshold; and control the vehicle via the driver assistance system operated in accordance with the threshold.
https://worldwide.espacenet.com/publicationDetails/biblio?II=48&ND=3&adjacent=true&locale=en_EP&FT=D&date=20200702&CC=US&NR=2020207363A1&KC=A1
Drawings
http://www.freepatentsonline.com/20200207363.pdf

SEMANTIC SEGMENTATION USING DRIVER ATTENTION INFORMATION
Methods of creating trained semantic segmentation network models and operating vehicles using the model. One example method includes an outside view camera configured to capture images that represent an artificial representation of the driver's view, a driver-facing camera configured to capture a driver's eye movements, and an electronic controller. The electronic controller is configured to receive images from the cameras; calibrate the image of the driver's eye movement with the artificial driver view; create a pixel weighted heat map of the calibrated images; create a trained semantic segmentation neural network model and a trained attention neural network model using the pixel weighted heat map and the artificial driver view; and operate the vehicle using the trained semantic segmentation neural network model and the trained attention neural network model.
https://worldwide.espacenet.com/publicationDetails/biblio?II=14&ND=3&adjacent=true&locale=en_EP&FT=D&date=20200702&CC=US&NR=2020210765A1&KC=A1
Drawings
http://www.freepatentsonline.com/20200210765.pdf

SEMANTIC SEGMENTATION USING DRIVER ATTENTION INFORMATION
Methods of creating trained semantic segmentation network models and operating vehicles using the model. One example method includes an outside view camera configured to capture images that represent an artificial representation of the driver's view, a driver-facing camera configured to capture a driver's eye movements, and an electronic controller. The electronic controller is configured to receive images from the cameras; calibrate the image of the driver's eye movement with the artificial driver view; create a pixel weighted heat map of the calibrated images; create a trained semantic segmentation neural network model and a trained attention neural network model using the pixel weighted heat map and the artificial driver view; and operate the vehicle using the trained semantic segmentation neural network model and the trained attention neural network model.
https://worldwide.espacenet.com/publicationDetails/biblio?
II=14&ND=3&adjacent=true&locale=en_EP&FT=D&date=20200702&CC=US&NR=2020210765A1&KC=A1
Drawings
http://www.freepatentsonline.com/20200210765.pdf