London South East | The UK's favourite investor portal.

LINE-OF-SIGHT GUIDANCE DEVICE, ALERT SYSTEM, ALERT METHOD, AND PROGRAM
The purpose of the present invention is to make it possible to reduce instances in which a driver fails to notice an object that should be viewed. A line-of-sight guidance device 10 guides the line-of-sight of a driver of a vehicle 1 to an object present to the front of the vehicle 1, the line-of-sight guidance device comprising: an object recognition unit 31 that recognizes an object; and a display control unit 35 that displays, in a display device 11, a line-of-sight guidance virtual image GVI that guides the line-of-sight of the driver to a position corresponding to the object recognized by the object recognition unit 31 in the windshield of the vehicle 1. The display control unit 35 sets the length, in the vehicle width direction, of the line-of-sight guidance virtual image displayed in the display device 11 to a length corresponding to the state of the object present to the front of the vehicle 1.
https://worldwide.espacenet.com/publicationDetails/biblio?CC=WO&NR=2022190630A1&KC=A1&FT=D&ND=3&date=20220915&DB=&locale=en_EP
https://www.freepatentsonline.com/WO2022190630A1.pdf

Accordingly, methods, systems, and vehicles are provided for assessment of an alertness of a driver of the vehicle. In various embodiments, the alertness of the driver is determined based on how closely the driver's eyes are focused on environmental characteristics outside the vehicle during the vehicle drive. In certain embodiments, the alertness of the driver is determined based on a comparison of a predicted gaze angle versus a measured gaze angle of the driver's eyes based on a trajectory curvature of the roadway ahead of the vehicle, and in certain embodiments also based on road signs, traffic lights, and/or infrastructure and/or objects along the roadway in front of the vehicle.

In various embodiments, the disclosed techniques may provide an indication of driver alertness (or lack thereof) more quickly as compared with existing techniques, for example in that the disclosed techniques compare the driver's gaze angle with what is predicted based on the specific environment in front of the vehicle. For example, in certain embodiments, even though a driver may be looking outside the vehicle (or even at the roadway in general), it is possible that the driver's eyes may not be zeroed in on the upcoming trajectory curvature of upcoming segments of the roadway (which, for example, may be observed more quickly with the disclosed techniques as compared with other techniques that do not focus on the trajectory curvature or other features of the external environment outside of and in front of the vehicle).

Also in various embodiments, the disclosed techniques are also utilized for taking one or more vehicle control actions. In certain embodiments, notifications are provided for the driver and/or one or more other interested parties. Also in various embodiments, movement of the vehicle may be automatically controlled based on an assessment of the driver alertness, such as automatic braking, automatic steering, automatic control of the drive system (e.g., for deceleration), adjusting threshold parameters for lane keeping functions and/or one or more other automatic functions, and so on.

PREDICTIVE DRIVER ALERTNESS ASSESSMENT
In an exemplary embodiment, a system is provided that includes one or more first sensors, one or more second sensors, and a processor disposed onboard a vehicle. The first sensors are configured to at least facilitate obtaining first sensor data with regard to an external environment outside the vehicle. The second sensors are configured to at least facilitate obtaining second sensor data with regard to one or more eyes of a driver of the vehicle. The processor is configured to at least facilitate: determining a predicted gaze angle of the one or more eyes of the driver based on the external environment outside the vehicle, using the first sensor data; determining a measured gaze angle of the one or more eyes of the driver, using the second sensor data, and controlling one or more vehicle actions based on a comparison of the predicted and measured gaze angles.

Also in an exemplary embodiment, the one or more first sensors include one or more exterior cameras that face outside the vehicle and that are configured to obtain external camera images of the external environment outside the vehicle; and the one or more second sensors include one or more interior cameras that face inside the vehicle and that are configured to obtain internal camera images of the driver inside the vehicle.

Also in an exemplary embodiment, the processor is further configured to at least facilitate: determining a trajectory curvature of a roadway on which the vehicle is travelling, based on the first sensor data; and determining the predicted gaze angle of the one or more eyes of the driver based on the trajectory curvature of the roadway.

Also in an exemplary embodiment, the processor is further configured to at least facilitate: determining one or more infrastructure objects of a roadway on which the vehicle is travelling, based on the first sensor data; and determining the predicted gaze angle of the one or more eyes of the driver based on the one or more infrastructure objects.
https://worldwide.espacenet.com/publicationDetails/biblio?II=0&ND=3&adjacent=true&locale=en_EP&FT=D&date=20220915&CC=US&NR=2022289151A1&KC=A1
https://www.freepatentsonline.com/20220289151.pdf
https://patents.justia.com/patent/20220289151

[0043] An operator 102 moves in an environment during simulation training or a real mission, illustrated by a platform 104. The platform used is a complex system11 operated by the operator, such as for example an aircraft (or a vehicle or a control station, etc.) from which it is possible to retrieve contextual information about the mission in progress, about the state of the complex system and about the situation of the environment (in the aeronautical context: weather conditions, state of air traffic, etc.). Without limitation, the state of the system in the context of avionics may be information relating to states inherent to normal operation, that is to say the percentage of power of the engines, whether the fins are folded or unfolded, the status of fuel tanks, etc., and information relating to states inherent to abnormal operation, that is to say the left engine is faulty, the right pedal of the co-pilot seat is faulty, or any sort of potential fault. The platform may also provide information about the actions performed by the operator via its HMIs.

[0070]In the case of the results being used by an application 122 intended for an operator, various services are likely to be offered, such as for example:
- information and indicators about the instantaneous physiological state;
- information about the (intra-individual) comparison of the instantaneous physiological state with the history under the same conditions;
- alerts about intrinsically abnormal physiological states (incapacity, loss of alertness, signs of drowsiness, etc.) or alerts about contextual situations requiring established physiological states that are not complied with (alertness, wakefulness, etc.);
- recommendations for actions on the various aspects associated with the physiological states (for example if there are signs of drowsiness, recommendation to rest);
- recommendations for actions on various aspects associated both with the physiological states and with the operational situations (for example,18 recommendation for training or practice on certain aspects or on certain operational phases);
- indications of actions to be performed if a procedure that should be initiated is not, or if the physiological state of the operator is divergent, or if the operator encounters a problem that he is not used to solving (for example, reminder of the landing procedure in civil aviation, or advice on the actions to be performed in the event of a technical failure of one of the components of the aircraft);
- indications about the theoretical procedures that apply to a given situation, depending on the state of the complex system as measured by the sensors. In one embodiment, these indications may be enriched by virtue of the collective capitalization module that has (anonymously) recorded the reactions of various operators faced with similar situations.

Method and device to assist with monitoring the cognitive states of an individual
The invention relates to a computer-implemented method and device to assist with monitoring the states of an individual operating a complex system during a mission. The device is configured to regularly collect, during the mission, data relating to the state of the complex system and its environment, data relating to the mission context, physiological data relating to the operator, and data relating to the tasks carried out by the operator, in particular manipulation, observation and communication tasks; to determine, from the data collected, contextualisation parameters and an instantaneous individual state of the operator; to perform an intra-personal compliance analysis with respect to the operator's own previous individual reference states, based on the contextualisation parameters and the instantaneous individual state of the operator; to perform an interpersonal compliance analysis with respect to previous collective reference states, based on the results of the intra-personal compliance analysis; and to report the results of the analyses. The device can also be configured to perform a theoretical conformity analysis of the tasks performed with respect to the theoretical rules of the tasks to be performed or procedures to be carried out.

[0045] The operator's environment (and/or the operator himself) is equipped with various sensors 106 that make it possible to record physiological data and task or manipulation data from the operator during training or the mission. The physiological and manipulation data that are acquired are for example, without limitation: i) heart rate, by way of a heart rate monitor bracelet or a chest strap; ii) breathing rate, using the same chest strap or using a sensor pad located in or on the seat; iii) eye-tracking12 data collected by way of a camera located either at a distance from the pilot on the instrument panel or on glasses worn by the pilot; iv) skin resistance, by way of a heart rate monitor watch or a dedicated bracelet, v) motion sensors worn by the operator on his head, torso, upper and lower limbs or placed on his manipulation means: the speed and the acceleration of the rudders, the control stick and the levers, vi) all of the observation actions performed by the operator and the areas observed in the ****pit, vii) all of the manipulation actions performed by the operator on the instrument panel and more generally in the ****pit, viii) all of the oral communications made by the operator.
https://worldwide.espacenet.com/publicationDetails/biblio?CC=AU&NR=2020407832A1&KC=A1&FT=D&ND=3&date=20220728&DB=&locale=en_EP
https://www.freepatentsonline.com/WO2021122900A1.pdf

System for determining a flight rest scenario for an aircraft crew
A system is (10) for determining a flight rest scenario for an aircraft crew (12) comprising at least two pilots able to fly the aircraft (12) during a flight. The system (10) comprises a display device (16) comprising a screen (20), a module (32) for acquiring flight context and constraints inputs, and a module (34) for determining at least one crew rest scenario for the flight from the acquired inputs. The determination module (34) is able to determine the rest scenario from at least one simulation by a biomathematical fatigue model having at least one of the acquired inputs as variables. The system also includes a display management module (36) configured to display the determined rest scenario on the screen (20) of the display device (16)

1. A system for determining an in-flight rest scenario for a crew of an aircraft, the crew comprising at least two pilots able to fly the aircraft during a flight of the aircraft, the system comprising: a display device comprising a screen; an acquisition module configured for acquiring flight context and constraints inputs; a determination module configured for determining at least one in-flight rest scenario for the crew from the acquired flight context and constraints inputs, the determination module being configured to determine the in-flight rest scenario from at least one simulation by a biomathematical fatigue model having at least one of the acquired flight context and constraints inputs as variables; and a display management module configured to display the determined in-flight rest scenario on the screen of the display device.

2. The system according to claim 1, wherein the in-flight rest scenario is determined for a cruise phase of flight and comprises a timeline of at least one rest period, each rest period being assigned to one of the pilots, and the flight context and constraints inputs comprise at least one piloting constraint on piloting by the crew of the aircraft, the determination module being configured to determine at least one timeline of at least one rest period compatible with each piloting constraint, the determination module being configured to simulate and determine at least one fatigue level of each pilot, for each compatible timeline, by the biomathematical model, the determination module being configured to classify each compatible timeline according to an order relationship established from the simulated fatigue levels, the determined in-flight rest scenario comprising the compatible timeline that has a best classification according to the order relationship.
https://uspto.report/patent/app/20220245537#:~:text=Crew%20Fatigue
https://www.freepatentsonline.com/20220245537.pdf

SYSTEM FOR MONITORING THE OPERATIONAL STATUS OF AN AIRCRAFT CREW, AND ASSOCIATED METHOD
A system for monitoring the operational status of an aircraft crew comprises a unit for determining at least one pilot state based on first monitoring data of a high design assurance level and second monitoring data of a lower design assurance level. The determination unit comprises a first evaluation module capable of obtaining a high-level pilot state from the first data, regardless of the second data; a second evaluation module capable of determining a low-level pilot state, from at least the second data; and a consolidation module for determining a consolidated pilot state. The consolidated pilot state is obtained as a function of the high-level pilot state and the low-level pilot state.

[0078] The camera system 30 comprises at least one camera 36, and an analysis device 38 of the images produced by the camera 36, for determining presence and movement data of a crew member present in the aircraft control interface, as well as a direction of vision of the crew member and advantageously other physiological and cognitive parameters associated with the crew member (such as drowsiness, distraction, etc.)

[0100] The state of inattention is measured using the camera system 30, by measuring the position of the user's gaze, in particular if this position is away from the positions the user should be looking at in relation to the interfaces or ****pit, for example because they are consulting a mobile phone or tablet. Furthermore, this is corroborated with a lack of pilot activity on the controls, and possibly a lack of pilot response to a prompt.

[0104] It comprises at least a first state machine 80, which is able to implement a deterministic algorithm based on the first high design assurance level monitoring data. The first state machine 80 is intended to determine the high-level pilot state, between a normal state and a degraded state as defined above.

[0105] Preferably, the first high-level evaluation module 70 presents a state machine 80 for each pilot state to be determined, between the normal pilot state and the degraded pilot state. It therefore comprises at least one state machine 80 for the absence state and at least one state machine 80 for the incapacity and/or sleep state.
https://worldwide.espacenet.com/publicationDetails/biblio?II=3&ND=3&adjacent=true&locale=en_EP&FT=D&date=20220825&CC=US&NR=2022265187A1&KC=A1
https://www.freepatentsonline.com/20220265187.pdf

https://www.dassault-aviation.com/en/

USER-VEHICLE INTERFACE
There is provided a user-vehicle interface, comprising: a gesture control system, arranged to sense a gesture of a user of the vehicle; the gesture control system being arranged to process the sensed gesture to control interaction with content on a display, in accordance with that sensed gesture; and the user-vehicle interface further comprises a gesture control support, arranged to physically support a body part of the user associated with a provision of the gesture.

In addition, user-vehicle interfaces are now being used in a wider variety of vehicles, ranging from cars, to boats, to aeroplanes, to helicopters, and throughout the entire range of vehicles. It is often desirable to ensure that the user-vehicle interface is tailored to the particular vehicle, or use conditions, not only in terms of look and feel, but in terms of ease of use, or particular use, of a user of that vehicle. These latter points are often ignored or overlooked in proposed implementations.

[0145] The selection system 120 could take one of a number of different forms, ranging from eye-tracking (including gaze detection), voice control, a physical actuator, or even a (e.g. crude) form of gesture control, such as head tracking or similar. Advantageously, the selection system comprises an eye-tracking component for receiving eye-movement based input from the user 12. This allows the user to quickly, readily and intuitively look around the vehicle and associated displays and focus on and select a particular region for interaction. The selection system 120 might comprise a confirmatory input device for confirming a region of the display as the selected region based on the eye-tracking. This could be a dedicated input device or could be part of the eye-tracking components, or part of the engagement input device 18, or physical actuator of previous embodiments, audio input, and so on. There could even be a timing element built into the system, such that if particular region or content is looked at for a particular time (e.g. gazed at) then that region is determined as being the selected region.
https://worldwide.espacenet.com/publicationDetails/biblio?II=0&ND=3&adjacent=true&locale=en_EP&FT=D&date=20220908&CC=US&NR=2022283645A1&KC=A1
https://www.freepatentsonline.com/20220283645.pdf

METHOD OF ASSESSING A PILOT EMOTIONAL STATE
A method 140 of assessing an operator emotional state 131 and sending an alert 144 based on the emotional state 131. The method 140 includes tracking 141 during a time period, using at least one sensor 103, 105, 106, 112, 117, one of an image sensor data, voice data or a biometric parameter of an operator. Determining 142, using a controller 120 that is operatively connected to at least one sensor 103, 105, 106, 112, 117, a probability of a likely emotional state 131 from a list of emotional states 131 of an operator based on one of the image sensor data, voice data or the biometric parameter. Comparing 143, using a processor, the probability of one of the likely emotional states 131 of the operator with a baseline emotional state 131 of the operator. Sending 144, using the controller 120, an alert if most likely emotional state deviates from the baseline emotional state by a predetermined threshold.

[0029] The tracking system 100 can include at least one imaging module 102 and at least one audio module 104. The imaging module 102 can include an image sensor 103 configured to sense visual information about a pilot or co-pilot, such as rapid eye movement, eyes moving one direction or another, being open or closed, a direction of gaze, or a facial state such as eyebrows raised or lowered, by way of non-limiting examples and provide an output signal based thereon. In addition, the image sensor 103 can be configured to capture or sense images beyond the human visible spectrum such as in the infrared, ultraviolet, electromagnetic or other range. An eye movement or body movement parameter can be stored by the tracking system 100 The imaging module 102 or the flight control computer 22 can also be in signal communication with any of the flight displays 52, such as to display a visual indication based on the sensed visual information from the imaging module 102.

[0042] In one example, the facial recognition software 53 can be programmed to calculate a probability 133 of likely emotional states 132 such as very angry, angry, slightly angry, joyful, depressed, stressed, etc. from an image 130 captured by the image sensor 103. The facial recognition software 53 can analyze the data points 134, or data areas 139, for facial characteristics such as pilot mouth, right eye, or left eye being opened or closed. The data can be stored and analyzed to determine a pilot emotional state 131. The facial recognition software 53 can be stored on the ground station 31, in the flight management computer 22, or in a separate processing unit.
https://worldwide.espacenet.com/publicationDetails/biblio?CC=US&NR=2022284737A1&KC=A1&FT=D&ND=3&date=20220908&DB=&locale=en_EP
https://www.freepatentsonline.com/20220284737.pdf

AUGMENTED REALITY FOR VEHICLE OPERATIONS
Systems, methods, and computer products according to the principles of the present inventions may involve a training system for a pilot of an aircraft. The training system may include an aircraft sensor system affixed to the aircraft adapted to provide a location of the aircraft, including an altitude of the aircraft, speed of the aircraft, and directional attitude of the aircraft. It may further include a helmet position sensor system adapted to determine a location of a helmet within a ****pit of the aircraft and a viewing direction of a pilot wearing the helmet. The helmet may include a see-through computer display through which the pilot sees an environment outside of the aircraft with computer content overlaying the environment to create an augmented reality view of the environment for the pilot. A computer content presentation system may be adapted to present computer content to the see-through computer display at a virtual marker, generated by the computer content presentation system, representing a geospatial position of a training asset moving within a visual range of the pilot, such that the pilot sees the computer content from a perspective consistent with the aircraft's position, altitude, attitude, and the pilot's helmet position when the pilot's viewing direction is aligned with the virtual marker.

[0052] The system and method may further include a head position tracking system adapted to identify a viewing direction of a user of an augmented reality see-through computer display, wherein the presentation of the geospatially located content is further based on the viewing direction of the user. The presentation of the geospatially located content may also involve positioning the content within a field-of-view of the see-through computer display based on the viewing direction of the user. The system and method may further comprise an eye direction detection system (e.g. a camera system or other sensor system for imaging and tracking the position and movement of the user's eyes, wherein the presentation of the geospatially located content within the field-of-view is further based on a measured eye position, direction, or motion of the user.
https://worldwide.espacenet.com/publicationDetails/biblio?CC=US&NR=2022108623A1&KC=A1&FT=D&ND=3&date=20220407&DB=&locale=en_EP
https://www.freepatentsonline.com/20220108623.pdf

https://red6ar.com/

Latest Share Chat

Member Info for johnchucka

Followers

Following

Posts (Last 30 Days)

Posts (All Time)

Last Post

Member Since

Share Chat Post Distribution (Last 30 Days)

Activity Log (Last 30 Days)

Latest Share Chat

Member Info for johnchucka

Followers

Following

Posts (Last 30 Days)

Posts (All Time)

Last Post

Member Since

Share Chat Post Distribution (Last 30 Days)

Activity Log (Last 30 Days)

Please enter at least 2 characters

No results found

Search results

Recently viewed

Trending