Honda 2020-01-09/ Robotic gaze Sensors17 Jan 2020 08:32
ASSISTANCE SYSTEM, METHOD, AND PROGRAM FOR ASSISTING A USER IN FULFILLING A TASK
[0016] The assistance system enables a successful cooperation between the user and the robot by an efficient and intuitive communication. Improved communication minimizes human effort and maximizes task progress. The assistance system benefits from vastly available unstructured and semi-structured task descriptions by harvesting them and making them available to the cooperation of the user and the robot in performing the task. Moreover, the assistance system is able to adapt quickly to an individual user and task idiosyncrasies.
[0020] The invention is particularly suited when applying belief tracking of the task progress, communication state, and human state, e.g. using methods known from dialog state tracking as Partially Observable Markov Decision Processes (POMDP), possibly also implemented as a hierarchical belief tracking ( Williams, J., Raux, A., Ramachandran, D., & Black, A. (2013). The dialog state tracking challenge. In: Proceedings of the SIGDIAL 2013 Conference (pp. 404-413 ); Williams, J. & Raux, A. & Henderson, M. (2016). The dialog state tracking challenge series: A review. Dialogue & Discourse, 7(3), 4-33 ). The personalization of the communication between the user and the assistance system or the robot is achieved by learning user characteristics in the interaction between the assistance system and the user.
[0024] In a preferred embodiment, the processor is configured to monitor the task progress by interpreting the user input, the user input including at least one of an oral (acoustic) user input, at least one visually perceivable gesture of the user, a tactile input of the user, and by interpreting the image data acquired from a visual sensor, for example, at least one video camera. The visual sensor captures the image data which depicts the user and preferably the at least one object currently involved in the task.
[0025] A close cooperation between the user and the assistance system is achieved. The assistance system mirrors traditional cooperation between human workers acting together, for example, using speech as a typical communication channel for negotiating a shared workflow when performing a task. The cooperation with the assistance system is therefore intuitively understandable for the user. This is in particular the case because the entire task is divided into individual steps and/or subtasks. The subtasks may each consist of a plurality of steps. The representation of the entire task defines the relation of the individual subtask and/or steps and therefore is not limited to a step-by-step sequence having a strict timely order.
https://worldwide.espacenet.com/publicationDetails/biblio?II=36&ND=3&adjacent=true&locale=en_EP&FT=D&date=20200109&CC=US&NR=2020012670A1&KC=A1#
Similarity here between
Toyota 2020-01-16
METHODS AND SYSTEMS FOR DETERMINING TELEOPERATING USER INTENT VIA EYE TRACKING
https://tinyurl.com/wghxa