The cognitive engine (CE) provides a teleoperation assistance in the face of communication latencies. Before a mission, in the onshore control center, the cognitive engine records demonstrations of tasks (turning a valve, pushing a lever, etc.) executed in the virtual reality environment. These demonstrations are encoded into statistical models, which are subsequently transferred to the ROV (offshore part of the CE). These models are used to locally adapt to changing situations in an online manner, drastically reducing effects of the communication latencies in teleoperation and the cognitive load of the teleoperator.
Contact: Dr Sylvain Calinon at IDIAP, http://calinon.ch
Technical description and applications
The cognitive engine (CE) provides the robot with a semi-autonomous capability when the teleoperation signals are interrupted/delayed. The CE relies on an approach combining task-parameterized hidden semi-Markov models (TP-HSMM)with linear quadratic tracking (LQT), which has been developed by the Robot Learning & Interaction Group at the Idiap Research Institute.
The CE comes with a graphical user interface, where the demonstrations and model information can be displayed. It also provides an interface to update the models by adding new demonstrations. Over several missions, a library of tasks can be demonstrated and encoded, which can be reused in further missions. The source code of the CE is written in C++ with very limited dependencies so that the software remains as portable as possible.
Any application requiring teleoperation of a robot that cannot be achieved in a direct control mode. It includes teleoperation and human-robot collaboration applications that need to cope with delays, limited bandwidth or applications with a high number of degrees of freedom to be controlled simultaneously. The CE allows a single teleoperator to control several arms/robots by providing high-level commands, while the robot can automatically cope with perturbations and local adjustments in an autonomous manner.
The CE can be used either individually or together with the other DexROV components, including teleoperation devices (VR, exoskeleton), robot controllers (force feedback) and perception (detection and localization of landmarks, objects and tools).