Embedded Files
What does it take to play a board game with a humanoid robot? We have developed HARMONIOUS, a real-time reactive motion control system for a humanoid which uses two arms to play the game while perceiving and avoiding contact with humans through touch, proximity, and vision.
Citation: Rozlivek, J.; Roncone, A.; Pattacini, U. & Hoffmann, M. (2025), 'HARMONIOUS – Human-like reactive motion control and multimodal perception for humanoid robots', IEEE Transactions on Robotics 41, 378 - 393.
Full text: [DOI - IEEE Xplore][pdf-arxiv]
Videos:
Main video - playing the card game: https://youtu.be/gw8JB-1R3bs
Additional supplementary videos: https://ieeexplore.ieee.org/document/10758235/media#media
Press release (in Czech): https://fel.cvut.cz/cs/aktualne/novinky/36488-robotici-z-fel-cvut-vyvinuli-algoritmus-diky-nemuz-se-humanoidni-roboti-budou-moci-bezpecne-pohybovat-mezi-lidmi
In the news, Dec 11, 2024: Reports on "HARMONIOUS" featured in:
Události, Česká televize: Čeští vědci se snaží zdokonalit roboty. [mp4]
Český rozhlas plus, Věda plus (at https://www.mujrozhlas.cz/veda-plus/jak-podle-odborniku-udrzet-vanocni-stromek-cerstvy-co-nejdele-3020734?t=1198)
wired.cz (https://www.wired.cz/clanky/na-prazskem-cvut-maji-humanoida-s-empatii-a-hmatem-ktery-by-o-nas-mohl-pecovat).
Contributions:
We have developed HARMONIOUS, a real-time reactive motion control system for upper body control of a humanoid robot sensing and avoiding contact with humans in its close proximity. Our specific contributions are the following.
HARMONIOUS is human-like in the following two important aspects: 1) it employs multimodal sensing around the whole body of the robot, “visuo-tactile awareness” resembling PPS representations; 2) it produces minimum jerk movement profiles, which are characteristic of human motion. Together, these provide a basis for safe and natural human–robot interaction (HRI)—physical and social.
We developed a unified representation of the space around the robot that feeds a reactive motion controller. Dynamically moving obstacles ranging from physical contact with the robot (zero distance, perceived through touch) to obstacles in close or far proximity can be remapped onto the robot's body parts, weighted, and transformed into kinematic constraints. Thus, there is a dense and multimodal representation of a PPS analogue, which we refer to as the “perirobot space”. It is demonstrated here by combining touch, proximity, and visual sensors, but other contact-related or range-based sensors could be used.
We have extensively and experimentally evaluated HARMONIOUS and demonstrated its superior performance to state-of-the-art robot controllers where possible . Moreover, to authors' knowledge, this is the first work to show real-time control of a humanoid robot upper torso (in total 17 DoFs, with the possibility of different tasks for the two arms) faced with tens of dynamically moving obstacles around the whole robot body surface. Furthermore, kinematic singularities are handled through velocity damping and preferred postures are rewarded in the optimization problem formulation (see Table I for an overview). As the final evaluation, we tested HARMONIOUS in an interactive board game scenario with the human player dynamically perceived, simultaneously processing keypoints on his body, proximity signals, and physical contacts on the whole robot body.
The problem formulation is highly modular and both the minimization criteria (e.g., motivating preferred postures) or constraints to the quadratic program (e.g., obstacles) can be easily removed or added on the run.
Social media posts:
Page updated
Google Sites
Report abuse