Abstract: Human hand manipulation represents one of the most sophisticated forms of embodied intelligence, combining high-dimensional control with adaptable, task-dependent coordination. A key challenge lies not only in control itself, but in translating human sensorimotor principles into effective mechatronic design. Kinematic synergies offer a powerful framework for understanding how the central nervous system reduces the complexity of hand control. While grasping has traditionally been characterized by lowdimensional, highly coupled patterns, manipulation suggests a different paradigm in which coordination coexists with a greater degree of finger independence. This balance enables continuous object interaction, fine adjustments, and task-specific adaptability. These insights motivate a rethinking of prosthetic and robotic hand design. Rather than relying on rigid, predefined grasp taxonomies, next-generation systems should adopt flexible architectures capable of redistributing control across digits. From a mechatronic perspective, this implies combining underactuation and synergy-based control with selective decoupling of key joints, thereby achieving both efficiency and dexterity. A similar approach extends to sensing, where embedded multimodal strategies can support the development of biomimetic feedback. Within this framework, the contribution of less dominant fingers becomes particularly significant, emphasizing the need for distributed actuation and sensing. These principles inform the development of advanced systems such as the DexterHand, a device under development by Istituto Italiano di Tecnologia and INAIL, in which mechanical architecture and control strategies are co-designed to support both coordinated and independent finger behaviors. Furthermore, fully integrated multimodal sensing provides rich information to both the user and the system, enabling shared autonomy and enhancing dexterous performance. Bridging insights from human motor control with mechatronic design and embodied sensing opens new directions for robotic prosthetic hands, moving beyond grasping toward more natural, adaptive, and functionally rich manipulation in real-world scenarios.
Abstract: Robotic dexterous manipulation still trails human capability despite rapid progress in learning-based and model-based control. One under-explored avenue is to bypass replicating human dexterity from scratch and instead capture it directly from a human operator via biosignals, then transfer that motor intent to a robot. Surface electromyography (sEMG) is uniquely suited to this role: it samples the neural drive to skeletal muscle upstream of mechanical actuation, is robust to visual occlusion, and can be acquired non-invasively from a wristband at multi-kHz rates. However, deploying sEMG-driven robot interfaces faces a fundamental biological barrier — inter-user variability in muscle anatomy, skin impedance, and electrode placement causes the same gesture to produce very different signals across people, and population models degrade substantially on unseen users. We introduce REACT, a lightweight user-adaptive conditioning framework for sEMG-to-pose decoding. REACT distills a handful of short calibration recordings into a per-user embedding via a four-stage pipeline (characteristic CNN, bidirectional GRU, transformer set encoder, FiLM projection) and uses Feature-wise Linear Modulation to personalize a frozen pretrained backbone at inference time, requiring no gradient updates. On the large-scale emg2pose benchmark, REACT improves over the state-of-the-art baseline across all three generalization splits and both regression and tracking modes, reducing angular error by up to 3.9% with under 45s of per-user calibration. We position REACT as an early stepping stone toward sEMG-driven dexterous robot teleoperation, and outline open challenges such as pose-to-robot retargeting, integration with compliant actuation, closed-loop haptic feedback, and self-supervised calibration that connect EMG decoding to the workshop’s broader question of what robots are missing.
Abstract: The human visual system faces a fundamental computational challenge: retinal projections of objects change with viewing distance, yet our perceptual experience remains remarkably stable. This stability depends on size constancy — the ability to perceive an object’s size as constant despite variations in retinal input. While proprioception has been shown to partially restore size constancy during grasping in the absence of visual depth cues, the underlying neural mechanisms remain poorly characterized. To address this gap, we employed an ecological paradigm in which real 3D spheres (2.5 and 5 cm diameter) were positioned at 20 and 40 cm, respectively, to subtend the same visual angle and share identical retinal size. Participants performed manual estimation or grasping tasks under full- and restricted-viewing conditions (i.e., with or without visual depth cues), while proprioceptive distance information was provided by holding the stimulus pedestal with the non-dominant hand. Electrophysiological activity was continuously recorded with a 64-channel EEG system, while hand kinematics were tracked via motion capture. EEG analyses revealed reliable discrimination between the two object sizes despite identical retinal input. Critically, this result emerged in early visual components and was preserved even when visual depth cues were removed, pointing to a proprioceptive contribution to size constancy. Kinematic data further corroborated these findings: peak Grip Aperture was accurately scaled to physical object size in both full and restricted viewing conditions. These findings advance our understanding of multisensory integration, demonstrating that the brain rapidly combines visual distance and retinal size information to construct accurate object representations supporting both perception and action. Crucially, proprioceptive cues can disambiguate ambiguous retinal input when visual depth cues are unavailable, with greater benefits for sensorimotor control than perception alone. These results have broader implications for neural models of visuomotor control and the development of robotic and prosthetic systems.
Abstract: In many real-world scenarios, bimanual dexterous grasping is essential for object interaction, such as lifting a pot or moving furniture and household appliances. However, the task remains highly challenging due to the large action space introduced by the high degrees of freedom of two dexterous hands and the diversity of object geometry. To address these challenges, we present BiDexAffordance, a collaborative, affordance-driven framework for bimanual dexterous grasp generation. At its core, BiDexAffordance predicts 3D bimanual affordance maps that extract generalizable interaction priors from object point clouds. These learned priors identify regions of objects that are easier to manipulate using minimal simulated interaction data and guide the subsequent optimization process toward physically interactable regions. By doing so, the framework significantly reduces the action space while preserving pose diversity, and significantly increases generalizability among unseen objects with similar local grasping areas. We evaluate BiDexAffordance across diverse object categories in simulation and validate its effectiveness through real-world experiments, demonstrating its ability to accelerate grasp synthesis while improving success rates on unseen objects.
Abstract: Robots are rarely trusted with delicate tasks demanding both a secure grip and a gentle touch. Grasp stability depends on friction through force-closure constraints. Yet, friction is often treated as a fixed prior. In practice, even small misestimation alters the admissible wrench space and can lead to grasp failure. Humans instead infer friction at initial contact and adjust grip forces accordingly. When friction is insufficient, the grasp can be adapted. Once the object is lifted, forces are further regulated to prevent slip. This relies on extracting frictional and slip-related information independently from other factors such as geometry or loading. In robotics, high-resolution tactile sensors provide dense measurements of deformation of their soft elastomeric surface, offering detailed access to contact mechanics. However, these signals vary with geometry, contact location, and loading, making it difficult to directly recover frictional information under uncertainty. The challenge is therefore not sensing alone, but transforming tactile signals into representations that isolate friction and slip while filtering out irrelevant variability. Using the ShadowTac vision-based tactile sensor, which measures soft membrane deformation at around 200 points across the contact interface, we investigate this idea through two complementary approaches spanning the grasp cycle. Before lift, friction is estimated from a single normal press using a two-dimensional representation of shear deformation that generalizes across geometries, enabling detection of highly slippery contacts and repositioning of the end-effector. During grasp, incipient slip is estimated by projecting 3D deformation onto two dominant modes, yielding a continuous estimate of the safety margin to slip. These results suggest that a key missing ingredient in robotic grasping is a representation of contact that captures physically meaningful and practically useful information about friction.
Abstract: Achieving contact-rich robotic manipulation across a diverse set of environments and tasks requires dextrous hands that exceed the capabilities of simple parallel-jaw grippers. In today’s state-of-the-art dextrous hand hardware, solutions aim for complete anthropomorphism with torque-transparent actuation. Quasi-Direct Drive (QDD) actuators enable this transparency, but their size, weight, and heat generation make full actuation across 20+ joints an extremely di icult problem. To overcome these issues, we propose a novel, four-finger dextrous hand featuring an asymmetric morphology inspired by distinct digit responsibilities in human grasping. We allocate three radial fingers (thumb, index, middle) with torque-transparent QDD metacarpophalangeal (MCP) joints for precise, high-fidelity manipulation, and a fourth ulnar finger (ring) to improve grasp robustness through compliance and strength. The MCP joints of the middle and ring fingers are driven by a single, larger QDD motor. A novel pretensioned, compliant linkage mechanism is introduced to govern relative motion between the two fingers, enabling rigid actuation of the middle finger for precise movements under nominal loads. When this load is exceeded, the middle finger yields compliantly and drives an underactuated carpometacarpal (CMC) joint in the ring finger, generating an encompassing, “cupping” motion for more robust high-force grasps. Due to volumetric scaling properties of motors, our single actuator setup can double output torque across the two fingers while greatly reducing heat generation. By balancing directdrive actuation with bioinspired joint coupling in a dextrous hand, this ongoing work aims to provide human-like dexterity while tackling practical design requirements regarding weight, size, and continuous power.
Abstract: This work presents a methodology for the design of a pair of wearable robotic appendages, known as Supernumerary Robotic Limbs, or SuperLimbs for short. Specifically, we aim to design SuperLimbs for assisting astronauts while they perform partial-gravity Extra-Vehicular Activities (EVAs) on the Moon. NASA has identified recovering from a fall as a high-risk process needing an effective countermeasure. Preliminary human study data discovered uniform behavior in astronaut’s poses as they performed a postfall recovery wearing a space suit assembly. Based on this observation, this paper presents a complementary biomechanical model which estimates the torques required by an astronaut to navigate their body through a post-fall recovery. By comparing maximum allowable joint torques of the astronaut with the required joint torques, we identify the gap that SuperLimbs must fill by exerting necessary forces along a desired trajectory. A parametric optimization problem is formulated for designing SuperLimbs that meet these requirements with least energy consumption. A two-phase design optimization method is developed. Phase 1 consists of utilizing a coarse-grid AI searcher that rules out invalid designs that do not meet basic functional requirements. Phase 2 uses the optimal permutation from Phase 1 as an initial estimate for a finegrid parameter-sweep optimization where energy dissipation across the actuators and task-space tracking accuracy are used as performance metrics. Based on the optimal design, an Earth-based prototype is built in-house at the NASA Jet Propulsion Laboratory, and its feasibility for astronaut’s fall recovery assistance is evaluated.
Abstract: Accelerating autonomous motion in robotic manipulation is crucial for industrial efficiency. While behavior cloning via bilateral control is a common method for mimicking human skills, conventional fixed-impedance control faces a severe trade-off. High stiffness enables high-speed trajectory tracking but risks environmental damage during unforeseen contacts, whereas low stiffness ensures safety but fails to suppress inertial forces during rapid motion. To overcome this, we propose an approach inspired by human motor control based on the Equilibrium Point (EP) hypothesis. We suggest that what is currently missing in robotic manipulation is the dynamic coordination between the EP and stiffness. Instead of time-series position and force trajectories, our method models human operational force as a dynamic interaction defined by a virtual EP and joint stiffness. Since this framework treats motion as a continuous transition of force-equilibrium states, position errors are mechanically interpreted as the natural deflection of a virtual spring. This provides an intrinsic safety mechanism: when encountering environmental variations, the robot transitions into a safe equilibrium state rather than generating destructive compensatory torques. In this framework, the EP and joint stiffness are estimated as time-series data using a particle filter from bilateral demonstrations. To accelerate the task, greater operational force is required. Thus, during autonomous execution, the robot amplifies the estimated stiffness. This generates a powerful attractive force toward the EP, achieving high-speed motion while maintaining the underlying human-inspired impedance characteristics. We evaluate the robot’s performance when reproducing accelerated tasks under environmental variations, such as unexpected obstacles. Compared with conventional fixed-impedance methods, we quantitatively assess the robustness of the proposed EP-based approach. By mimicking human stiffness modulation relative to an attractor, this system achieves rapid task execution while maintaining safe environmental adaptability. This suggests the EP hypothesis serves as a fundamental principle for robust robotic dexterity.
Abstract: Dexterous manipulation in humans is remarkably fast. Yet, robotic systems equipped with state-of-the-art tactile sensors still lag far behind human throughput in contact-rich tasks. A central open question is which tactile sensing properties are most critical to sustain a high-throughput — and whether degrading them impairs the ability to manipulate at all, or merely the speed at which manipulation can be performed. We addressed this question by systematically degrading four key tactile properties in human participants: spatial resolution (SR), force threshold (FT), friction, and fingertip compliance. For each condition, we measured dexterous manipulation throughput on two standardized Lafayette dexterity tests. Our central finding is that degrading tactile sensing slows manipulation but does not prevent it: participants completed all tasks across the full range of degradation, with throughput reduced to as low as 10% of bare-finger performance under maximal degradation. Spatial resolution and friction emerged as the strongest individual predictors of throughput, while force threshold and compliance contributed significantly but with smaller effect sizes. Notably, friction and compliance exhibited a marginal interaction, suggesting some redundancy in how surface mechanics and fingertip deformability jointly support manipulation speed. These results reframe tactile sensing as a resource that governs how fast dexterous tasks can be executed rather than whether they can be executed at all. For robotics, this suggests that sensor development efforts should prioritize the properties that most constrain manipulation speed.
Abstract: Grasping mechanisms must both create and subsequently hold grasps that permit safe and effective object manipulation. Traditional mechanisms address the different functional requirements of grasp creation and grasp holding using a single morphology, but have yet to achieve the simultaneous strength, gentleness, and versatility needed for many applications. In this presentation, we present “loop closure grasping,” a method of robotic grasping that addresses these different functional requirements through topological transformations between open-loop and closed-loop morphologies. Topologically open-loop mechanisms (e.g., human fingers, most current gripper designs) enable versatile grasp creation via unencumbered tip movement around the object, but lack the simultaneous strength and compliance needed for holding heavy yet fragile objects. Closed-loop mechanisms (e.g., slings) can bear heavy loads in a passive cradled state with effectively infinite bending compliance, but present challenges for grasp creation because the object must somehow enter the loop. Loop closure grasping circumvents the tradeoffs of single-morphology designs by transforming the mechanism’s topology from open-loop to closed-loop between the grasp creation and holding stages, surpassing the capabilities of traditional human-inspired open-loop design paradigms. We formalize these morphologies for grasping, formulate the loop closure grasping method, and present a design architecture and implementation using soft growing inflated beams, winches, and clamps. Finally, we demonstrate grasps involving historically challenging objects (e.g., watermelon, human body, pile of pipes), environments (e.g., cluttered, long-distance), and configurations (e.g., interlocking, woven).
Abstract: Kinesthetic teaching through robot hand-guiding provides a natural interface for collecting demonstrations in imitation learning and programming-by-demonstration. However, extended sessions cause operator fatigue, reducing demonstration quality and limiting scalability. Current industrial hand-guiding approaches typically provide no active assistance, and alternatives require costly wrist-mounted force-torque sensors or rely on learned motion priors unavailable for new tasks. We propose RHOAS, a hand-guiding scheme that actively supports operator-intended motions using modelbased force estimation without additional hardware. Our approach considers robot hand-guiding as an actively controlled interaction by the human operator, rather than an interaction with a passive environment. Standard methods used for hand-guiding typically rely on general passivitybased compliant control architectures that unnecessarily increase operator effort and limit the range of demonstrable motions without providing the intended stability guarantees in active interaction. Instead, our design reflects the human capability to control and stabilize the robot, and matches the robot’s active response to human motor capabilities, i.e. providing enough responsiveness within the range of human physical limits. We address practical challenges of relying on observer-based force estimation, including measurement noise, reduced estimate accuracy close to kinematic singularities, and gravity compensation errors. In a user study with 16 participants on a KUKA LWR iiwa we demonstrate statistically significant reductions in physical effort, substantially improved maneuverability for both precise and agile tasks, and clear user preference over baseline methods.
Abstract: The human hand intricacy (more than 20 degrees of freedom) has driven researchers to investigate strategies aimed at reducing hardware and control complexity during the development of hand prostheses and exoskeletons. Foundational studies showed joint angles are not independently controlled but act synergistically due to tendon coupling, biomechanical limits, and neurological schemes, with common patterns called postural synergies. Over the years, the synergistic approach has been widely adopted in the hardware design of hand prosthesis and, to a lesser extent, in hand exoskeletons. In this study, we investigate a synergistic strategy on a 14-joint textile-based hand exoskeleton controlled by two degrees of actuation. To achieve this, each textile finger features a dual-pocket architecture, containing airtight chambers: an upper chamber for flexion and a lower one for extension. The flexion pocket exploits localized anisotropies to obtain bending motions that are kinematically compatible with a human finger. Four possible solutions were designed and manufactured using a computerized knitting machine, featuring different knit architectures and geometries in the finger joints. An experimental kinematic validation of the joint angles and curvature was conducted to assess the designs. Our preliminary results demonstrate that it is possible to tailor the textile structure to implement a synergistic control of the device, reducing the structure’s active degrees of freedom to two. Thanks to the ability to programme the anisotropies present in the actuator joints, it will be possible, by applying a certain pressure within the flexion and extension chambers, to control and shape the exoskeleton for gripping objects, thereby reducing the complexity and weight of the device.