Embedded Files
main.m4vAbstract
Abstract
Accurate hand motion capture (MoCap) is vital for applications in robotics, virtual reality, and biomechanics, yet existing systems face limitations in capturing high-degree-of-freedom (DoF) joint kinematics and personalized hand shape. Commercial gloves offer up to 21 DoFs, which are insufficient for complex manipulations while neglecting shape variations that are critical for contact-rich tasks. We present FSGlove, an inertial-based system that simultaneously tracks up to 48 DoFs and reconstructs personalized hand shapes via DiffHCal, a novel calibration method. Each finger joint and the dorsum are equipped with IMUs, enabling high-resolution motion sensing. DiffHCal integrates with the parametric MANO model through differentiable optimization, resolving joint kinematics, shape parameters, and sensor misalignment during a single streamlined calibration. The system achieves state-of-the-art accuracy, with joint angle errors of less than 2.7 degree, and outperforms commercial alternatives in shape reconstruction and contact fidelity. FSGlove’s open-source hardware and software design ensures compatibility with current VR and robotics ecosystems, while its ability to capture subtle motions (e.g., fingertip rubbing) bridges the gap between human dexterity and robotic imitation. Evaluated against Nokov optical MoCap, FSGlove advances hand tracking by unifying the kinematic and contact fidelity.
System Architecture & Hardware Design
System Architecture & Hardware Design
The system gathers IMU sensor data via an SBC and then sends them from the palm tracker to a host computer. The computer calibrates and models the hand shape, aligns the data, and outputs a final hand-and-object mesh.
The Raspberry Pi Zero 2W serves as the system’s core, receiving sensor data from a custom UART-USB daughter board over a USB bridge. The daughter board collects IMU data via a flex-printed circuit.
Dorsal Tracker Setup
Dorsal Tracker Setup
FSGlove offers multiple dorsal tracker options:
Basic: This version does not include a dorsal tracker and only captures finger kinematics.
With Optical Dorsal Tracker: This version is equipped with an optical dorsal tracker and is integrated with the Nokov Motion Capture system
With HTC Vive Tracker: This version features an HTC Vive tracker, allowing seamless integration with the Vive VR system for interaction in virtual reality environments.
Calibration with DiffHCal
Calibration with DiffHCal
We developed a 2-step calibration Flow. The one, pose calibration process (a), involves 3 simple poses that are covered with 2 movements the x-axis rotation and the y-axis rotation.
The second one, shape calibration process (b) requires each fingertip to touch the thumb tip. We sway the joints during this calibration to record more data.
For comprehensive details and mathematical derivations, please refer to our paper.
Experiment Results
Experiment Results
We have designed a series of experiments and selected VRTRIX Pro, Meta Quest 3, and Manus Metaglove for a comparative study. These represent mainstream hand tracking solutions. Notably, our glove offers the lowest cost while maintaining competitive sampling rates and pose degrees of freedom (DoFs).
Comparison of hand glove motion capture solutions.
Cost analysis of FSGlove.
Different hand tracking solutions are being tested.
The experiments conducted are as follows:
Single Joint Measurement: Aimed at validating the raw sensor performance.
Shape Reconstruction: Designed to test the raw performance of the mesh reconstruction algorithm.
Finger Pinch Tracking: Intended to evaluate the glove's performance in grasping teleoperation.
Hand-object Interaction Consistency: Focused on assessing the glove's performance in hand-object interaction scenarios.
Single Joint Measurement
In our experiments, we use the output from the Nokov system as the benchmark to gauge the error in pose estimation provided by the IMU system.
The non-linearity of the sensor is 0.7%. We can deem such an IMU system is reliable.
Shape Reconstruction
Our solution demonstrates comparable performance to the Quest3 and outperforms other commercially available gloves.
Specifically, the VRTRIX glove exhibits inferior performance in thumb articulation and demonstrates rigid movements, whereas the MANUS glove suffers from inadequate tracking accuracy in the little finger.
However, our analysis reveals that the Meta Quest3 demonstrates comparable, if not superior, performance in fingertip pinching tasks. This enhanced capability can be attributed to its computer vision based algorithm, which excels in capturing geometric details, albeit with sensitivity to occlusion.
Qualitative results of shape reconstruction.
Finger Pinch Tracking
Our method achieves the best average error. Notably, it attains the lowest error for the index and ring fingers, confirming its superior accuracy in most finger measurements.
Hand-object Interaction Consistency
Our proposed method outperforms other approaches on most objects. The relatively lower scores of MANUS and VRTRIX are largely attributable to unnatural hand motions induced by external forces during the interaction, which are consequently filtered by their built-in algorithms.
FSGlove Hardware Setup
FSGlove Hardware Setup
Here is a video demonstration of the glove hardware assembly process, which illustrates the fabrication of the PCB and its integration with the SBC. The complete manufacturing process will be published in the near future.
hardware_setup.m4vPage updated
Google Sites
Report abuse