ForceVLA: Enhancing VLA Models with a Force-aware MoE for Contact-rich Manipulation

Abstract

Vision-Language-Action (VLA) models have advanced general-purpose robotic manipulation by leveraging pretrained visual and linguistic representations. However, they struggle with contact-rich tasks that require fine-grained control involving force, especially under visual occlusion or dynamic uncertainty. To address these limitations, we propose ForceVLA, a novel end-to-end manipulation framework that treats external force sensing as a first-class modality within VLA systems. ForceVLA introduces FVLMoE, a force-aware Mixture-of-Experts fusion module that dynamically integrates pretrained visual-language embeddings with real-time 6-axis force feedback during action decoding. This enables context-aware routing across modality-specific experts, enhancing the robot’s ability to adapt to subtle contact dynamics. We also introduce ForceVLA-Data, a new dataset comprising synchronized vision, proprioception, and force-torque signals across five contact-rich manipulation tasks. ForceVLA improves average task success by 23.2% over strong π0-based baselines, achieving up to 80% success in tasks such as plug insertion. Our approach highlights the importance of multimodal integration for dexterous manipulation and sets a new benchmark for physically intelligent robotic control. 

Motivation

Robotic learning has rapidly advanced with Vision-Language-Action (VLA) models like OpenVLA and π0​, leveraging large-scale pre-training on manipulation datasets and strong Vision-Language Model (VLM) backbones for semantic understanding and efficient fine-tuning. However, existing VLAs primarily focus on semantic grounding and spatial planning, largely overlooking crucial force sensing for contact-rich manipulation. This deficiency, coupled with the fact that force requirements dynamically evolve across task phases, limits their ability to handle tasks like insertion or tool use, especially under visual ambiguity.

To address these limitations, we introduce ForceVLA, a novel framework that augments VLAs with a force-aware Mixture-of-Experts (MoE) module called FVLMoE. Grounded in treating 6D external force as a first-class modality, ForceVLA integrates force into the action expert module. FVLMoE performs modality- and phase-aware fusion of VLM-derived visual-linguistic representations with real-time force feedback via a dynamic gating mechanism over specialized expert subnetworks. By adaptively activating experts based on task instructions and interaction feedback, ForceVLA captures subtle, phase-dependent physical variations to generate precise, force-aware actions.

Method

ForceVLA is an end-to-end multimodal robotic policy designed for contact-rich manipulation. Building upon the π0 framework, it integrates vision, language, proprioception, and 6-axis force feedback to generate actions through a conditional flow matching model. Visual inputs from multiple RGB cameras and task instructions are encoded by a SigLIP-based vision-language model (based on PaliGemma) into contextual embeddings. These embeddings, combined with proprioceptive and force cues, condition an iterative denoising process that predicts the action trajectory.

FVLMoE is the core module enabling effective force integration. Force readings are linearly projected into dedicated tokens and fused with vision-language embeddings via a Mixture-of-Experts (MoE) module. Inspired by MoE’s strength in multi-task and modality-specific learning, FVLMoE adaptively routes and processes multimodal inputs. Its output serves as a rich guidance signal for the flow model, allowing ForceVLA to handle subtle contact dynamics and visually ambiguous scenarios with greater precision and robustness.

To evaluate the effectiveness of ForceVLA, we conducted experiments on five diverse contact-rich manipulation tasks: Bottle Pumping, Plug Insertion, USB Drive Insertion, Whiteboard Wiping, and Cucumber Peeling, as in Figure 4. These tasks were chosen to assess fine-grained control, adaptability to varied initial conditions, and the utility of multimodal feedback, particularly force sensing. Each task introduces unique physical challenges: Bottle Pumping requires precise vertical pressing; Plug and USB Drive Insertions involve accurate alignment and force-controlled insertion; Whiteboard Wiping demands smooth trajectory control and surface contact; and Cucumber Peeling tests the ability to apply and maintain controlled force during continuous surface interaction.

Main Results

ForceVLA achieves an average success rate of 60.5% across all five tasks, significantly outperforming all baseline configurations. Compared to the standard π0-base model without force feedback (π0-base w/ F), which achieved an average of 37.3%, ForceVLA shows an improvement of 23.2%. This highlights the substantial benefit of incorporating and effectively processing multimodal information, including force. 

Model Generalization

To evaluate ForceVLA’s generalization, we designed five challenging settings involving object and height variations, visual occlusion, and environmental instability. ForceVLA consistently demonstrated superior generalization, achieving high success rates (e.g., 80.00% in object variation and 90.00% under visual occlusion) by effectively scaling interaction forces and avoiding torque limit violations seen in other models. This robust performance across varied conditions underscores how ForceVLA’s FVLMoE architecture intelligently integrates force feedback—not just for sensing contact, but for modulating actions in response to dynamic physical conditions—enabling versatile and adaptive robotic manipulation.

Ablation Studies

To validate ForceVLA’s force integration strategy, ablation studies compared different fusion approaches. Early fusion methods, injecting force prior to the Vision-Language Model (VLM), severely degraded performance—notably, an MoE-based early fusion failed entirely (0% success)—likely by disrupting pre-trained VLM representations. While simple late fusion (concatenating force at the decoding stage) improved success to 60% over a no-force baseline, our proposed ForceVLA, which introduces force features post-VLM and uses the FVLMoE module for adaptive fusion, achieved a markedly superior 80% success rate. These results confirm that both introducing force after VLM encoding and employing sophisticated fusion via FVLMoE are critical for effective contact-rich robotic behavior.

Visualization and Case Studies

To analyze MoE routing dynamics, we examined expert selection probabilities, normalized temporally across episodes. We observed distinct expert utilization patterns per task: some, like insert plug and peel cucumber, showed clear temporal specialization with experts dominating specific phases, while tasks such as wipe board favored one expert more consistently. Notably, one expert (Expert 0) activated frequently across multiple tasks, suggesting a general-purpose role for common multimodal fusion or control primitives, contrasting with other more phase-specific experts. This reveals learned dynamic computation allocation and functional specialization, reflecting both task semantics and potential training-induced architectural biases.

Videos

ForceVLA inserting unseen plug

ForceVLA inserting plug with visual occlusion