VLAConf

VLAConf: Calibrated Task-Success Confidence for Vision-Language-Action Models

Dehao Huang, Aoxiang Gu, Chengjie Zhang, Bolin Zou, Wenlong Dong, Zilang Cen, Yue Wang, Hong Zhang

[1] Department of Electronic and Electrical Engineering, Southern University of Science and Technology, Shenzhen, China.

[2] Zhongguancun Academy, Beijing, China.

[3] National Cybersecurity Academy, Wuhan University, Wuhan, China.

Paper

Video (TBD)

Code (TBD)

Supplementary (TBD)

Abstract: Confidence estimation for Vision-Language-Action (VLA) models is essential for robots to perform manipulation tasks in the open world, providing crucial signals for risk-sensitive decision-making and failure anticipation. Existing confidence estimation methods typically rely on ensemble-based paradigms or action-token probabilities to predict the likelihood of task success. However, they still encounter challenges in computational efficiency and cross-architecture generalizability. These methods usually require repeated sampling, leading to inference inefficiency, and are restricted to VLA models with discrete action outputs, making them difficult to apply to continuous action spaces. To address this issue, we propose VLAConf, a one-class discriminative confidence framework. By leveraging frozen pretrained VLA internal representations, VLAConf directly estimates step-wise anomaly scores in a single forward pass using a lightweight confidence head, thereby eliminating the overhead of exhaustive resampling. We additionally use step-conditioned modeling to encode rollout-phase information along the manipulation trajectory. Experiments on the LIBERO benchmark demonstrate that VLAConf significantly improves the quality of the confidence signal constructed for post-hoc calibration, outperforming existing baselines by a large margin in inference efficiency. The effectiveness of VLAConf is further validated in real-robot experiments.

Methodology:

Problem formulation for VLA confidence estimation. Given an instruction and an observed rollout prefix, the goal is to estimate the probability that the current execution will eventually succeed.

Overview of VLAConf. Given a rollout prefix, VLAConf computes step-wise anomaly scores from frozen VLA features, aggregates them into a prefix-level signal, and calibrates it into task-success confidence.

Simulation Experiments:

Table 1. Main comparison on the standard LIBERO suites. VLAConf produces sharper task-success probabilities than token-based baselines while keeping inference lightweight: on OpenVLA-OFT, it substantially improves Brier score and NLL at nearly the same runtime as TokenProb, and is about 11 times faster than ConfidenceVLA. Because it operates on frozen internal representations rather than action-token probabilities, the same confidence head also transfers directly to the continuous-action π^0.5 backbone, where token-based methods are not applicable.

Table 2. Average robustness under shifted LIBERO benchmarks. Across LIBERO-Pro and LIBERO-Plus, VLAConf preserves its advantage beyond the original training distribution, delivering markedly better Brier score and NLL than ConfidenceVLA on OpenVLA-OFT while maintaining comparable task success rates. The method also remains effective on the continuous-action π^0.5 backbone, showing that representation-level confidence estimation generalizes across both distribution shifts and action parameterizations.

Real-world Experiments:

Real-world experimental setup

Table 3. Real-world evaluation across three manipulation tasks. Over 151 physical rollouts, VLAConf remains informative outside simulation, with stronger confidence quality on the easier cup-on-table task and larger errors on the more challenging towel-folding task. The results show that the learned confidence signal transfers to real robot execution while reflecting meaningful differences in task difficulty.

Examples of Success Trajectories

"Put the cup on the black table."