OpenVLA made it cheap to adapt a 7B robot policy: LoRA on one GPU matches full fine-tuning. Because OpenVLA writes actions as tokens, it also reports how sure it is of each one. I propose to test whether that cheap fine-tuning keeps the confidence honest, and to set each robot’s ask-for-help threshold in a digital twin of the room where it will work.
The paper
OpenVLA (Kim, Pertsch, Karamcheti et al., 2024) is a 7B vision-language-action model built on Prismatic-7B: SigLIP and DINOv2 image features feeding a Llama 2 language model. It was trained on 970k robot episodes from Open X-Embodiment, in 21,500 A100-hours.
The paper names its own limits: one camera image and no history, about 6 actions per second on an RTX 4090, and reliability below 90%. Its LoRA and 4-bit results come from a smaller OpenVLA variant (a SigLIP-only vision encoder and a smaller data mix, footnote 4 of the paper), on a few tasks, so they show the method works rather than measure the released model. Two of its strengths meet in this proposal. Fine-tuning is cheap enough that LoRA is the standard way to adapt OpenVLA to a new robot, and its token probabilities are the signal that recent work uses to decide when a robot should ask a person for help.
Proposed research
A robot that asks for help must know when it is likely to fail. Token-based VLAs give a probability for every action token, and recent monitors build on it in different ways. The simplest reads it directly as the chance of success (Zollo and Zemel). Others train a classifier on token uncertainty (INSIGHT), wrap it in conformal prediction (ReconVLA), or fit a separate calibrator on labelled rollouts (VLAConf). Each is fitted to the rollouts of one fine-tuned model. If the way a model is fine-tuned changes how its confidence relates to success, the raw confidence misleads, and the learned monitors must be refitted after every new fine-tune.
In language models, LoRA fine-tuning on small datasets makes models overconfident. Two Bayesian treatments of LoRA repair much of it: Laplace-LoRA, a post-hoc Laplace approximation over the trained adapter’s weights, and ensembles of separately trained adapters. I found no controlled comparison of fine-tuning method and data size for the calibration of a robot policy. Zollo and Zemel’s calibration study measured released OpenVLA checkpoints, all fine-tuned the same way (LoRA at rank 32), and did not vary the fine-tuning.
Why it matters. If H1 holds, raw confidence misleads after a cheap fine-tune, and a monitor fitted to one fine-tune needs new labelled rollouts for the next. If H2 holds, the repair is cheap: a posterior over the adapter or a few more adapters, not a new model. Whatever H1 shows, H2 tests whether simple rescaling is as good, and a learning curve over the calibration set shows how many labelled rollouts it needs.
| Model | OpenVLA-7B from its released pretrained weights. |
|---|---|
| Fine-tuning | Full; LoRA at rank 8; LoRA at rank 32, OpenVLA’s default. The same step budget and tuning effort for each. |
| Data | 10% and 100% of each LIBERO suite’s demonstrations; the 10% subset is drawn per task. Two sizes test a contrast, not a trend; a 30% point is added if the budget allows. |
| Confidence | Raw token confidence; temperature scaling; action-wise Platt scaling; Laplace-LoRA on the rank-32 adapter; an ensemble of four rank-32 adapters. All are scored on the same rollouts of one acting adapter, so success stays fixed. Running the ensemble as the policy is a separate comparison with its own rollouts. |
| Matching | For “similar success” in H1, LoRA and full fine-tuning checkpoints are matched on success measured with a different simulator seed, never on the test rollouts. |
| Tasks | LIBERO Spatial and Long first, then Object and Goal (OpenVLA’s own fine-tuning benchmark). |
| Shift (H3) | LIBERO-Plus camera, lighting and layout changes, one severity, from the same initial states, on the rank-32 LoRA and full fine-tuning checkpoints trained on all demonstrations. The primary contrast: LoRA’s confidence-minus-success gap over full fine-tuning’s, with and without the changes. Calibrators stay frozen; refitting them is a reference. |
| Metrics | Success rate; mean confidence minus success (the direction of the error); expected calibration error, Brier score and NLL of the first-step confidence, as in the calibration study; failure-prediction AUROC with failure as the positive class, added here; reliability diagrams; confidence over the course of an episode. |
| Trials | 500 rollouts per configuration (10 tasks × 50 initial states, one seed), as in the calibration study; OpenVLA’s paper averages three seeds, 1,500 rollouts. 95% bootstrap intervals. |
| Splits | Fitted fixes and thresholds use episode-level 20% / 80% calibration and test splits, repeated 1,000 times, as in the calibration study. Test episodes are never used for fitting or for choosing a method. A 20% split of 500 Spatial rollouts holds only about 15 failures, so a learning curve over the calibration-set size checks whether that is enough. |
Compute, estimated. The first pass covers two suites (Spatial and Long), three fine-tuning methods and two data sizes: 12 runs. The ensembles add three rank-32 adapters per suite and data size: 12 more, 24 runs in all, 4 of them full fine-tuning. OpenVLA reports, on its own Franka datasets, 10–15 A100-hours for a LoRA run (one GPU, about 60 GB at batch size 16) and 8 A100s for 5–15 hours for a full fine-tune (163 GB at batch size 16, sharded across two GPUs; §5.3, Table 1). At those rates the 20 LoRA runs take about 200–300 A100-hours and the 4 full runs 160–480: 360–780 in all, before Laplace fitting and repeat seeds. The first weeks time one LIBERO run before the grid is fixed. Evaluation: at OpenVLA’s 6 actions per second (model only, on an RTX 4090), 500 episodes capped at 220 steps (Spatial) or 520 (Long) take up to about 5 or 12 GPU-hours per checkpoint, plus simulator time; the 12 checkpoints need about 100 GPU-hours, and the H3 shift runs (3 changes × 2 checkpoints × 2 suites) about 100 more. Full fine-tuning needs 80 GB GPUs; adding Object and Goal roughly doubles everything.
Risks. Episode-success confidence built from token probabilities is a proxy, and the calibration study found it weakest late in failed episodes. LIBERO is near saturation for newer models, but OpenVLA still fails often enough to calibrate against: about 75 of 500 Spatial episodes and 230 of 500 Long episodes at its published 84.7% and 53.7%. An ensemble costs four times the inference when it acts.
Proposed application
Calibration needs rollouts with known outcomes from the place where the robot will work, and collecting them on a real robot is slow and sometimes unsafe. A digital twin can supply them. Scan the room with a phone, rebuild it in a physics simulator, and run the fine-tuned OpenVLA there hundreds of times before its first real trial. The outcomes set the room’s ask-for-help threshold and show which failures to fix first.
What a user gets, before the robot’s first real trial in a room:
What it builds on. The pieces exist. SIMPLER and ReVeal show that success in simulated and reconstructed scenes tracks real success well enough to rank policies, and RialTo builds twins from quick scans to train policies with reinforcement learning. This application puts them into one routine for deploying a fine-tuned OpenVLA, and adds the step it needs from the research project: setting the room’s ask-for-help threshold from the twin rollouts.
Why it is feasible, and what is still missing. OpenVLA already runs in simulators (LIBERO, SIMPLER) and in 4-bit on one GPU. ReVeal rebuilds workspaces from one to three minutes of phone video. RialTo builds a twin from a quick scan, but a person separates the objects and sets joints and physical parameters, and its policies use point clouds, not OpenVLA’s camera images. ReVeal leaves object-level fidelity, mass and friction to future work. So the twin is engineering with known gaps, and a pilot comes first.
The pilot, a go/no-go gate. One arm OpenVLA already supports (for example the WidowX of BridgeData, which SIMPLER also simulates), one tabletop and two or three rigid-object tasks with clear success checks. Three checkpoints run 30 times each in the twin and on the real robot, from matched start positions. The routine continues only if the twin ranks the checkpoints in the same order as the real robot and its success rates are within about 15 points of the real ones.
The aim. In a home, a lab or a classroom, a robot that asks for help earlier and fails less often for a given number of help requests. The threshold from the twin is a starting point; the first real trials test how well it transfers.
The test. Two scores: the first-step confidence, to decide before acting, and the per-step confidence, to stop during the task. Calibrators fitted on (a) LIBERO, (b) twin rollouts and (c) a few real rollouts, and the twin’s threshold, are frozen before any real test rollout. They are then compared on an untouched set of real rollouts from matched start positions: calibration error and the confidence-minus-success gap; at a fixed number of help requests, the share of failures caught, the false alarms and the failures among the episodes allowed to continue; and how many steps before the first unrecoverable error the request comes. The robot only logs when it would have asked and is not stopped, so a request does not change the outcome it predicts. A monotone rescaling changes the probabilities but not which episodes are flagged at a fixed budget, so the ranking measures compare raw scores and the calibration measures compare calibrators.
The risk. ReVeal finds that lower reconstruction fidelity weakens sim-to-real agreement, and it measures agreement in success rates, not in confidence, so the twin calibrator may transfer imperfectly; that transfer is what the experiment measures. Deformable objects and fine contact are out of scope for a first version.
One loop
Related work
| Work | What it does | What this proposal adds |
|---|---|---|
| Zollo and Zemel, 2025 | First calibration study of VLAs: released fine-tuned OpenVLA, MolmoAct, UniVLA and NORA on LIBERO, plus 8- and 4-bit OpenVLA; prompt ensembles and action-wise Platt scaling. | How the fine-tuning method and data size change calibration. |
| INSIGHT, 2025 | Trains classifiers that turn π0-FAST’s token uncertainty into help triggers. | How the fine-tuning method shifts the uncertainty such classifiers learn from; thresholds for a given room. |
| RECALL, 2026 | Uncertainty-guided recovery data for continual fine-tuning of autoregressive VLAs. | Calibration of the uncertainty it relies on. |
| ReconVLA, VLAConf, 2026 | Conformal prediction on action tokens; a representation-level success confidence with confidence-triggered handoffs. | Where the labelled rollouts for calibration come from in a new room. |
| Laplace-LoRA, LoRA ensembles, 2023 | A post-hoc Laplace posterior over LoRA weights, or an ensemble of adapters, reduces the overconfidence of fine-tuned language models. | A test on a robot policy. |
| SIMPLER, 2024; ReVeal, 2026 | Simulated and reconstructed scenes whose success rates track real ones. | Calibrating confidence, rather than ranking policies. |
| RialTo, 2024 | Digital twins from quick scans of a real scene, with objects and physical parameters set by a person, used to train point-cloud policies with reinforcement learning. | Using the twin to evaluate and calibrate before deployment. |
Timeline
Sources. OpenVLA (arXiv:2406.09246v3): §3.5 for training compute and inference speed, §5.3, Table 1 and Table 8 for parameter-efficient fine-tuning, §5.4 and Table 2 for quantization (footnote 4: both use a smaller SigLIP-only variant), §6 for limitations, Appendix E for LIBERO (LoRA at rank 32, 500 trials per suite for each of three seeds, 76.5% average). LIBERO episode limits from OpenVLA’s evaluation script. The related work was read on arXiv on 19 October 2026; the gaps named here are the ones I found, and I may have missed work. The compute figures are estimates from OpenVLA’s reported numbers, not measurements. Calibration protocol (first-step confidence, 12 equal-mass bins, 20% / 80% splits) from Zollo and Zemel §3–4 and Appendix C.