OpenVLA read-along

Absolute: 16.5 percentage points of the paper's success score, over 29 tasks on two robots; some Bridge tasks give half credit. On the 17 BridgeData V2 WidowX tasks (170 rollouts per policy) the means are 70.6% vs 50.6%; on the 12 Google robot tasks (60 rollouts) they are 85.0% vs 78.3%, with overlapping error bars, which the paper calls comparable. Pooling all 230 rollouts of Tables 4 and 6 gives 74.3% vs 57.8%, which matches the 16.5 points (our arithmetic; the paper does not show the calculation). (Abstract, Sec. 5.1, Figs. 3–4, Tables 4 and 6, Appendix B.1.1)

No. RT-2-X is better at semantic generalization (unseen objects, instructions and Internet concepts): 15.5 vs 14.5 points out of 40 trials on the four Bridge semantic tasks, counting half credit. The authors explain this by RT-2-X's larger Internet pretraining and its co-fine-tuning on Internet data, while OpenVLA is fine-tuned on robot data only. In every other category OpenVLA is comparable or better on average, though RT-2-X wins some single tasks, such as Put {Eggplant, Red Bottle} into Pot (8.5 vs 7.5). (Sec. 5.1, Fig. 3, Table 4, Appendix B.1.1)

Not fully. RT-2-X is closed and was reached through an API; trained on Bridge data that still has all-zero (no-op) actions, it often predicts the zero action and freezes, so the authors always take its second-most-likely action, as the Open X-Embodiment team did. OpenVLA's training drops the no-op first step of every Bridge demonstration. The authors credit part of OpenVLA's Bridge gain to this cleaning, besides more data (970k vs 350k trajectories) and the fused vision encoder. (Sec. 5.1, Appendix C)

Most parts are borrowed. The backbone is the Prismatic-7B VLM (a 600M-parameter DINOv2 + SigLIP encoder, a 2-layer MLP projector and Llama 2 7B), the action tokens follow RT-2, and the data-mixture weights follow Octo. New are bin edges at the 1st and 99th percentile instead of min–max, the design choices tested in Sec. 3.4, the study of fine-tuning, LoRA and quantization, and an open release of the weights, code and training data. (Sec. 2, Secs. 3.1–3.4, footnote 1)

Each of the 7 action dimensions is cut into 256 bins between its 1st and 99th percentile in the training data, and each bin is one token. The Llama tokenizer reserves only 100 special tokens, so the 256 least-used tokens (the last 256 of the vocabulary) are overwritten. The model predicts the 7 tokens with ordinary next-token cross-entropy on the action tokens only, and a de-tokenizer maps them back to position change, rotation change and gripper. (Sec. 3.2, Fig. 2)

One third-person camera image at 224 × 224 px and the language instruction. It has no wrist camera, no robot state and no observation history, and it predicts one action per step, with no action chunking. The authors list single-image input as the first limitation; the full Diffusion Policy baseline uses two steps of images and robot state, which is why a "matched" version is also tested. (Sec. 3.4, Sec. 5.2 (footnote 3), Sec. 6, Appendix E.1)

No. The WidowX (BridgeData V2, 13.3%) and the Google robot (Fractal, the RT-1 data, 12.7%) are among the largest parts of the training mix. The Bridge tests add shifts in scene, objects, lighting and start pose; the Google robot tests have 5 in-distribution tasks and 7 out-of-distribution tasks with unseen objects, backgrounds, instructions or concepts. The Franka arm is tested only after fine-tuning, and DROID's Franka data was in the pretraining mix (at 10%) for the first two thirds of training. (Sec. 3.3, Secs. 5.1–5.2, Appendix A (Table 3), Appendices B.1.2 and B.2.1)

No. Diffusion Policy trained from scratch matches or beats it on narrow single-instruction tasks such as Put Carrot in Bowl and Pour Corn into Pot, and it moves more smoothly and precisely. OpenVLA wins on the multi-instruction tasks with several objects and has the best averages: 67.2% vs 48.5% on Franka-Tabletop and 58.3% vs 35.0% on Franka-DROID. The abstract's 20.4% is the gap between the aggregate bars in Fig. 5 (63.8% vs 43.4%); these scores include partial credit, and Wipe Table gives up to 2 points per trial. (Abstract, Sec. 5.2, Fig. 5, Table 7, Appendix B.3.1)

On the tasks tested, yes, on average: 68.2% vs 69.7% success. Rank 32 trains 97.6M parameters, about 1.4% of the 7,188.1M-parameter model, and rank 64 trains 195.2M (about 2.7%); both average 68.2%, with different per-task results. The test is small: 33 rollouts per method on two Franka-Tabletop tasks (Put Carrot in Bowl, Move <object> onto Plate). It also uses a different OpenVLA, with a SigLIP-only encoder and pretrained on Octo's smaller data mix, not the main model. (Sec. 5.3 (footnote 4), Table 1, Table 8)

The paper says so but reports no such run. The LoRA run in Table 1 needs 59.7 GB of GPU memory at batch size 16, and the reported time is 10–15 hours on one A100, 8x less compute than full fine-tuning on 8 A100s. Consumer GPUs appear only for inference: about 6 Hz in bfloat16 on an RTX 4090 (Sec. 3.5) and the speed comparison in Fig. 6. (Abstract, Sec. 3.5, Sec. 5.3, Table 1, Fig. 6)

Not on average: on 8 Bridge tasks (80 rollouts per method), the SigLIP-only variant scored 71.9 ± 4.7% in int4 (7.0 GB) and 71.3 ± 4.8% in bfloat16 (16.8 GB), though single tasks varied (Lift AAA Battery: 3 vs 7 of 10). int8 drops to 58.1% because it runs at only 1.2 Hz on the A5000 against a 5 Hz controller; with blocking control all three are comparable. The paper does not reconcile its memory figures: 15 GB for the final model in bfloat16 (Sec. 3.5), 16.8 GB for the variant (Table 2), and 16 GB GPUs as enough (Sec. 5.4). (Sec. 3.5; Secs. 5.3–5.4 (footnote 4); Tables 2 and 5; Appendix D.4 (Table 11))

On 8 Bridge tasks, dropping the Open X mix cost far more than dropping DINOv2. Trained on BridgeData V2 alone, the model falls from 76.3% to 45.6% (about 30 points); removing DINOv2 as well costs another 5 points (40.6%). In separate early tests with a different setup, fine-tuning the vision encoder, unlike common VLM practice, gave 80.0% vs 46.7% with it frozen; the paper says these results are not directly comparable to its other ones. Going from 224 to 384 px gave no gain and tripled training time. (Sec. 3.4, Appendix D.1–D.3, Tables 9–10)