Octo read-along

No. Open X-Embodiment has about 1.5M episodes, and Octo uses 800k of them from 25 datasets. The authors drop datasets without images or without delta end-effector actions, and ones that are too repetitive, low-resolution or niche; then they reweight the rest by hand, doubling the more diverse ones. RT-1-X and RT-2-X were trained on a 350K-episode subset. (Sec. II, Sec. III-B, Fig. 3, Table III)

No. Of the nine evaluation setups, the three zero-shot setups (WidowX BridgeV2, UR5, RT-1 Robot) use robots from the pretraining data. The Figure 5 comparison uses tasks from their own datasets, with changes in object positions, lighting, backgrounds and distractors. Table VII separately tests new objects, scenes and skills on WidowX. Robots outside the mix, the ViperX in Berkeley Coke and the two-arm ALOHA, are tested only after finetuning on about 100 demonstrations. (Sec. IV, Sec. IV-A, Fig. 5, Table VII, Appendix F)

52 percentage points of average success, not a 52% relative gain. In Table I, finetuned Octo averages 72% over six setups, a ResNet+Transformer trained from scratch 20%, and VC-1 15%. The scratch baseline has 28M parameters in total: a ResNet visual encoder with FiLM language conditioning and a small transformer action decoder trained with a diffusion objective, because Octo's own architecture overfit quickly when trained from scratch on so few demonstrations. Table I says each domain is averaged over 20 trials, but Appendix F gives 10 trials for Berkeley Bimanual. (Sec. IV, Sec. IV-B, Table I, Appendix F-C)

Octo-Base (93M) had on average 29% higher success than RT-1-X (35M) over three robots, with two language tasks per robot and 10 trials per task; the paper does not say whether this is points or a relative gain. It performed similarly to the 55B RT-2-X on WidowX and the RT-1 Robot. RT-2-X is not openly available, so its WidowX numbers come from another paper (Black et al. [7]) on the same tasks, and its RT-1 Robot runs were done by the RT-2-X authors. (Sec. IV-A, Fig. 5 and footnote 1)

No. Fig. 2 says new inputs and action heads can be added without modifying pretrained parameters, but the finetuning recipe updates the full model, which worked better than freezing parts of it. Every setup uses the same recipe: about 100 trajectories, 50k steps, cosine learning-rate decay with linear warmup, and about 5 hours on one 24 GB A5000 for Octo-Base. (Fig. 2, Sec. III-C, Sec. III-D)

Berkeley Insertion adds force-torque input. Berkeley Pick-Up uses joint position control, and Berkeley Bimanual gets a new head for ALOHA's 14-dimensional joint and gripper actions. Berkeley Coke (one ViperX arm) and Bimanual (two ViperX arms) are robots outside the pretraining mix, which has only single-arm data. CMU Baking and Stanford Coffee keep end-effector delta actions on new tasks. (Table I, Fig. 4, Appendix F-C)

No. Octo is a transformer of 27M (Small) or 93M (Base) parameters trained on robot data. Images go through a shallow CNN into 16×16 patch tokens, and language goes through a frozen T5-base encoder (111M) as 16 tokens. Appendix E reports that larger or finetuned T5 encoders did not help. (Sec. III-A, Table V, Appendix D, Appendix E)

The parts are borrowed, and the paper says so: a transformer backbone, language and goal-image conditioning, a diffusion action head, action chunks, and ViT layouts and learning-rate schedules. The claims are the combination for cross-embodiment policies, being the first generalist policy shown to finetune to new observation and action spaces, and a fully open release of checkpoints, training and finetuning code, and data loaders. (Sec. I, Sec. II, Sec. III-E)

Tables II and VI test Octo-Small on WidowX, with 40 trials across two language-conditioned and two goal-image-conditioned tasks. The full model scores 83%; the MSE head 35%, discretized actions 18%, the 11-dataset RT-X mix 60%, Bridge data only 43%, and ResNet-50 + Transformer 70%. Separately, Figure 6 compares Octo-Tiny (10M), Small (27M) and Base (93M) on UR5 and WidowX, with 10 trials on one language-conditioned task per robot. (Sec. IV-C, Table II, Table VI, Fig. 6)

Goal images, in the one direct comparison: zero-shot on WidowX, goal-image conditioning had a "25% higher success rate" than language; the paper does not say whether that is points or a relative gain. The authors suggest two possible reasons: goal images give more information about the task, and only 56% of the pretraining data has language annotations. Training drops the language or the goal image at random, and datasets without language always use goal images. (Sec. III-D, Sec. IV-A, Sec. V)

No, by the authors' own account. Finetuning results were often stronger with only the third-person camera than with third-person plus wrist, and only 27% of the pretraining data has wrist images. Wrist images are also resized to 128×128 (64 tokens), against 256×256 (256 tokens) for the third-person view. (Sec. V, Appendix D)

Well to new objects, less to new environments, hardly to new skills. In Table VII (Octo-Small, 20 trials over two tasks per row), success averages 85% in distribution, 80% with novel objects, 40% in a novel environment and 5% for novel skills such as flipping a cup. The text calls the drop in a new scene slight, and the caption says Octo generalizes well to novel environments, but the table shows the average falling by more than half. (Sec. IV-A, Table VII)