View, scene and instruction in π0.5-DROID

DROID argues that diversity makes policies generalize: 564 scenes, 52 buildings, 86 tasks, and 1,417 exterior camera viewpoints, because the collection software prompts collectors to move and re-calibrate the cameras. The paper expects that "task or camera viewpoint diversity allows for greater generalization to new instructions and camera angles". π0.5-DROID is π0.5 fine-tuned on DROID and released by Physical Intelligence. Without a robot, how much do its actions depend on the camera view, on the scene and on the instruction?

H1, camera diversity buys view invariance. For the same state and the same sampling noise, the actions predicted from exterior camera 2 are about as close to those from camera 1 as two sampling seeds are to each other.

H2, language matters early. Replacing the instruction moves the actions well above the sampling noise early in an episode, and less late.

Both hypotheses and their decision rules were written down before the GPU run.

The training recipe of the released checkpoint is not public. openpi's open-source DROID loader, which openpi calls "an approximate open-source reproduction of the pi05-DROID training pipeline", picks exterior camera 1 or 2 and one of the instructions at random for each sample, and uses only successful episodes.

Camera swap: a small effect

Swapping exterior camera 1 for camera 2 changes the actions by 0.023 [0.019, 0.027], 0.15× [0.12, 0.16] the noise floor. The change is above the floor in 1.4% of states. It stays small in every phase of the episode (0.10× to 0.18×) and in all 13 labs (0.06× to 0.24×, descriptive), and the median paired change in the error against the recorded actions is 0.000 [−0.001, 0.001]. The two views of one scene are very different images: the median pixel difference is 67 of 255.

The model does not ignore the exterior image. The exterior image of an episode from another lab changes the actions by 0.31, 1.9× the floor and 13× the swap, and raises the error against the recorded actions by 0.14. So the model uses the exterior image, but changes little between the two recorded views of one scene. The pre-set rule for H1 expected a ratio near 1 (an interval that overlaps 0.8 to 1.25), so it leaves 0.15× unclassified. Read after the fact, the result supports low sensitivity to this camera swap. It does not show that camera diversity in training causes it, and unseen viewpoints are not tested.

Instruction: H2 fails

A wrong instruction moves the actions less than the sampling noise in the median state. The trend runs against H2: the ratio is 0.51 early in the episode and 0.82 late, and the difference is not significant. The effect is concentrated in a few states: 19% change by more than twice the floor, and the largest by 11×. The true instruction still carries information: another episode's instruction raises the error against the recorded actions by 0.019 [0.011, 0.029], and an empty one by 0.008. Paraphrases are almost free. These probes do not test language following in closed loop, and the swapped-in instructions usually name objects from another scene.

Open loop: error close to the sampling spread

The error against the recorded actions is 0.185 [0.156, 0.217], 1.17× [1.04, 1.33] the noise floor. States with a larger sampling spread also have a larger error (Spearman 0.72). These comparisons do not show how much of the error sampling causes. It beats zero velocity (0.281) and the mean action (0.399). Repeating the previous recorded action, which the model does not get, wins for the first action and over 8 steps; π0.5-DROID wins over the full 1 s chunk (paired −0.020 [−0.036, −0.005]). On the 19 failed episodes the error is 0.370, against 0.153 on the 81 successful ones.

Surprises in droid_100

54 of the 100 episodes have no instruction, and all 19 failures are among them, so the instruction probes use the 46 annotated episodes. The likely reason is the version: openpi's training notes say that v1.0.0 "only has annotations for 30k episodes", and v1.0.1 has about 75k. One lab (TRI) has 29 of the 100 episodes, so a 100-episode slice does not show the diversity of the full dataset.

Viewpoints: consistent, in-distribution. The tested checkpoint gives nearly the same predictions from either recorded exterior view, while it still depends strongly on the exterior image. That fits the paper's argument. It is not the paper's own experiment: the paper's diffusion policies use both exterior cameras together and end-effector actions, and are tested in closed loop after co-training on DROID and demonstrations of each evaluation task.

Scenes: transfer not tested. The actions depend on the images (1.9× to 2.3× the floor for an image from another lab), so the policy is not a motion prior that ignores its cameras. The paper tests robustness in closed loop after co-training with demonstrations from each evaluation task. It does not test zero-shot transfer to unseen scenes, and this probe tests neither transfer nor task success.

Language: a small median effect offline. openpi describes π0.5-DROID as having "good language-following". On 230 states, another episode's instruction moves the predictions less than the sampling noise in the median state, and by more than the noise in 33% of states. This does not contradict good language following on a robot, where instructions matter most when several tasks are possible in one scene. Instructions from other episodes usually name objects that are not in the scene, so feasible alternatives in the same scene remain untested.

Challenges. The GPU was shared with two text-to-speech jobs, and the run waited 69 minutes for one of them. The run needed 24 GiB on a disk that was 95% full, and all of it was deleted after. The training recipe of the released checkpoint is not public; it is inferred from openpi's open loader. All 500 states are in DROID, so the policy may have trained on these frames, and the view invariance may be partly memorised.

Insights. Compare a policy with its own sampling noise: an error of 0.185 alone looks large, but it is 1.17× the noise. In these offline probes, a wrong instruction moves the predicted actions less than the sampling noise in most states; closed-loop language following is a separate question.

What I would test next

Hypothesis. The view invariance holds for camera poses outside DROID, and the language effect rises when two tasks are possible in one scene.

1. Views: new exterior camera poses on a DROID setup, then the same swap probe.
2. Language: pairs of feasible instructions in one scene, such as putting the cup into the bowl or taking it out.
3. Measure: the same ratios to the noise floor, then success in closed loop.

Sources. DROID (arXiv:2403.12945), Sec. III–V and Appendix F; π0.5 (arXiv:2504.16054); knowledge insulation (arXiv:2505.23705); openpi (Physical-Intelligence/openpi, commit 215abfb): the pi05_droid checkpoint, its README and the DROID training notes, read 1 November 2026; droid_100 on gs://gresearch/robotics. Run on 1 November 2026.