Teacher recoverability in DAgger for VLAs

DAgger assumes a good expert. Theorem 3.2 gives J(DAgger policy) ≤ J(expert) + u·T·ε + O(1), where T is the horizon and ε is the loss of the best policy in hindsight. The constant u bounds the extra cost of one learner action followed by the expert (Theorem 2.2). The bound is linear in T when u stays bounded, for example when the expert returns to its own state distribution within a few steps. The theorem also needs its learning assumptions (a no-regret learner, a strongly convex loss), which a neural network trained for three rounds does not meet. A VLA teacher can fail from the states that a student reaches, so its labels there can be wrong.

Two recent papers choose which states get labels. Kintsugi-VLA restores saved simulator states, runs a fixed privileged expert from them many times, and estimates the probability that the task still finishes. It uses this “interventional recoverability” to choose recovery starts for a SmolVLA policy. PIVOT-Q lets a quantized VLA act and has its full-precision version label only the states where their actions differ most; with 7.4% of the state-level labels it beats labeling all states. I found no study that compares the two selectors for a VLA teacher on the states of a student.

Hypotheses

1. From the student’s states, the teacher finishes the task less often than from matched start states of the same tasks.
2. At the same number of labels and, separately, at the same total cost, labels on recoverable states give higher student success than labels on high-disagreement states, on all states, or on random states.
3. The gain from on-policy labels over offline distillation grows with the student’s imitation error on teacher data. The analysis of Foster, Block and Misra motivates this: with the log loss and a realizable policy class, online imitation cannot improve on offline imitation without further assumptions.

Why it matters. If H1 holds, the labels of a VLA teacher need a check before use. If H2 holds, the check is worth its cost.

Measuring teacher recoverability

1. The student runs, and its state s is saved every 10 policy steps.
2. From each tested state, the teacher continues n times. The share that finishes the task is the recovery rate r(s).
3. From the same state, the student continues n times. This control measures how often the student finishes the task from s by itself.

Definitions. r(s) is the share of n teacher continuations from the restored state s that finish the original task within the remaining step budget of the episode, reported with a Wilson interval; n = 10 with fresh sampling seeds, adapted after the pilot. The restore includes the simulator state, the controller state, the elapsed steps and the policy’s action queue and history. Selection and evaluation use separate continuations. r(s) is a diagnostic related to u, not u itself: u bounds a difference in cost-to-go over all actions, and the test measures task completion after a takeover.

The student control matters. In a study of a frozen π0.5 on RoboTwin, 30% of the initially failed episodes over four tasks succeeded when the same policy restarted from the same initial scene. Saved-state continuations of the student apply the same lesson here.

Design

Compute (estimate). Each round tests 300 states with 10 teacher and 10 student continuations each, so three rounds need 18,000 continuations. With fine-tuning and evaluation, the plan needs about 200 GPU-hours on one RTX 5090. The pilot measures simulation, continuation and training speed first, and the final scope follows from it.

Risks. The teacher is strong on LIBERO, so few states may be unrecoverable; harder tasks and the LIBERO-Plus shifts may expose more. An inexact restore makes r(s) wrong, so the pilot checks that a restored state replays. PIVOT-Q already selects by disagreement, so the new part is recoverability with a VLA teacher and the comparison at equal cost.

The router collects recovery data in simulation. Each failure goes to the expert with the lowest measured cost that passes a recovery threshold in the test above. A person in VR is asked only after the automated experts fail the test. A failure that no expert recovers from is logged as unresolved and reset to an earlier state.

1. A failure state is saved. A policy expert gets the recoverability test. If it passes, it labels the state.
2. If not, a planner with full simulator state gets the same test. If it passes, its labels are used.
3. If both fail, a person in VR corrects the robot, or the failure is logged as unresolved.

Outputs. For simulated household tasks: recovery data for each failure type, human time only after the automated experts fail, and a table per task of which expert handles which failures and which stay unresolved.

Prior work and gaps. Kintsugi-VLA chooses recovery starts with one privileged expert. HG-DAgger collects a person’s takeover segments, and RaC collects a person’s rewind-and-correct interventions. The winner of the 2025 BEHAVIOR Challenge trained on successful demonstrations only, with no recovery demonstrations, and added hand-written correction rules.

Pilot (go/no-go). LIBERO-Long, the round-0 student and 300 saved states, stratified by task, time and episode outcome. The work goes on if a restored state replays exactly and r(s) varies enough between states to select on. A high r(s) everywhere is kept as a negative result for H1.

Router test. Two or three household tasks in simulation, with a policy expert, a motion planner with full simulator state and a person in VR. Policy-only, human-only and router collection run at the same total cost. The measures are student success, human minutes, and the share of failures that each expert recovers from.

Risk. A label from an expert that recovers is not always one that the student can learn: a planner with privileged state can act on information that the student does not see (ADVISOR). The test therefore measures student gains separately for policy, planner and human labels, and recovery is one filter, not proof of a good label.

Sources. DAgger (arXiv:1011.0686), Theorems 2.2 and 3.2 for the bound and the constant u. The teacher checkpoint and its LIBERO results come from the openpi LIBERO example (examples/libero/README.md), and the student from SmolVLA (arXiv:2506.01844). The RoboTwin re-run figure is pooled over four tasks in arXiv:2610.06921. I read the related work on arXiv on 21 and 22 October 2026, and the list can be incomplete. The compute figures are estimates, not measurements.