FAST read-along

FAST: Efficient Action Tokenization for Vision-Language-Action Models. Pertsch, Stachowicz, Ichter, Driess, Nair, Vuong and others, arXiv 2025.

Paper: FAST, arXiv:2501.09747 read-along Video 1:21:35 Watch on YouTube Sketchnote (PDF)

In short

Summary

FAST turns robot action chunks into tokens for autoregressive vision-language-action models: a discrete cosine transform on each action dimension, rounding, then byte-pair encoding.

Per-timestep binning makes consecutive tokens nearly redundant at high control rates, and policies trained with it make no progress on 20 Hz table bussing or 50 Hz T-shirt folding; FAST needs up to 13.2 times fewer tokens (53 instead of 700 for one second of shirt folding) and trains effective policies on both.

With the universal FAST+ tokenizer and 10k hours of robot data, π0-FAST matches the diffusion π0 model with 5 times fewer GPU hours of training, but at inference it needs about 750 ms per one-second chunk, while diffusion π0 typically needs 100 ms or less on an NVIDIA 4090.

Sketchnote

One-page summary

Hand-drawn summary of FAST: FAST turns robot action chunks into tokens for autoregressive vision-language-action models: a discrete cosine transform on each action dimension, rounding, then byte-pair encoding. Per-timestep binning makes consecutive tokens nearly redundant at high control rates, and policies trained with it make no progress on 20 Hz table bussing or 50 Hz T-shirt folding; FAST needs up to 13.2 times fewer tokens (53 instead of 700 for one second of shirt folding) and trains effective policies on both. With the universal FAST+ tokenizer and 10k hours of robot data, π0-FAST matches the diffusion π0 model with 5 times fewer GPU hours of training, but at inference it needs about 750 ms per one-second chunk, while diffusion π0 typically needs 100 ms or less on an NVIDIA 4090.
A one-page summary in sketchnote style. Download the PDF (page 1 the sketchnote, page 2 the listening sheet) to annotate it in Concepts or GoodNotes.

Listen for

In class

Keep these in mind while the presenter talks. Tick them on page 2 of the PDF as they come up.

  1. Why per-timestep binning breaks as the control frequency rises (the spline toy task)
  2. Which steps of FAST lose information (rounding) and which do not (DCT, BPE)
  3. Which results use per-dataset FAST and which use the universal FAST+
  4. Whether the gain holds on a second backbone: the OpenVLA ablation
  5. What the 5× measures, and what becomes slower

Questions worth asking

  1. π0-FAST needs about 750 ms per 1-second chunk. Which tasks would break first, and would LLM inference methods like speculative decoding be enough?
  2. On DROID the autoregressive model follows instructions better than diffusion π0. Is that the decoder, the full backbone, or something else?
  3. FAST+ was tested on hands and humanoids only for compression. What policy experiment would show that it transfers to a new robot?

In class, on an iPad

  1. Before: watch or listen to the read-along (the chapters are below) and skim the questions below.
  2. Open the sketchnote PDF in Concepts or another note app. Page 1 is the summary drawing, with room to annotate. Page 2 is the listening sheet.
  3. During the talk: tick each "listen for" item as it comes up, and write one line in each box of the sheet.
  4. After: mark the one answer below that the talk changed for you, and ask your question.

Common questions and misunderstandings

FAQ

What does "13.2× compression" mean?

It is a ratio of token counts per 1-second action chunk. For T-shirt folding (14 action dimensions at 50 Hz), per-timestep binning needs 700 tokens and FAST needs 53, at comparable reconstruction error. The text says FAST gives roughly 30 tokens per arm per chunk, but Table I prints averages of 20, 29 and 28 for BridgeV2, DROID and bussing, and 53 for the two arms of shirt folding. Across these four datasets the ratio goes from 1.75× (BridgeV2, 5 Hz) to 13.2× (shirt folding, 50 Hz). (Sec. VI-B, Table I)

Is FAST lossless?

No. The DCT is invertible and the BPE step is lossless, but the scale-and-round step drops small DCT coefficients. The rounding scale γ trades reconstruction error against token count; the paper uses 10 for all single-dataset tokenizers. Appendix B plots this trade-off for FAST, binning and FSQ. (Sec. V-B, Sec. VI-B, Appendix B (Fig. 12))

Does "5× faster" mean the policy runs 5 times faster?

No. It is about training: π0-FAST needed 5 times fewer GPU hours than the π0 model of Black et al., and the abstract says "up to 5x". Inference is slower: π0-FAST needs about 750 ms per 1-second chunk, and diffusion π0 typically needs 100 ms or less on an NVIDIA 4090. The cause is that π0-FAST decodes 30–60 tokens one by one with the full 2B-parameter backbone. (Sec. VI-E, Sec. VI-F, Fig. 1)

What is the difference between FAST and FAST+?

The BPE vocabulary, the only learned part. FAST trains it on each dataset, which takes a few minutes; FAST+ is one vocabulary trained on about 1M one-second action chunks from many robots, action spaces and control rates (the abstract calls them "trajectories"). In policy training FAST+ closely matches the per-dataset tokenizers. The OpenVLA ablation uses FAST+, and Sec. VII says π0-FAST does too. (Sec. V-C, Sec. VI-C, Sec. VI-D, Sec. VII, Fig. 6)

What is new in FAST, and what is borrowed?

Both building blocks are borrowed: the DCT is the transform behind JPEG, and BPE is the text tokenizer of language models. New are their combination on robot action chunks, the analysis of why per-timestep binning fails, and the universal FAST+ tokenizer. Putting the low frequencies of all dimensions first is a design choice: the authors found that it gives more stable rollouts. The paper argues that any sufficiently effective compression of the action targets should help, and picks DCT because it is simple and fast. (Sec. I, Sec. V-A, Sec. V-B)

Why does per-timestep binning fail at high frequency?

Because consecutive tokens become near copies of each other. A next-token model learns from the new information in each token, and for a smooth signal that information approaches zero as the time step shrinks. In the toy spline task, the binning model's error rises steeply as the sampling rate grows from 25 to 800 steps, until it copies the first action; with DCT tokens the error stays low at every rate. (Sec. IV, Fig. 3)

Does FAST replace diffusion or flow-matching action heads?

The paper leaves that open. FAST adds no new weights: action tokens overwrite the least-used tokens of the VLM vocabulary. Against π0 (a flow-matching model that the paper calls "diffusion"), π0-FAST is comparable on small single-task datasets, reaches high performance with 3 times fewer steps on table bussing and follows language better on DROID, but it decodes more slowly. The authors say the best VLA architecture is still open and leave DCT tokens with diffusion decoding to future work. (Sec. VI-A, Sec. VI-E, Sec. VII)

Is the BPE step needed?

Yes, it matters. Without BPE, policies on table bussing and T-shirt folding do worse, though they still beat naive binning. The paper explains that the DCT already puts most of the information in a few coefficients, but the many repeated zero tokens left without BPE dilute the learning signal and make the model decode hundreds of tokens. (Sec. VI-D)

Does a learned tokenizer (VQ or FSQ) beat FAST?

Not in this paper. FSQ, a simpler alternative to VQ-VAE, also compresses actions and trains much better than binning, but FAST is as good or better, especially on the dexterous high-frequency tasks, and needs no network training. In the compression sweep FSQ is more efficient at low fidelity, while FAST keeps improving at high fidelity, which fine-grained control needs. (Sec. VI-B, Fig. 6, Appendix B (Fig. 12))

Was FAST tested on humanoids, hands or new robots?

Only for compression. FAST+ cuts the token count by 2 times or more on 14 datasets it never saw, including dexterous hands, humanoids, UMI and Waymo driving data. All policies in the paper run on static manipulators, and FAST+ was trained on data that includes the real-robot evaluation tasks. Policy tests on other robot types are future work. (Sec. VI-A, Sec. VI-C, Fig. 8, Table III, Sec. VII)

Is the DROID policy really zero-shot?

Yes, as the paper defines it: trained on the 75k successful DROID episodes, then tested in unseen scenes with a new background, objects, viewpoint and table height, without co-training or fine-tuning. The scored suite has 44 trials graded by task progress; Appendix E says 16 tasks, but Table II lists 17. The tests on three university campuses are qualitative, with no success rates. (Sec. VI-A, Sec. VI-B, Appendix D, Appendix E, Table II, Fig. 7)

Are all generalist π0-FAST results zero-shot?

No. Sec. VI-F calls the comparison with diffusion π0 zero-shot, and Appendix E says grocery bagging and toast out of toaster were evaluated out of the box on the pretrained model. Laundry folding, as in Black et al., used a model fine-tuned on a small amount of high-quality task data. (Sec. VI-F, Appendix E, Fig. 11)

Chapters

Video

  1. 0:00Abstract
  2. 1:471. Introduction
  3. 8:252. Related work
  4. 13:453. Preliminaries
  5. 15:454. Case study: how does tokenization affect VLA training?
  6. 20:435. Efficient action tokenization via time-series compression
  7. 33:126. Experiments
  8. 1:03:147. Discussion and future work
  9. 1:06:19Appendix

Key terms

Glossary

Action chunk
A block of future actions predicted at once; FAST tokenizes 1-second chunks.
Control frequency
Actions per second in the data: 5 Hz for BridgeV2, 50 Hz for T-shirt folding.
Naive (binning) tokenization
Each action dimension at each time step becomes one of N bins (most often 256), one token each.
DCT
Discrete cosine transform: writes a signal as a sum of cosines; smooth signals need few coefficients (as in JPEG).
BPE
Byte-pair encoding: losslessly merges frequent symbol sequences into new tokens; per-dataset FAST uses a 1,024-token vocabulary.
Autoregressive VLA
A vision-language model fine-tuned to output action tokens one at a time by next-token prediction.
Diffusion VLA
Generates continuous actions by iterative denoising; π0 uses flow matching and a 300M-parameter action expert.
FSQ
Finite scalar quantization, a simpler alternative to VQ-VAE; the paper's learned-compression baseline.
Zero-shot (here)
Tested in a new scene with no co-training or fine-tuning, prompted only in language.