Action chunking changes what a policy predicts at each inference step: instead of outputting the single next action, it outputs a short sequence — a chunk — of the next several actions at once. It sounds like a minor architectural choice, but it's one of the more consequential design decisions in current imitation-learning policies, because it directly addresses the failure mode that single-step behavior cloning is most prone to.
The compounding-error problem it addresses
A policy trained with behavior cloning predicts an action conditioned on the current observation. At deployment, each predicted action is slightly off from what a demonstrator would have done, which shifts the next observation slightly out of the training distribution, which produces a slightly worse prediction, and so on — small errors compound over a long horizon into a trajectory the policy was never trained to recover from. This is the classic distributional-shift problem in imitation learning, and it gets worse the longer the effective task horizon is.
Predicting a chunk of k actions per inference step reduces the number of independent decision points across an episode by roughly a factor of k. Fewer decision points means fewer chances for error to compound between them, which is the core mechanism behind chunking's effect on task success rate — not a smoothing trick so much as a reduction in how many times drift gets to accumulate.
Handling pauses and multimodal demonstrations
Single-step prediction has a second, subtler failure mode: when a task has a natural pause (waiting at a contact point, holding a grasp before transport) or the demonstration data contains multiple valid ways to proceed from the same state, a model trained to predict one action at a time tends to average across those modes — producing a blurry, indecisive action that satisfies none of them well. Predicting a whole chunk gives the model more context to commit to one coherent continuation rather than averaging step by step, and it's a large part of why chunk-based policies handle contact-rich, multi-stage tasks noticeably better than their single-step counterparts.
Temporal ensembling
Chunking alone would make execution jerky if the policy queried a new chunk and discarded the old one wholesale at chunk boundaries. Temporal ensembling fixes this: because a new chunk is predicted at every timestep (not just at chunk boundaries), multiple overlapping chunks end up making predictions for the same future timestep. Rather than picking one, the executed action is a weighted average across all the chunks that covered that timestep, with exponentially decaying weights so more recent predictions dominate. The result is a smoother trajectory than either committing to a stale chunk for its full length or replanning from scratch with no memory of the previous plan.
ACT and ALOHA
Action chunking as a technique for real-robot policies was introduced in Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware (Zhao, Kumar, Levine, and Finn; RSS 2023; arXiv:2304.13705), which paired the method — Action Chunking with Transformers, or ACT — with ALOHA, a low-cost open-source bimanual teleoperation rig used to collect the paper's demonstrations. ACT trains a conditional VAE whose decoder is a transformer: it takes in multi-camera images and joint state and outputs an action chunk, with the VAE's latent capturing the stylistic variation across demonstrations of the same task. The paper reported that fine manipulation tasks — opening a condiment cup, slotting a battery — reached high success rates from a small number of minutes of teleoperated demonstration, a result that's frequently cited as evidence for how much chunking (versus architecture alone) drives sample efficiency in imitation learning. ACT has since become one of the standard baseline policies in the LeRobot ecosystem, alongside diffusion policies and their descendants.
Chunk-size trade-offs
Chunk size is a real hyperparameter with a real trade-off, not a default to copy blindly:
- Fewer independent decision points, less compounding error over a fixed horizon
- More context per prediction, better handling of multimodal demonstrations
- Fewer inference calls per episode, lower compute at execution time
- More reactive to observations that change mid-chunk (a moved object, a new obstacle)
- Faster to adapt when a human intervenes or the scene deviates from training
- Less exposed to error if a single bad chunk is predicted, since it covers less time
The right size depends on task dynamics — a fast-changing contact task tolerates less commitment per chunk than a slower reach-and-place sequence — and is worth sweeping per task rather than inheriting from a different project.
What it demands of recorded data
Chunking's benefits assume the demonstrations it trains on support the assumption baked into the method: that a chunk of k consecutive actions is a coherent, learnable unit. That requires a consistent control rate across the dataset — action chunking over a stream sampled at drifting or mixed frequencies teaches the model horizons that don't correspond to a fixed amount of real time — and dense, gap-free action logging, since a chunk with dropped or interpolated steps hides exactly the fine-grained detail chunking is meant to exploit. Datasets collected with inconsistent control loops or sparse action sampling lose much of the benefit chunking is supposed to provide, regardless of how the policy is architected on top of them.
Two data-collection details matter more for chunked policies than for single-step ones specifically. First, the action stream needs to reflect what was actually executed, sampled at the same rate as the rest of the pipeline — if the action log is downsampled relative to the camera and proprioception streams, chunks end up spanning an inconsistent amount of wall-clock time across the dataset, which undermines the fixed-horizon assumption the method relies on. Second, episode boundaries matter more too: a chunk that spans an episode boundary — the tail of one attempt bleeding into the start of the next — teaches the model a transition that never should have existed as a training example, which is one more reason episode boundaries are worth marking cleanly at capture time rather than reconstructing later.
Beyond ACT: where chunking shows up now
Action chunking has outgrown its original transformer-decoder implementation. Diffusion-policy methods, which generate an action chunk by iteratively denoising from noise rather than by direct transformer regression, adopted the same chunk-and-ensemble structure and are now a common alternative baseline alongside ACT in the LeRobot policy zoo. Vision-language-action models built on top of large pretrained backbones typically chunk their action output as well, for the same compounding-error reasons that motivated the original method — chunking has become closer to a default assumption in current robot-policy architectures than a technique specific to any one paper.