repo_id in place of the tutorial’s. The links at the end of this page are good places to start.
Which policy to train
Start with ACT (Action Chunking with Transformers). It is small (about 80M parameters), trains in a few hours on one GPU, and often works well with about 50 demonstrations. Once you have a working baseline, you can try larger vision-language-action models such as SmolVLA.Train it
Install thelerobot extra on a computer with a CUDA GPU. Then train from your dataset’s Hub repo_id:
outputs/train/.../checkpoints. With --policy.repo_id, the trained policy is also uploaded to the Hub, so the robot and any inference server can download it. Plan for a few hours for about 100k steps on one GPU.
No GPU? The ACT tutorial links a Google Colab notebook for training in the cloud.
Mantis datasets: use axol mantis.train
Episodes recorded with the Mantis rigs (axol collect-data --mantis true) store absolute gripper poses. Each take’s frame is set where the operator engaged. A policy trained on these absolute poses learns positions in that one frame, so it doesn’t carry over when the robot starts somewhere else.
axol mantis.train fixes this. It runs lerobot-train with the same flags and checkpoint format, and trains the policy to predict each chunk of actions relative to the pose it starts from. The checkpoint stores this setting, so you run it with axol run-policy --policy_path <ckpt> ... as usual.
- Train on your own computer.
axol mantis.trainrefuses remote HF Jobs (--job.targetother thanlocal). - Chunk blending is approximate. Each relative chunk is tied to the pose it was predicted from. Running each chunk from the queue as predicted is exact.
--aggregate_fn temporal_ensembleinrun-policyblends overlapping chunks, which works well in practice but is an approximation.
LeRobot tutorials
ACT
The recommended first policy: architecture, training and tips.
Imitation learning walkthrough
Record, train and evaluate with
lerobot-train.SmolVLA
Fine-tune Hugging Face’s small vision-language-action model.
LeRobot docs
Full documentation, models and datasets.
Next steps
Run Policy
Run your trained checkpoint on the robot.
Data Collection
Record more episodes to improve the policy.
