Because the data is plain LeRobot format, any LeRobot training tutorial applies directly — just substitute your own dataset
repo_id. The links below are the best places to start.Which policy to train
ACT (Action Chunking with Transformers) is the recommended starting point: it’s lightweight (~80M params), trains in a few hours on a single GPU, and often reaches high success rates with as few as ~50 demonstrations. Once you have a baseline working, you can explore larger vision-language-action models like SmolVLA.Train it
With thelerobot extra installed on a machine with a CUDA-capable GPU, train directly from your dataset’s Hub repo_id:
outputs/train/.../checkpoints, and (with --policy.repo_id) pushes the trained policy to the Hub so it’s reachable from the robot machine and any inference server. Expect a few hours for ~100k steps on a single GPU.
No local GPU? The ACT tutorial links a Google Colab notebook you can use to train in the cloud.
Mantis datasets: use axol mantis.train
Episodes recorded with the Mantis handheld rigs (axol collect-data --mantis true) store absolute base-frame end-effector poses in the Cartesian schema. Each take is anchored to wherever the demonstrator engaged, so a policy trained on absolute targets learns positions in that one world frame and does not transfer when the robot starts somewhere else.
axol mantis.train fixes that at training time. It wraps lerobot-train — same CLI surface (draccus dotted overrides, --config_path, wandb, resume), same checkpoint format — and installs a chunk-relative end-effector processor in the policy’s pipeline: every predicted action chunk is expressed relative to the observation it was predicted from, and the action/state normalisation statistics are recomputed over the relativised values. The relativisation lives in the checkpoint’s processor pipeline, so deployment is unchanged: axol run-policy --policy_path <ckpt> ... as usual.
- Local only. Remote HF Jobs (
--job.targetother thanlocal) are rejected: the remote pod runs stocklerobot-trainand cannot carry the processor patch. Run the command on the training machine. - Chunk execution. With relative actions, chunks predicted from different observations are anchored to different reference poses. The default chunk-queue execution in
run-policyis exact;--aggregate_fn temporal_ensembleblends near-identical absolute actions from overlapping chunks and works well in practice, but the blend is an approximation.
LeRobotTutorials
ACT
The recommended first policy — architecture, training, and tips.
Imitation learning walkthrough
End-to-end record → train → evaluate with
lerobot-train.SmolVLA
Fine-tune Hugging Face’s lightweight vision-language-action model.
LeRobot docs
Full documentation, models, and datasets.
Next steps
Run Policy
Deploy your trained checkpoint on the robot.
Data Collection
Record more episodes to improve the policy.
