Skip to main content
Runs a trained policy on the robot while the operator watches in VR and steps in to correct it, recording the whole session — policy segments and human corrections — to a LeRobot-format dataset. It’s the DAgger loop: gather demonstrations exactly where the policy struggles, so a retrain fixes those failures. Inference runs in-process (one action per control tick), and everything else — cameras, recording, the guarded return-to-rest — follows collect-data. Loops until Ctrl+C. The VR grip (“side”) buttons drive the intervention during an episode: while the policy is driving, either grip alone freezes it (the robot holds pose and recording pauses); both grips hand control to the operator (teleop engages at the robot’s current pose and recording resumes); either grip alone again hands it back to the policy. Nothing is recorded during the frozen gap, so the dataset flows straight from the policy’s motion into the correction. Rows recorded while the operator drives are tagged with LeRobot’s per-frame intervention flag. Episode boundaries work like collect-data: press the record button to start an episode (the policy starts driving and recording begins), press it again to save, or press it with the reset button held to discard and re-record. On stdin, s saves, r discards, and q discards and quits; --episode_time_s is a safety cap that saves the episode if nothing is pressed. With --subtasks supplied, typing a number 1..N switches the policy’s instruction mid-episode without ending it.
For the end-to-end workflow — from a trained checkpoint to recorded corrections — see the DAgger Collection guide.
This command is configured via draccus. The policy side mirrors run-policy (--policy_path / --policy_type / --device, but inference is in-process and synchronous), and the recording side mirrors collect-data (the nested robot_config / teleop_config configs — cameras, per-joint gains, IK, VR server). Nest into them with dots or pass a whole-config file. See Command configuration.

​
Intervention flow

The grip buttons move control between the policy and the operator during an episode. A takeover engages teleop at the robot’s measured pose (the IK worker and smoothing filters are synced to it first, so engaging never drags the arm toward a stale target), and the grippers adopt the controller triggers immediately — pre-set the triggers before you squeeze the grips (hold a trigger down so a held part stays gripped). When you hand back, the policy backend is reset so it re-plans from the corrected pose instead of continuing from its pre-freeze state. Between episodes the grips work as plain teleop (both to engage, one to disengage — nothing is recorded) so you can reset the scene with the arms, and the VR reset button homes them on demand. The VR reset button and the startup trajectory are disabled inside an episode: episodes end from the record button, and homing is the collision-aware IKResetController’s job. If the headset link goes stale mid-intervention, tracking auto-disengages and control hands back to the policy.

​
DAgger annotation

Every dataset collect-dagger creates declares LeRobot’s native per-frame intervention feature — a bool tagged True on rows recorded while the operator was driving and False on the policy’s own rows — the same annotation LeRobot’s DAgger rollout strategy uses, so downstream training can weight or filter the corrections. The frozen gap between a freeze and a takeover is not recorded, so an episode plays straight through from policy motion into the correction and back; because LeRobot timestamps are index-based, the capture clock re-anchors on resume. Resuming an existing dataset requires it to already carry the intervention feature (one recorded by this command); a dataset created before the feature existed is rejected rather than recorded into untagged.
The camera and recording plumbing is collect-data’s out-of-process split, with one twist: the video relay owns the ZED cameras and streams the headset view, and its raw branch is forced onto the shared-memory (pyshm) transport so the frames are readable both here (the policy builds its observations from them) and by the dataset-recorder subprocess. The relay is required — there is no in-process camera fallback, since per-frame camera work in the control process would starve the policy and teleop loops. Install the GStreamer stack (gst.install + gst.build-zed) if the relay can’t start.