Skip to main content
Records teleoperation episodes from the VR controllers and the ZED cameras attached to the robot, and saves them to a LeRobot-format dataset. It runs until you press Ctrl+C.
For the end-to-end workflow, see the Data Collection guide.
This command is configured through draccus. The robot and teleop settings live in the nested robot_config and teleop_config configs: cameras, per-joint gains, IK and the VR server. Set them with dotted flags or a whole-config file (see Command configuration). The table below lists the common flags. Every nested field is listed with its default in Data Collection Config Parameters. On the Jetson, h264 is encoded in hardware at a capped bitrate (about 3 Mbps at SVGA and 30 fps). Every camera’s video stays about the same size, even from a noisy sensor.

​
Stereo cameras

Stereo ZED X cameras are detected from their serial number, both on the CLI and in the control panel:
  • A stereo overhead camera records both eyes, as overhead_left and overhead_right.
  • A stereo wrist camera (left_arm or right_arm) records its left eye under the slot name, like a mono camera.
Setting stereo: true by hand turns off detection and uses eyes instead. The default both records X_left and X_right. left or right records one eye under the slot name X. To force a single-eye wrist camera, pass eyes: left too. To stream both eyes to the headset for depth while recording only one, set eyes and stream_eyes separately:

​
Headset camera views

During collection the camera feeds also stream to the headset, as in teleop. The operator sees the overhead and both wrist views at once and can move and resize them. See the VR Interface guide. Each camera is captured once and feeds two outputs:
  • The headset gets full resolution at 30 fps.
  • The dataset gets --fps, scaled down to --dataset_resolution.
If the installed GStreamer camera plugin doesn’t support exposure timestamps, collection opens the cameras through the ZED SDK instead. The video runs in a separate process, so it doesn’t slow the control loop.

​
VR controller controls

Axol and Quest-source Mantis episodes are driven from the Quest controllers. The session is locked to data collection mode. For the full controller layout and step-by-step flow, see the Data Collection guide. With a Lighthouse or Ultimate Mantis source, the rig triggers control the takes:
  • Start: squeeze both triggers together and release them. The rigs engage at their current pose and recording starts.
  • Save: while recording, fully squeeze and release either trigger three times quickly. The saved take ends where the first squeeze began, so the clicks aren’t recorded.
  • Discard: squeeze four times quickly. A triple press waits a moment in case a fourth press follows.
A triple press while idle is ignored. Quest Mantis uses A instead. When you launch collection from the control panel, it also shows buttons to start, save and discard a take and to end a contact hold. You can run a session with the headset off. The controllers stay live, and whichever input arrives first ends the take.

​
After each episode

On Axol, the arms return to the rest pose after each episode. A torque watchdog guards the move:
  • If a joint pushes against something, such as a gripper caught on the scene or a person holding an arm, the move stops. The arms go limp in a gravity-compensation hold so you can guide them clear. The episode keeps saving in the background.
  • Press X when the arms are clear. They return to rest from where they are.
  • You can’t start recording while the arms are limp. Reset first.
With --teleop_config.vr_teleop_config.teleop_torque_threshold set, the watchdog also guards the arms while you drive them. A stop mid-take discards the episode and drops the arms limp. Press X to return to rest and record again. Mantis has no arms to return. It turns off both gripper motors between takes, and you must engage the rigs again before the next recording.

​
Recording with Mantis

--mantis true records with the Mantis rigs: tracked devices mounted on grippers, plus head and wrist cameras.
  • The grippers use the Mantis CAN channels saved in Mantis settings (default can_mantis_l and can_mantis_r).
  • The head stereo camera uses the overhead slot and can record both eyes along with both wrists.
  • At startup both grippers open fully once to calibrate, then release.
  • The gripper motors are off between takes. Starting a recording turns them on, and every save, discard, stop or error turns them off again.
  • Engage at the agreed start pose. That fixes the robot base for the session. Rows use the Cartesian end-effector schema.
  • Each row’s pose, gripper position and grip command are matched to the tracker’s capture time.
The pose source works like this:
  • Quest waits for the headset’s WebXR session and uses A for episode control.
  • Lighthouse and Ultimate start and stop their tracker readers automatically, from the CLI and the control panel. Don’t launch a second tracker.bridge.
  • Recording can’t start until the tracker sends two fresh poses and the triggers report in.

​
Starting and stopping

Starting needs the arms near zero. On Axol, collect-data refuses to start while any arm joint is more than 25° from zero, the arms’ natural resting pose. It names each joint that is out. Move them back by hand, for example under axol gravity-comp, and start again. The first move then goes through zero on its way to the rest pose. See Starting needs the arms near zero. Stopping parks the arms. When you end a session with Ctrl+C or Stop in the control panel:
  1. The arms return to the rest pose.
  2. With --teleop_config.vr_teleop_config.park_to_zero (the default), they continue to the all-zero pose, hanging straight down, holding their grasp.
  3. The motors turn off.
The return to rest has 6 s and the move to zero has 10 s. If a move runs out of time or hits something, the motors turn off where the arms are. A second Ctrl+C stops the park, and the motors still turn off. Set park_to_zero to false to stop at the rest pose. The park is skipped on Mantis, when a contact hold has left the arms limp, when the IK process has stopped, or when the CAN bus has stopped reporting positions.

​
Saving and resuming

Each episode is written to disk completely as soon as it saves. If the process is killed, you lose at most the episode being recorded, and Ctrl+C stops quickly. On the Jetson, an episode is discarded rather than saved if its camera timestamps are missing, frames are dropped, or the cameras and arm state are too far out of step. If a dataset already exists at --root, collection resumes it. On resume, collect-data checks that the saved episodes are complete and numbered without gaps. If an earlier session was killed while saving:
  • It cuts the dataset back to the last complete episode.
  • It deletes the leftover files and logs what it removed.
  • It resumes recording after the last complete episode.
If no complete episode survives, it refuses to start and asks you to delete the dataset. If a session ends before any episode is saved, the empty dataset folder is removed.