almond_axol.lerobot

LeRobot-compatible wrappers for the Axol hardware. They need the lerobot extra. These classes implement the LeRobot Robot, Teleoperator and Camera interfaces, so Axol works with any LeRobot training or data-collection pipeline. The collect-data and run-policy commands are built on them.

Treat policy checkpoints and LeRobot datasets as trusted input. LeRobot reads dataset metadata and loads model files without a sandbox. Only load checkpoints and datasets you created or got from a source you trust, especially on a computer connected to the robot.

from almond_axol.lerobot.robot import AxolRobot, AxolRobotConfig
from almond_axol.lerobot.teleop import AxolVRTeleop, AxolVRTeleopConfig
from almond_axol.lerobot.camera import ZedCamera, ZedCameraConfig

Use with the stock LeRobot CLIs

The lerobot_robot_axol plugin on PyPI registers these wrappers with LeRobot, so the stock LeRobot CLIs (lerobot-teleoperate, lerobot-record, lerobot-replay and others) can drive Axol. Installing the plugin also installs almond-axol[lerobot]. Then select the devices by type:

pip install lerobot_robot_axol

lerobot-teleoperate --robot.type=axol --teleop.type=axol_vr

lerobot-record \
    --robot.type=axol \
    --robot.cameras "{overhead: {type: zed, serial: 41234567, stereo: true}}" \
    --teleop.type=axol_vr \
    --dataset.repo_id=${HF_USER}/my-dataset \
    --dataset.num_episodes=5

Kind	`type`	Class
Robot	`axol`	`AxolRobot`
Teleoperator	`axol_vr`	`AxolVRTeleop`
Camera	`zed`	`ZedCamera`

Teleoperation still uses the VR headset: open the VR app and connect to the robot’s computer. For the best data collection performance, use collect-data and run-policy instead. They add ZED capture in a separate process, NVENC dataset encoding on the Jetson, and video streaming to the headset.

The plugin needs LeRobot 0.6.1, Linux and Python 3.12 or later on the robot’s computer. The hosted installer includes Python 3.13. The zed camera also needs pyzed (zed.install). To add the plugin to an axol environment from the one-command installer, see the plugin README.

If you install the plugin from PyPI into the hosted Axol environment, Axol updates from the installer or the control panel keep the same plugin version. If you installed it any other way, such as from a local path or a Git URL, the hosted update stops before it changes anything. Update that environment by hand, from the same source.

`AxolRobot`

LeRobot Robot that wraps the async Axol hardware driver. A background thread runs an asyncio event loop, so motor telemetry keeps streaming while the synchronous get_observation() and send_action() calls block the calling thread.

from almond_axol.lerobot.robot import AxolRobot, AxolRobotConfig
from almond_axol.lerobot.camera import ZedCameraConfig

config = AxolRobotConfig(
    cameras={
        "overhead":  ZedCameraConfig(serial=41234567),
        "left_arm":  ZedCameraConfig(serial=41234568),
        "right_arm": ZedCameraConfig(serial=41234569),
    },
)
with AxolRobot(config) as robot:
    obs = robot.get_observation()           # joints + camera frames
    joint_obs = robot.get_joint_observation()  # joints only, for tight control loops
    robot.send_action(obs)                  # hold current position

`AxolRobotConfig` fields

Field	Default	Description
`cameras`	`{}`	`ZedCameraConfig` instances keyed by name. Each opens a local ZED camera by serial number. A stereo camera can expose one or both eyes (see `ZedStereoCamera`).
`axol_config`	`AxolConfig()`	Per-joint gains and safety parameters passed to the hardware driver.
`observe_torques`	`False`	Include joint torques in `observation.state`.
`observe_cartesian`	`False`	Use each arm’s 6-axis end-effector pose and gripper instead of 16 joint positions. Observations come from FK, and `send_action()` resolves poses to joint targets with IK.
`left_channel`	`"can_alm_axol_l"`	SocketCAN interface for the left arm.
`right_channel`	`"can_alm_axol_r"`	SocketCAN interface for the right arm.
`video_backend`	`"auto"`	Camera capture backend: `"gst"` (GPU pipeline, low latency), `"sdk"` (ZED Python SDK), or `"auto"` (`gst` when installed, otherwise the SDK). Use the same backend for collection and inference, because their frames look different.

Key methods

Method	Description
`get_observation()`	Returns new camera frames with the nearest robot state, plus torques if enabled. See Frame and state timing.
`get_observation_with_capture_timestamp()`	Returns the same observation and its median camera exposure time on the `perf_counter` clock.
`get_joint_observation()`	Returns joint positions only, with no camera reads. Use it in fast teleop loops.
`send_action(action)`	Sends joint position targets, or end-effector poses with `observe_cartesian`. The arms use impedance control and the grippers use position-force control.
`positions`	`(left, right)` arm positions from the latest telemetry, each shape `(8,)`.

Frame and state timing

get_observation() picks the camera frames first, then the robot state that matches them. It never waits for a new exposure:

It finds the newest exposure time that every camera has already delivered.
For each camera, it takes the stored frame nearest that time. Shared-memory readers keep a short history of frames for this (latest_capture_ts(), read_nearest(target, tolerance_s)). An in-process reader gives its newest frame.
From the realtime core’s 240 Hz state history, it takes the joint state nearest the frames’ median exposure time. FK, if enabled, runs after this.

It never returns the same frame set twice, so a loop faster than the cameras waits at most one frame period. It raises an error instead of returning mismatched data when:

a camera has gone silent (its newest frame is a couple of frame periods old),
no state samples fall on both sides of the frame time,
the cameras and the state are too far apart in time.

`AxolVRTeleop`

LeRobot Teleoperator that wraps VRTeleop. It runs the VR WebSocket server and the IK process on a background thread, so get_action() doesn’t block and is safe to call from any thread.

from almond_axol.lerobot.teleop import AxolVRTeleop, AxolVRTeleopConfig

teleop = AxolVRTeleop(AxolVRTeleopConfig())
pos_l, pos_r = robot.positions
teleop.connect(q_start_left=pos_l, q_start_right=pos_r)

while True:
    action = teleop.get_action()
    events = teleop.get_teleop_events()

`AxolVRTeleopConfig` fields

Field	Default	Description
`vr_teleop_config`	`VRTeleopConfig()`	Rest poses, IK rate and filter settings. See `almond_axol.teleop`.
`kinematics_config`	`KinematicsConfig()`	IK solver weights. See `almond_axol.kinematics`.
`vr_server_config`	`VRServerConfig()`	WSS port and TLS certificate paths. See `almond_axol.vr`.

Key methods

Method	Description
`get_action()`	Returns the latest smoothed joint positions as a LeRobot `RobotAction` dict.
`get_teleop_events()`	Returns and clears pending episode events (`start_recording`, `TERMINATE_EPISODE`, `RERECORD_EPISODE`).
`request_reset()`	Starts a collision-aware move back to the rest pose.
`is_resetting`	`True` while the reset move is pending or running.
`send_feedback_state(state)`	Sends a `VRState` override, such as `SAVING`, to all connected headsets.

`ZedCamera`

LeRobot Camera for a ZED camera attached to this computer. It opens the camera by serial number through the ZED SDK and grabs frames on a background thread. It needs pyzed (zed.install).

from almond_axol.lerobot.camera import ZedCamera, ZedCameraConfig

cam = ZedCamera(ZedCameraConfig(serial=41234567))
cam.connect()
frame = cam.read_latest()   # newest frame, shape (H, W, 3), non-blocking
cam.disconnect()

Each frame carries two timestamps on the local perf_counter clock:

capture_perf_ts: when the sensor exposed the frame (from TIME_REFERENCE.IMAGE).
receive_perf_ts: when this process retrieved it.

On connect(), the camera measures the gap between the two and logs a warning if the average is outside the expected range.

Method	Description
`read_latest(max_age_ms=500)`	Newest frame, without blocking. Raises `TimeoutError` if it is older than `max_age_ms`.
`read_latest_with_ts()`	`(frame, capture_perf_ts, receive_perf_ts)` for the newest frame.
`read_at_or_after(target_capture_perf_ts, timeout_ms=500)`	Blocks until a frame with `capture_perf_ts >= target` is available.

`ZedStereoCamera`

Wrapper for a stereo ZED X. It opens one SDK connection and gets both eyes from each grab. The eyes are exposed as left_view and right_view, which have the same read methods as ZedCamera, so the rest of the pipeline treats them as ordinary cameras.

from almond_axol.lerobot.camera import ZedStereoCamera, ZedCameraConfig

stereo = ZedStereoCamera(ZedCameraConfig(serial=41234569, stereo=True))
stereo.connect()
left = stereo.left_view.read_latest()    # (H, W, 3), left eye
right = stereo.right_view.read_latest()  # (H, W, 3), right eye
stereo.disconnect()

When an AxolRobot camera has stereo=True, its eyes setting decides the observation keys (see AxolRobotConfig.observation_cameras()):

"both": camera X becomes X_left and X_right, and the dataset records both eyes. This is the default for the overhead camera.
"left" or "right": that eye only, under the plain name X, so it looks like a mono camera. This is the default for wrist cameras.

You rarely set stereo or eyes by hand. collect-data and run-policy call AxolRobotConfig.apply_detected_stereo(), which detects a stereo ZED X from its serial and sets eyes to "both" for the overhead camera and "left" for the others. stream_eyes sets which eyes go to the headset, separately from what is recorded. For example, stream "both" for depth during teleop while recording only "left".

`ZedCameraConfig` fields

Field	Default	Description
`serial`	`0`	Serial number of the camera to open. `0` means unassigned, and `collect-data` and `run-policy` skip it, so set only the cameras you have (at least one). List connected cameras with `list_zed_devices()`.
`fps`	`60`	Capture frame rate. `None` uses the camera’s default.
`width`	`960`	Frame width. With `height`, it must name a supported ZED resolution: `SVGA` 960×600, `HD1080` 1920×1080 or `HD1200` 1920×1200. `None` detects it on connect.
`height`	`600`	Frame height (SVGA).
`color_mode`	`RGB`	Output color channel order.
`warmup_s`	`1`	Seconds to read frames during `connect()` before returning.
`stereo`	`False`	Open the camera as a stereo ZED X, with both eyes from one grab. Detected from the serial at startup.
`eyes`	`"both"`	Which eyes of a stereo camera to record: `"both"`, `"left"` or `"right"`. See `ZedStereoCamera`. Ignored when `stereo=False`.
`stream_eyes`	`None`	Which eyes of a stereo camera to stream to the headset. `None` follows `eyes`. Used by `collect-data` and `teleop` only, since `run-policy` streams no video.
`record`	`True`	Whether to record this camera to the dataset. With `False` it can still stream.
`stream`	`True`	Whether to stream this camera to the headset. With `False` it can still be recorded.

Get Started

Operations

Mantis

Remote Teleop

Web Interfaces

Advanced

almond_axol.lerobot

Use with the stock LeRobot CLIs

`AxolRobot`

`AxolRobotConfig` fields

Key methods

Frame and state timing

`AxolVRTeleop`

`AxolVRTeleopConfig` fields

Key methods

`ZedCamera`

`ZedStereoCamera`

`ZedCameraConfig` fields

​Use with the stock LeRobot CLIs

​AxolRobot

​AxolRobotConfig fields

​Key methods

​Frame and state timing

​AxolVRTeleop

​AxolVRTeleopConfig fields

​Key methods

​ZedCamera

​ZedStereoCamera

​ZedCameraConfig fields

Use with the stock LeRobot CLIs

`AxolRobot`

`AxolRobotConfig` fields

Key methods

Frame and state timing

`AxolVRTeleop`

`AxolVRTeleopConfig` fields

Key methods

`ZedCamera`

`ZedStereoCamera`

`ZedCameraConfig` fields