almond_axol.lerobot

LeRobot-compatible wrappers for the Axol hardware. Requires the lerobot extra. These classes implement the LeRobot Robot, Teleoperator, and Camera interfaces so the Axol works with any LeRobot training or data-collection pipeline without modification. The collect-data and run-policy CLI commands are built on top of this layer.

Treat policy checkpoints and existing LeRobot datasets as trusted input. The pinned upstream runtime parses dataset metadata and deserializes model files; it is not a sandbox for arbitrary Hub repositories or directories supplied by someone else. Only load artifacts you created or obtained from a source you trust, especially on a machine connected to robot hardware.

from almond_axol.lerobot.robot import AxolRobot, AxolRobotConfig
from almond_axol.lerobot.teleop import AxolVRTeleop, AxolVRTeleopConfig
from almond_axol.lerobot.camera import ZedCamera, ZedCameraConfig

Use with the stock LeRobot CLIs

almond-axol is published to PyPI, and the companion lerobot_robot_axol plugin registers these wrappers through LeRobot’s plugin auto-discovery — so the stock LeRobot CLIs (lerobot-teleoperate, lerobot-record, lerobot-replay, …) drive the Axol with no changes to LeRobot itself. Installing the plugin pulls in almond-axol[lerobot]; then select the devices by type:

pip install lerobot_robot_axol

lerobot-teleoperate --robot.type=axol --teleop.type=axol_vr

lerobot-record \
    --robot.type=axol \
    --robot.cameras "{overhead: {type: zed, serial: 41234567, stereo: true}}" \
    --teleop.type=axol_vr \
    --dataset.repo_id=${HF_USER}/my-dataset \
    --dataset.num_episodes=5

Kind	`type`	Class
Robot	`axol`	`AxolRobot`
Teleoperator	`axol_vr`	`AxolVRTeleop`
Camera	`zed`	`ZedCamera`

Teleoperation still uses the VR headset — open the VR app and connect to the robot machine. For the highest-rate data collection (out-of-process ZED capture, on-Jetson NVENC dataset encoding, headset video streaming), prefer the SDK’s own collect-data / run-policy, which are built on these same classes.

Requires the pinned LeRobot 0.6.1 release, Linux, and Python 3.12+ (the hosted installer bundles 3.13) on the robot machine; the zed camera additionally needs pyzed (zed.install). See the plugin README for adding it to an existing one-command-install axol environment.

When installed from PyPI into the hosted Axol environment, the plugin’s exact version is preserved automatically across installer/control-panel Axol updates. A direct, VCS, or otherwise customized plugin install instead blocks the hosted force update before mutation; update that deployment manually with an explicit plan for the same source.

`AxolRobot`

LeRobot Robot wrapping the async Axol hardware driver. A background thread runs a dedicated asyncio event loop so motor telemetry keeps streaming while the synchronous get_observation() and send_action() calls block on the calling thread.

from almond_axol.lerobot.robot import AxolRobot, AxolRobotConfig
from almond_axol.lerobot.camera import ZedCameraConfig

config = AxolRobotConfig(
    cameras={
        "overhead":  ZedCameraConfig(serial=41234567),
        "left_arm":  ZedCameraConfig(serial=41234568),
        "right_arm": ZedCameraConfig(serial=41234569),
    },
)
with AxolRobot(config) as robot:
    obs = robot.get_observation()           # joints + camera frames
    joint_obs = robot.get_joint_observation()  # joints only — use in tight control loops
    robot.send_action(obs)                  # hold current position

`AxolRobotConfig` fields

Field	Default	Description
`cameras`	`{}`	`ZedCameraConfig` instances keyed by name, each opening a locally attached ZED camera by serial number. A camera with `stereo=True` expands into two observation keys, `<name>_left` / `<name>_right`, backed by a single grab (see `ZedStereoCamera` below).
`axol_config`	`AxolConfig()`	Per-joint gains and safety parameters forwarded to the hardware driver
`telemetry_hz`	`120.0`	Background joint telemetry polling rate in Hz
`observe_torques`	`False`	Include joint torques in `observation.state`
`observe_cartesian`	`False`	Work in Cartesian end-effector space instead of 16 joint positions: observations report per-arm 6-axis EE poses (+ gripper) via FK, and `send_action()` accepts the same and resolves it to joint targets via IK
`left_channel`	`"can_alm_axol_l"`	SocketCAN interface for the left arm
`right_channel`	`"can_alm_axol_r"`	SocketCAN interface for the right arm
`video_backend`	`"auto"`	Camera capture backend: `"gst"` (GPU-resident zed-gstreamer pipeline, low latency), `"sdk"` (ZED Python SDK), or `"auto"` (prefer `gst` when its stack is installed, else fall back to the SDK). `run-policy` overrides this to `"sdk"` since it streams no headset video.

Key methods

Method	Description
`get_observation()`	Returns joint positions (+ torques if enabled) and latest camera frames
`get_joint_observation()`	Returns joint positions only — no camera reads; use in the high-frequency teleop path
`send_action(action)`	Sends joint position targets — or, with `observe_cartesian`, per-arm Cartesian EE poses resolved to joint targets via IK — via impedance control (arm) and position-force control (gripper)
`positions`	`(left, right)` cached arm positions from telemetry, each shape `(8,)`

`AxolVRTeleop`

LeRobot Teleoperator wrapping VRTeleop. Runs the VR WebSocket server and IK subprocess on a background thread so get_action() is non-blocking and safe to call from any thread.

from almond_axol.lerobot.teleop import AxolVRTeleop, AxolVRTeleopConfig

teleop = AxolVRTeleop(AxolVRTeleopConfig())
pos_l, pos_r = robot.positions
teleop.connect(q_start_left=pos_l, q_start_right=pos_r)

while True:
    action = teleop.get_action()
    events = teleop.get_teleop_events()

`AxolVRTeleopConfig` fields

Field	Default	Description
`vr_teleop_config`	`VRTeleopConfig()`	Rest poses, IK frequency, filter parameters — see `almond_axol.teleop`
`kinematics_config`	`KinematicsConfig()`	IK solver weights — see `almond_axol.kinematics`
`vr_server_config`	`VRServerConfig()`	WSS port and TLS certificate paths — see `almond_axol.vr`

Key methods

Method	Description
`get_action()`	Returns the latest smoothed joint positions as a LeRobot `RobotAction` dict
`get_teleop_events()`	Returns and clears latched episode-control events (`start_recording`, `TERMINATE_EPISODE`, `RERECORD_EPISODE`)
`request_reset()`	Triggers a collision-aware trajectory back to the rest pose
`is_resetting`	`True` while the reset move is pending or in progress
`send_feedback_state(state)`	Broadcasts a `VRState` override (e.g. `SAVING`) to all connected VR headsets

`ZedCamera`

LeRobot Camera for a locally attached ZED camera. Opens the camera by serial number via the ZED SDK and grabs frames in a background thread. Requires pyzed (zed.install).

from almond_axol.lerobot.camera import ZedCamera, ZedCameraConfig

cam = ZedCamera(ZedCameraConfig(serial=41234567))
cam.connect()
frame = cam.read_latest()   # shape (H, W, 3), non-blocking, returns most recent frame
cam.disconnect()

Each grabbed frame carries two timestamps on the local perf_counter clock: capture_perf_ts (when the sensor exposed the frame, derived from TIME_REFERENCE.IMAGE) and receive_perf_ts (when this process retrieved it). Since capture is local, both live on the same clock — no cross-machine sync needed. The retrieve-vs-capture skew is sampled on connect() and a warning is logged if the mean falls outside the expected range.

Method	Description
`read_latest(max_age_ms=500)`	Most recent frame, non-blocking; raises `TimeoutError` if it is older than `max_age_ms`.
`read_latest_with_ts()`	`(frame, capture_perf_ts, receive_perf_ts)` for the most recent frame.
`read_at_or_after(target_capture_perf_ts, timeout_ms=500)`	Block until a frame with `capture_perf_ts >= target` is available. Used by `collect-data` to align every camera and the joint sample on the same capture timeline.

`ZedStereoCamera`

Wrapper for a stereo ZED X. Opens one SDK connection and, on every grab, retrieves both eyes — so the camera is grabbed once. The eyes are exposed as left_view / right_view, which present the same read API as ZedCamera so the rest of the pipeline can treat them as ordinary cameras.

from almond_axol.lerobot.camera import ZedStereoCamera, ZedCameraConfig

stereo = ZedStereoCamera(ZedCameraConfig(serial=41234569, stereo=True))
stereo.connect()
left = stereo.left_view.read_latest()    # (H, W, 3) — left eye
right = stereo.right_view.read_latest()  # (H, W, 3) — right eye
stereo.disconnect()

When an AxolRobot camera config has stereo=True, the observation keys depend on eyes (see AxolRobotConfig.observation_cameras()): "both" expands camera X into X_left / X_right (the head camera records both eyes in the dataset), while "left" / "right" exposes a single eye under the plain name X — indistinguishable downstream from a mono camera (the wrist convention). Stereo is auto-promoted from the serial by AxolRobotConfig.apply_detected_stereo() — collect-data / run-policy call it so a stereo ZED X opens correctly without setting stereo by hand.

`ZedCameraConfig` fields

Field	Default	Description
`serial`	`0`	Serial number of the camera to open. `0` is the “unassigned” sentinel: `collect-data` / `run-policy` seed all three slots and drop the ones left at `0` (`AxolRobotConfig.select_assigned_cameras()`), so you only assign the cameras you have (at least one). A real ZED serial is positive. List connected cameras with `list_zed_devices()`.
`fps`	`60`	Capture frame rate. `None` adopts the camera default on connect.
`width`	`960`	Frame width (SVGA). With `height`, must name a supported ZED resolution (`SVGA` 960×600, `HD1080` 1920×1080, `HD1200` 1920×1200). `None` auto-detects on connect.
`height`	`600`	Frame height (SVGA).
`color_mode`	`RGB`	Output color channel order.
`warmup_s`	`1`	Seconds to read frames during `connect()` before returning
`stereo`	`False`	Open the camera as a stereo ZED X carrying both eyes on one grab. Auto-detected from the serial at startup (`apply_detected_stereo()`), so it’s rarely set by hand.
`eyes`	`"both"`	Which eye(s) of a stereo camera to expose as recorded observations: `"both"` → `X_left` and `X_right` (the head convention); `"left"` / `"right"` → that single eye under the plain name `X`, so it looks like a mono camera downstream (the wrist convention). `apply_detected_stereo()` sets this to `"both"` for the overhead and `"left"` for any other slot. Ignored when `stereo=False`.
`stream_eyes`	`None`	Which eye(s) of a stereo camera to stream to the headset, independently of `eyes`. `None` follows `eyes` (the coupled default); set it to decouple the headset feed from the recording — e.g. stream `"both"` for depth in teleop while recording only `"left"`. Honored by the collect-data / teleop relay only (`run-policy` streams no video). Ignored when `stereo=False`.
`record`	`True`	Whether this camera is recorded to the dataset at all. `False` drops it from `observation_cameras()` while still allowing it to stream.
`stream`	`True`	Whether this camera is streamed to the headset at all. `False` drops it from the relay’s encoded feed while still allowing it to be recorded.

Get Started

Operations

Mantis

Remote Teleop

Web Interfaces

Advanced

almond_axol.lerobot

Use with the stock LeRobot CLIs

`AxolRobot`

`AxolRobotConfig` fields

Key methods

`AxolVRTeleop`

`AxolVRTeleopConfig` fields

Key methods

`ZedCamera`

`ZedStereoCamera`

`ZedCameraConfig` fields

​Use with the stock LeRobot CLIs

​AxolRobot

​AxolRobotConfig fields

​Key methods

​AxolVRTeleop

​AxolVRTeleopConfig fields

​Key methods

​ZedCamera

​ZedStereoCamera

​ZedCameraConfig fields

Use with the stock LeRobot CLIs

`AxolRobot`

`AxolRobotConfig` fields

Key methods

`AxolVRTeleop`

`AxolVRTeleopConfig` fields

Key methods

`ZedCamera`

`ZedStereoCamera`

`ZedCameraConfig` fields