Skip to main content
Teleoperation drives Axol’s arms live from a Meta Quest headset. You can drive the real robot or the browser sim. With Mantis selected, teleoperation drives only the two grippers from the rig triggers. Start it from the web control panel or with axol teleop.

​
Before you start

Sim needs no CAN setup or motor check.

​
Authorize the TLS certificate

The VR app connects to the robot over HTTPS with a self-signed certificate. Approve it once in each headset’s browser, or the connection fails without an error.
  1. In the VR app, press Connect.
  2. If the certificate isn’t approved yet, the connection fails and an Authorize certificate button appears. Tap it.
  3. Proceed past the warning in the popup. The app reconnects.
To approve it by hand instead, open https://<hostname>.local:8000 or https://<local-ip>:8000 in the headset browser and proceed past the warning.

​
Run it

1

Connect the host and robot

Open the control panel and connect the Axol Host. With Axol selected, the panel connects the robot and runs can.setup the first time. If the tile isn’t green, press Connect on it. For sim, skip the robot connection. See Connections.
2

Assign cameras (optional)

To see the camera feeds in the headset, open Axol settings → Cameras and assign the ZED cameras to the overhead, left arm and right arm slots. See Cameras.
3

Select Teleoperation and Start

Pick Teleoperation from the operation selector. Turn on sim to drive the browser visualizer instead of the robot. Press Start.
4

Connect the headset

Put on the Quest and open axol.almond.bot. Enter the host’s address, press Connect, then press Start. Press both grips together to engage the arms. See the controller layout.
From the CLI, on the robot’s computer:
To drive the sim, add --sim. This opens a visualizer at http://localhost:8002 and runs the same VR server on port 8000. Use it to check the network and tracking before you power the robot.
The terminal prints the hostname and IP address. In the headset, open axol.almond.bot, enter either address, press Connect, then press Start.
To show the ZED camera feeds in the headset, pass their serial numbers: --cameras "{overhead: 41234567, left_arm: 41234568, right_arm: 41234569}". You can move and resize each feed with the controllers. See Camera views.
See teleop for every flag, including stiffness, gripper limits, per-joint gains and config files.

​
Quest controller layout

These bindings are for Axol. For the Mantis gestures used in data collection, see Start gesture and recovery. Quest controller diagram

​
Engaging and freezing

  • From rest, press both grips together to start moving the arms.
  • After that, a click on one grip freezes or resumes that arm only. A frozen arm holds its pose and its grasp, which helps steady a grasped object.
  • A reset, a contact stop or a dropped link disengages both arms. Press both grips again to re-engage.
To move an arm only while you hold its grip, turn on Hold grips to engage (--teleop.hold_to_engage). Each grip then engages the arm on its side, from rest too, and releasing the grip freezes that arm. After a forced disengage, release any grip you were still holding before it engages again.

​
Safety behavior

​
Starting

Teleoperation, data collection, DAgger, run policy and replay refuse to start while any arm joint is more than 25° from zero. Zero is the pose the arms settle into when the motors are off. The error names each joint that is out of range. Move the arms back by hand, for example under axol gravity-comp, then start again. Once the check passes, the arms move to zero and then out to the rest pose. collect-dagger --start_from_current_pose skips this check unless --home_on_start is also set.

​
Returning to rest

On the real robot, every return to rest is guarded, including the X reset and the first move at startup. If an arm meets resistance, such as a held object or a person, the move stops.
  1. The arms go limp in a gravity-compensation hold.
  2. Guide them clear by hand.
  3. Press X to plan a new return from where they are.
The hold has no timeout. If you leave VR or the link drops, the arms stay limp until you put the headset back on and press X. Set the sensitivity with Return-to-rest contact stop in Axol settings → Robot (--teleop.reset_torque_threshold). 0 turns the check off. Sim has no torque feedback, so it has no guard.

​
Contact stop while driving

You can also stop the arms when they push hard while you drive them. This check is off by default, because teleoperation often pushes on the scene on purpose. Set Teleop contact stop in Axol settings → Robot (--teleop.teleop_torque_threshold). The panel suggests 16 Nm. When a joint pushes past the threshold for a moment, tracking disengages and the arms go limp in the same hold. Guide them clear, then press X to return to rest. This works on the real robot only. If pose updates stop while the arms are engaged, the arms freeze in place after about half a second. This happens when you take off the headset, open the system menu, quit the VR app or lose the network. The arms hold that pose until you come back. Re-entering VR doesn’t move the arms. Press both grips to re-engage, and tracking resumes from where the arms are. Press X to send them to the rest pose.

​
Stopping

When you press Stop in the control panel or Ctrl+C on the CLI, the arms return to the rest pose. With Park to zero on stop on (--teleop.park_to_zero, the default), they then move to zero and hang straight down, holding their grasp. Only then do the motors turn off, so the arms don’t drop.
  • Each move is guarded like a return to rest. The move to rest can take up to 6 s and the move to zero up to 10 s.
  • If a move times out or hits something, the motors turn off where the arms are.
  • Press Ctrl+C again to stop the park where it is.
  • Axol skips the park in --sim, while the arms are limp in a contact hold, and when the IK planner or the CAN feedback has failed.
Turn off Park to zero on stop to stop at the rest pose.

​
Driving Jelly

On Axol on Jelly, the thumbsticks drive Jelly’s base and lift. See Jelly. Axol detects Jelly from the CAN devices that are attached.

​
Network tip

If VR tracking feels jittery or arrives in bursts, set these on your router or access point: These settings stop the access point from holding packets between beacons, which causes latency spikes in VR.

​
Latency tuning

Teleoperation smooths the pose stream before it reaches the arms, so tracking glitches and network jitter don’t shake them. With the defaults, the arm lags your hand by about 100 to 120 ms on a clean link. Each setting below is a CLI flag or a config file field. If teleoperation feels too slow, change one setting at a time, in this order, and test each on the real robot. The same smoothing makes contact-heavy work such as deburring or insertion feel steady.
  1. --vr_server.interp_smooth_window_s 0.08 saves 20 ms and still removes most short glitches.
  2. --teleop.ik_alpha 0.4 saves a few ms with little loss of smoothness.
  3. --teleop.pose_cutoff 3.5 cuts filter delay but lets more hand tremor through. Watch for joint jitter.
  4. --vr_server.interp_smooth_window_s 0 turns the smoother off. Use it only over a USB connection with well-tracked controllers.
The network buffer only adds delay while the network is bursty. For network lag, a USB connection and the router settings help more than any setting here.If the headset’s Wi-Fi stalls and then delivers a backlog, the arms don’t replay the late motion. They hold, then move to your current pose when on-time frames resume. A frame counts as late when it arrives more than --vr_server.interp_max_lag_s behind (default 0.5 s, 0 turns this off). The log shows VR pose stream arriving … late when a stall starts and VR pose stream on time again after … when it ends.

​
Next steps

Config Parameters

Every setting for teleop, with its default.

Remote Teleop

Drive the robot over the internet from anywhere with Tailscale.

Data Collection

Record teleoperation episodes to a LeRobot dataset.