VR Interface

The Axol VR interface is a WebXR app that runs in a Meta Quest browser and streams hand/elbow pose from the headset to the Axol SDK over a secure WebSocket. It’s what you connect to during teleoperation and data collection. The app’s source lives in this repository under web/ (it was previously the separate axol-vr repo). The same web/ build also produces the web control panel, and axol serve serves both.

Hosted vs. self-hosted

	URL	When to use
Hosted	axol.almond.bot	The easiest path — open it on the Quest, enter your host’s address, connect. Recommended for most users.
Self-hosted	`https://<host>:8001/`	Served from your own machine by `axol serve`, or by `npm run dev` for development. Use when iterating on the interface or running fully offline.

Either way, the headset connects to the VR WebSocket server on port 8000 that axol teleop (and the control panel’s teleop/collect operations) runs on the host.

On networks where Wi-Fi adds latency, tick Quest over USB on the connect screen to stream controller poses over a USB cable instead — the camera feeds keep using the LAN. See Quest over USB.

The WSS link is self-signed, so it needs a one-time approval in the Quest browser. When you press Connect and the certificate hasn’t been approved yet, the connection fails and an Authorize certificate button appears — tap it, proceed past the warning in the popup, and the app reconnects. (Equivalently, visit https://<hostname>.local:8000 or https://<local-ip>:8000 and proceed past the warning yourself.) The cert is cached in ~/.almond/vr/certs/ and shared with axol serve.

Repository layout

web/
├── app/                      # Vite + React WebXR app (and the control panel)
│   └── dist/                 # build output — served by `axol serve`
└── packages/
    └── axol-vr-client/       # reusable R3F components + hooks (@almond/axol-vr-client)

Build & develop

From web/:

npm install

# Production build — client package first, then the app (outputs web/app/dist,
# which axol serve picks up)
npm run build --workspace=packages/axol-vr-client
npm run build --workspace=app

# Local dev server with hot reload
npm run dev --workspace=app

For dev, open the printed localhost URL in the Quest browser, enter the hostname of the machine running the Axol SDK, and press Connect, then Start to enter the XR session.

Deployment

The hosted app is built on Vercel; vercel.json builds the client package first so it’s available as a local workspace dependency:

{
  "buildCommand": "npm run build --workspace=packages/axol-vr-client && npm run build --workspace=app",
  "outputDirectory": "app/dist",
  "installCommand": "rm -f package-lock.json && npm install"
}

Camera views

When the host has camera video enabled (axol teleop --cameras, collect-data, or the control panel with camera slots assigned), the robot’s camera feeds stream to the headset over WebRTC and render as screens over passthrough. All streamed feeds are shown at once — the overhead centered (rendered per-eye in true stereo when the overhead is a stereo ZED X) with the left and right wrist cams as bottom-corner picture-in-picture panes. There’s no view to switch between; you arrange the screens to taste instead. The screens are world-anchored — like TVs placed where you were looking when the session started, so you can move your head freely while they stay put. While arm tracking is disengaged you can rearrange them:

Move a screen — point a controller at it and hold the trigger, then move the controller to reposition it. Each screen remembers its own spot.
Resize a screen — grab the same screen with both triggers and move your hands apart to make it bigger or together to make it smaller.
Reset — click the right thumbstick to re-anchor every screen to your current gaze and clear all moves and resizes.

The headset only shows the feeds for cameras that are actually being streamed (set in the control panel’s Cameras settings tab or via the relevant CLI flags), so streaming only the overhead, say, shows just the overhead. While video is still negotiating — or any streamed camera hasn’t produced its first frame — a global Connecting cameras… indicator shows; it clears once every streamed camera is live, and never appears when nothing is being streamed.

Controller bindings

#	Button	Action
1	Left grip	Press both grips (1 + 2) together to enable arm tracking; press either alone to disable it (toggle, not hold)
2	Right grip	See above
3	Left trigger	Actuate left gripper. While tracking is disengaged: point at a camera screen and hold to move it — grabbing one screen with both triggers resizes it
4	Right trigger	Actuate right gripper. While tracking is disengaged: point at a camera screen and hold to move it — grabbing one screen with both triggers resizes it
5	X	Reset pose; cancels a recording countdown. While recording, arms the Discard episode? confirmation — press X again to discard and re-record, or A to cancel and keep recording
6	A	Record: start a take (3-second countdown). While recording, arms the Save episode? confirmation — press A again to save, or X to cancel and keep recording. Data collection only (no effect during plain teleop)
7	Y	Exit the XR session — sends a reset first, so the arms return to rest and disengage instead of holding the last pose and jerking on the next entry
—	Right thumbstick (click)	Reset the camera screens — re-anchor to the current gaze and clear moves + resizes

The session’s mode is fixed by the command that launched it — there’s no in-headset toggle between the two. axol teleop runs plain teleop (recording is unavailable, so A does nothing and the HUD stays on Teleop), while axol collect-data runs data collection (recording enabled, HUD starts on Data Collection).

State machine

In teleop mode the headset stays in Teleop with the recording controls inert. In data collection mode it starts in Data Collection and drives episodes with A / X:

DataCollection ──[A]──► (3s countdown) ──► Recording
      ▲                                          │
      │                             [A]=save · [X]=discard
      │                            (arms a confirm popup — press
      │                             the same button again to commit)
      │                                          │
      ├──────── Saving ◄──(server push)◄─── [A→A] save
      │                                          │
      └────────────────────────────────── [X→X] discard

During the countdown the state sent to the server stays DataCollection; it becomes Recording once the countdown completes. Stopping a recording is confirmation-gated: the first A (save) or X (discard) press while recording arms an in-headset Save episode? / Discard episode? popup instead of stopping immediately — pressing the same button confirms, the other cancels and keeps recording. Nothing is committed server-side until you confirm a save; confirming a discard drops the take (via the reset flag) so you can re-record.

During data collection the HUD also shows an Episode N readout (top-right, under the recording status) so you can see which episode you’re about to record. The server announces it on connect — a headset joining mid-session shows the right number immediately — and it’s hidden in plain teleop. See the VR server API for the episode feedback message.

Saving and Error are server-driven. When recording stops, the SDK pushes {"type": "state", "value": "saving"} and blocks every control except Y (exit) until the episode is written, then pushes data_collection to re-enable controls. Broadcasting error shows an error indicator in the headset. See almond_axol.vr for the frame protocol and send_feedback_state().

VR Interface

Hosted vs. self-hosted

Repository layout

Build & develop

Deployment

Camera views

Controller bindings

State machine

Next steps

Teleoperation

VR server API

​Hosted vs. self-hosted

​Repository layout

​Build & develop

​Deployment

​Camera views

​Controller bindings

​State machine

​Next steps

Teleoperation

VR server API

Hosted vs. self-hosted

Repository layout

Build & develop

Deployment

Camera views

Controller bindings

State machine

Next steps