Skip to main content
The Axol VR interface is a WebXR app that runs in a Meta Quest browser and streams hand/elbow pose from the headset to the Axol SDK over a secure WebSocket. It’s what you connect to during teleoperation and data collection. The app’s source lives in this repository under web/ (it was previously the separate axol-vr repo). The same web/ build also produces the web control panel, and axol serve serves both.

​
Hosted vs. self-hosted

URLWhen to use
Hostedaxol.almond.botThe easiest path — open it on the Quest, enter your host’s address, connect. Recommended for most users.
Self-hostedhttps://<host>:8001/Served from your own machine by axol serve, or by npm run dev for development. Use when iterating on the interface or running fully offline.
Either way, the headset connects to the VR WebSocket server on port 8000 that axol teleop (and the control panel’s teleop/collect operations) runs on the host.
The WSS link is self-signed, so it needs a one-time approval in the Quest browser. When you press Connect and the certificate hasn’t been approved yet, the connection fails and an Authorize certificate button appears — tap it, proceed past the warning in the popup, and the app reconnects. (Equivalently, visit https://<hostname>.local:8000 or https://<local-ip>:8000 and proceed past the warning yourself.) The cert is cached in ~/.almond/vr/certs/ and shared with axol serve.

​
Repository layout

web/
├── app/                      # Vite + React WebXR app (and the control panel)
│   └── dist/                 # build output — served by `axol serve`
└── packages/
    └── axol-vr-client/       # reusable R3F components + hooks (@almond/axol-vr-client)

​
Build & develop

From web/: 
npm install

# Production build — client package first, then the app (outputs web/app/dist,
# which axol serve picks up)
npm run build --workspace=packages/axol-vr-client
npm run build --workspace=app

# Local dev server with hot reload
npm run dev --workspace=app
For dev, open the printed localhost URL in the Quest browser, enter the hostname of the machine running the Axol SDK, and press Connect, then Start to enter the XR session.

​
Deployment

The hosted app is built on Vercel; vercel.json builds the client package first so it’s available as a local workspace dependency: 
{
  "buildCommand": "npm run build --workspace=packages/axol-vr-client && npm run build --workspace=app",
  "outputDirectory": "app/dist",
  "installCommand": "rm -f package-lock.json && npm install"
}

​
Camera views

When the host has camera video enabled (axol teleop --zed_host, collect-data, or the control panel with a connected ZED box), the robot’s camera feeds stream to the headset over WebRTC and render as screens over passthrough. The screens are world-anchored — like TVs placed where you were looking when the session started, so you can move your head freely while they stay put. While arm tracking is disengaged, point a controller at a screen and hold the trigger to grab and reposition it (positions are remembered per view). Click the right thumbstick to re-anchor everything to your current gaze and reset moved screens. Flick the right thumbstick to switch views (a latched toggle — no holding; flicking the active direction returns to the default):
FlickView
—Default — passthrough with the left/right wrist cams as bottom-corner picture-in-picture
UpFullscreen overhead (rendered per-eye in true stereo when the overhead is a stereo ZED X)
LeftFullscreen left wrist
RightFullscreen right wrist
DownSplit — left + right wrist cams side-by-side

​
Controller bindings

Quest controller diagram
#ButtonAction
1Left gripPress both grips (1 + 2) together to enable arm tracking; press either alone to disable it (toggle, not hold)
2Right gripSee above
3Left triggerActuate left gripper. While tracking is disengaged: point at a camera screen and hold to move it
4Right triggerActuate right gripper. While tracking is disengaged: point at a camera screen and hold to move it
5XReset pose; cancels a recording countdown; stops and discards an in-progress take
6ARecord: start a take (3-second countdown), or stop and save the current take
7YExit the XR session
8BToggle between Teleop and Data Collection (disabled while recording or counting down)
—Right thumbstick (flick)Switch camera view — see Camera views
—Right thumbstick (click)Reset the camera screens — re-anchor to the current gaze and clear moved positions

​
State machine

Teleop ──[B]──► DataCollection ──[A]──► (3s countdown) ──► Recording
   ▲                 ▲                                          │
   └────────[B]──────┘                                    [A or X]
                                                               │
                                                          (server push)
                                                               │
                                                             Saving ──► DataCollection
During the countdown the state sent to the server stays DataCollection; it becomes Recording once the countdown completes. Saving and Error are server-driven. When recording stops, the SDK pushes {"type": "state", "value": "saving"} and blocks every control except Y (exit) until the episode is written, then pushes data_collection to re-enable controls. Broadcasting error shows an error indicator in the headset. See almond_axol.vr for the frame protocol and send_feedback_state().

​
Next steps

Teleoperation quickstart

Go from install to a live VR session.

VR server API

The WSS server, frame schema, and feedback messages.