Input Sources

Input Sources is introduced in the WebXR Device API and OpenXR 1.1 to provide a way to interact with the XR environment. It represents the physical or virtual input devices that can be used to interact with the XR environment. The input sources can be controllers, hands, or other input devices.

Input System in screen-based platforms

In desktop and mobile devices, the input system is mainly based on specific input devices, such as a mouse, keyboard, touch screen, or gamepad. To interact with these devices, the development frameworks commonly provide specific APIs to the corresponding input devices.

For example, in Web development, the Pointer Events API is used to handle mouse, touch, and pen input, the Keyboard Events API is used to handle keyboard input, and the Gamepad API is used to handle gamepad input.

This is because both the input devices are clearly defined for different use cases, but in XR development, the input devices are more diverse and complex.

Input System in XR

In XR development, the input system is more complex because the input devices are more diverse, for example:

• Controllers: Handheld devices with buttons, triggers, and joysticks.
• Hands: The user's hands are tracked in the XR environment.
• Eyes: The user's gaze direction is tracked in the XR environment.
• Voice: The user's voice commands are recognized in the XR environment.

To handle these diverse input devices, the input source is introduced to represent the physical or virtual input sources that can be used to interact with the XR environment.

Note also that an input source doesn't represent a specific input device, for example, at visionOS Safari, there is a transient-pointer input source to be introduced, that will be available only when the user pinches and compute the target ray based on the gaze direction and the pinched hand pose, this is an example showing that an input source can from the collaboration of multiple input devices, because the WebXR input source is designed to be more abstract and flexible.

Want to learn more about Safari's natural input design? Check out this blog post: Introducing Natural Input for WebXR in Apple Vision Pro.

Input Sources in JSAR

JSAR implements the WebXR input sources, and provides the common input sources in Rokid XR devices, such as: main-controller, hand-left, hand-right and eye, for the full specification of JSAR implemented input sources, please refer to the References/WebXR Input Sources.

Next

In the next section, let's recap the HTML, the basis of Web development, and why do we still decide to use HTML to build XR applications.