XR Input

Due to the flux in APIs and packages related to XR and Input in Unity, there is a wide range of libraries and frameworks available to use. Luckily, Nova's simple interaction API makes it straightforward to support XR controllers and/or hand tracking using whichever library best suits your project.

See the Grid Inventory Sample and the XR Hand Menu Sample for examples of XR controller and XR hand input in action, respectively.

See Input Overview for an overview of Nova's input system.

Controllers

The following snippet will get you started providing XR controller input to Nova using Unity's XR Input APIs, however it can be easily modified to work with the input library or package you are using. The key Nova-relevant aspects are:

1. Calling Interaction.Point with the controller's world-space ray and its pressed state.
2. Calling Interaction.Scroll with the controller's world-space ray and thumbstick value.

using Nova;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.XR;

public class XRControllerInputSample : MonoBehaviour
{
    private const uint RightPointControlID = 1;
    private const uint RightScrollControlID = 2;

    private const uint LeftPointControlID = 3;
    private const uint LeftScrollControlID = 4;

    private InputDevice rightController;
    private InputDevice leftController;

    private void Start()
    {
        // Subscribe to device connect/disconect events
        InputDevices.deviceConnected += (_) => UpdateControllers();
        InputDevices.deviceDisconnected += (_) => UpdateControllers();
        UpdateControllers();
    }

    private void Update()
    {
        UpdateController(rightController, RightPointControlID, RightScrollControlID);
        UpdateController(leftController, LeftPointControlID, LeftScrollControlID);
    }

    private void UpdateController(InputDevice controllerDevice, uint pointID, uint scrollID)
    {
        if (!controllerDevice.isValid)
        {
            // Controller not connected, nothing to do
            return;
        }

        if (!controllerDevice.TryGetFeatureValue(CommonUsages.devicePosition, out Vector3 position))
        {
            // Couldn't get position of controller
            return;
        }

        if (!controllerDevice.TryGetFeatureValue(CommonUsages.deviceRotation, out Quaternion rotation))
        {
            // Couldn't get rotation of controller
            return;
        }

        // Try to get the thumbstick and primary button state. If these fail,
        // we will just use the default values
        controllerDevice.TryGetFeatureValue(CommonUsages.primaryButton, out bool pressed);
        controllerDevice.TryGetFeatureValue(CommonUsages.primary2DAxis, out Vector2 thumbstickValue);

        // Convert position and rotation to world-space ray
        Ray ray = new Ray(position, rotation * Vector3.forward);

        if (thumbstickValue != Vector2.zero)
        {
            // If the thumbstick has a value, use it to scroll
            Interaction.Update scrollUpdate = new Interaction.Update(ray, scrollID);
            Interaction.Scroll(scrollUpdate, thumbstickValue);
        }

        // Point with the controller's ray and pressed state
        Interaction.Update pointUpdate = new Interaction.Update(ray, pointID);
        Interaction.Point(pointUpdate, pressed);
    }

    /// <summary>
    /// Tries to get the controllers
    /// </summary>
    private void UpdateControllers()
    {
        List<InputDevice> controllers = new List<InputDevice>();
        InputDeviceCharacteristics desiredCharacteristics = InputDeviceCharacteristics.HeldInHand | InputDeviceCharacteristics.Controller;

        // Get the right controller
        InputDevices.GetDevicesWithCharacteristics(desiredCharacteristics | InputDeviceCharacteristics.Right, controllers);

        if (controllers.Count > 0)
        {
            rightController = controllers[0];
        }

        // Get the left controller
        InputDevices.GetDevicesWithCharacteristics(desiredCharacteristics | InputDeviceCharacteristics.Left, controllers);
        if (controllers.Count > 0)
        {
            leftController = controllers[0];
        }
    }
}

Additional Buttons

Most XR controllers have several buttons, and you may want your Nova content to react differently based on which button(s) were pressed. This can be accomplished with the UserData field of Interaction.Update, which allows you to pass arbitrary data to Gesture handlers. The Right Clicks section of the MouseInput article shows how to do this for providing both left and right mouse buttons, and can be extended for arbitrarily complex controllers.

Hands

The Sphere-based Interaction.Point method can be used to make Nova content hand-interactable on devices which support articulated hand tracking. The following code snippet demonstrates how to do this, only in your project you'd replace the finger-tip-position/hand-tracked stubs with calls to the hand tracking API you are using.

Note

Sphere-based Interaction.Point gesture events will be more reliable when the target UIBlocks have a non-zero Z-Size. Due to the noisy behavior of hand-tracking-based input, adding depth to the collidable volume will make "poking all the way through" an interactable target less likely to occur accidentally. Z-Size and other Z-Axis layout properties is supported on all UIBlock types. If you don't see the Z-Axis fields in the UIBlock Inspector, expand Tools and click the 3D toggle button.

Warning

To get the most reliable behavior between potentially conflicting press, drag, and scroll gestures while using the Sphere-based Interaction.Point path, the entry point of the overlapping target UIBlocks must all be coplanar. If the entry points aren't coplanar, say a scrollable ListView's front face is positioned behind a draggable list item's front face, attempts to scroll the ListView will likely fail, and the list item will be dragged instead.

using Nova;
using UnityEngine;

public class XRHandInputSample : MonoBehaviour
{
    private const uint RightFingerTipControlID = 1;
    private const uint LeftFingerTipControlID = 2;

    /// <summary>
    /// The radius of interaction for the finger tip.
    /// I.e. how close the finger must be to be considered as "interacting" with
    /// Nova content. Conceptually similar to a SphereCollider's radius.
    /// </summary>
    public float FingerTipCollisionRadius = .01f;

    /// <summary>
    /// Replace these with calls to the hand tracking library you are using.
    /// E.g.: OVRHand.IsTracked
    /// </summary>
    private bool RightHandTracked => true;
    private bool LeftHandTracked => true;

    /// <summary>
    /// Replace these with calls to the hand tracking library you are using.
    /// Or if you have a GameObject that is placed on/follows the index finger tip,
    /// these could be <c>fingerTipTransform.position</c>.
    /// </summary>
    private Vector3 RightFingerTipWorldPosition => Vector3.zero;
    private Vector3 LeftFingerTipWorldPosition => Vector3.zero;

    private void Update()
    {
        if (RightHandTracked)
        {
            // Create a Nova.Sphere from the right index finger tip position
            Sphere rightFingerTipSphere = new Sphere(RightFingerTipWorldPosition, FingerTipCollisionRadius);

            // Call Point with the newly created sphere and the right finger control ID.
            Interaction.Point(rightFingerTipSphere, RightFingerTipControlID);
        }

        if (LeftHandTracked)
        {
            // Create a Nova.Sphere from the left index finger tip position
            Sphere leftFingerTipSphere = new Sphere(LeftFingerTipWorldPosition, FingerTipCollisionRadius);

            // Call Point with the newly created sphere and the left finger control ID.
            Interaction.Point(leftFingerTipSphere, LeftFingerTipControlID);
        }
    }
}