XR Input
Due to the flux in APIs and packages related to XR and Input in Unity, there is a wide range of libraries and frameworks available to use. Luckily, Nova's simple interaction API makes it straightforward to support XR controllers and/or hand tracking using whichever library best suits your project.
See the Grid Inventory Sample and the XR Hand Menu Sample for examples of XR controller and XR hand input in action, respectively.
See Input Overview for an overview of Nova's input system.
Controllers
The following snippet will get you started providing XR controller input to Nova using Unity's XR Input APIs, however it can be easily modified to work with the input library or package you are using. The key Nova-relevant aspects are:
- Calling Interaction.Point with the controller's world-space ray and its pressed state.
- Calling Interaction.Scroll with the controller's world-space ray and thumbstick value.
using Nova;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.XR;
public class XRControllerInputSample : MonoBehaviour
{
private const uint RightPointControlID = 1;
private const uint RightScrollControlID = 2;
private const uint LeftPointControlID = 3;
private const uint LeftScrollControlID = 4;
private InputDevice rightController;
private InputDevice leftController;
private void Start()
{
// Subscribe to device connect/disconect events
InputDevices.deviceConnected += (_) => UpdateControllers();
InputDevices.deviceDisconnected += (_) => UpdateControllers();
UpdateControllers();
}
private void Update()
{
UpdateController(rightController, RightPointControlID, RightScrollControlID);
UpdateController(leftController, LeftPointControlID, LeftScrollControlID);
}
private void UpdateController(InputDevice controllerDevice, uint pointID, uint scrollID)
{
if (!controllerDevice.isValid)
{
// Controller not connected, nothing to do
return;
}
if (!controllerDevice.TryGetFeatureValue(CommonUsages.devicePosition, out Vector3 position))
{
// Couldn't get position of controller
return;
}
if (!controllerDevice.TryGetFeatureValue(CommonUsages.deviceRotation, out Quaternion rotation))
{
// Couldn't get rotation of controller
return;
}
// Try to get the thumbstick and primary button state. If these fail,
// we will just use the default values
controllerDevice.TryGetFeatureValue(CommonUsages.primaryButton, out bool pressed);
controllerDevice.TryGetFeatureValue(CommonUsages.primary2DAxis, out Vector2 thumbstickValue);
// Convert position and rotation to world-space ray
Ray ray = new Ray(position, rotation * Vector3.forward);
if (thumbstickValue != Vector2.zero)
{
// If the thumbstick has a value, use it to scroll
Interaction.Update scrollUpdate = new Interaction.Update(ray, scrollID);
Interaction.Scroll(scrollUpdate, thumbstickValue);
}
// Point with the controller's ray and pressed state
Interaction.Update pointUpdate = new Interaction.Update(ray, pointID);
Interaction.Point(pointUpdate, pressed);
}
/// <summary>
/// Tries to get the controllers
/// </summary>
private void UpdateControllers()
{
List<InputDevice> controllers = new List<InputDevice>();
InputDeviceCharacteristics desiredCharacteristics = InputDeviceCharacteristics.HeldInHand | InputDeviceCharacteristics.Controller;
// Get the right controller
InputDevices.GetDevicesWithCharacteristics(desiredCharacteristics | InputDeviceCharacteristics.Right, controllers);
if (controllers.Count > 0)
{
rightController = controllers[0];
}
// Get the left controller
InputDevices.GetDevicesWithCharacteristics(desiredCharacteristics | InputDeviceCharacteristics.Left, controllers);
if (controllers.Count > 0)
{
leftController = controllers[0];
}
}
}
Additional Buttons
Most XR controllers have several buttons, and you may want your Nova content to react differently based on which button(s) were pressed. This can be accomplished with the UserData field of Interaction.Update, which allows you to pass arbitrary data to Gesture handlers. The Right Clicks section of the MouseInput article shows how to do this for providing both left and right mouse buttons, and can be extended for arbitrarily complex controllers.
Hands
The Sphere-based Interaction.Point method can be used to make Nova content hand-interactable on devices which support articulated hand tracking. The following code snippet demonstrates how to do this, only in your project you'd replace the finger-tip-position/hand-tracked stubs with calls to the hand tracking API you are using.
Note
Sphere-based Interaction.Point gesture events will be more reliable when the target UIBlocks have a non-zero Z-Size. Due to the noisy behavior of hand-tracking-based input, adding depth to the collidable volume will make "poking all the way through" an interactable target less likely to occur accidentally. Z-Size and other Z-Axis layout properties is supported on all UIBlock types. If you don't see the Z-Axis fields in the UIBlock Inspector, expand Tools
and click the 3D
toggle button.
Warning
To get the most reliable behavior between potentially conflicting press, drag, and scroll gestures while using the Sphere-based Interaction.Point path, the entry point of the overlapping target UIBlocks must all be coplanar. If the entry points aren't coplanar, say a scrollable ListView's front face is positioned behind a draggable list item's front face, attempts to scroll the ListView will likely fail, and the list item will be dragged instead.
using Nova;
using UnityEngine;
public class XRHandInputSample : MonoBehaviour
{
private const uint RightFingerTipControlID = 1;
private const uint LeftFingerTipControlID = 2;
/// <summary>
/// The radius of interaction for the finger tip.
/// I.e. how close the finger must be to be considered as "interacting" with
/// Nova content. Conceptually similar to a SphereCollider's radius.
/// </summary>
public float FingerTipCollisionRadius = .01f;
/// <summary>
/// Replace these with calls to the hand tracking library you are using.
/// E.g.: OVRHand.IsTracked
/// </summary>
private bool RightHandTracked => true;
private bool LeftHandTracked => true;
/// <summary>
/// Replace these with calls to the hand tracking library you are using.
/// Or if you have a GameObject that is placed on/follows the index finger tip,
/// these could be <c>fingerTipTransform.position</c>.
/// </summary>
private Vector3 RightFingerTipWorldPosition => Vector3.zero;
private Vector3 LeftFingerTipWorldPosition => Vector3.zero;
private void Update()
{
if (RightHandTracked)
{
// Create a Nova.Sphere from the right index finger tip position
Sphere rightFingerTipSphere = new Sphere(RightFingerTipWorldPosition, FingerTipCollisionRadius);
// Call Point with the newly created sphere and the right finger control ID.
Interaction.Point(rightFingerTipSphere, RightFingerTipControlID);
}
if (LeftHandTracked)
{
// Create a Nova.Sphere from the left index finger tip position
Sphere leftFingerTipSphere = new Sphere(LeftFingerTipWorldPosition, FingerTipCollisionRadius);
// Call Point with the newly created sphere and the left finger control ID.
Interaction.Point(leftFingerTipSphere, LeftFingerTipControlID);
}
}
}