🙌 Godot Ultraleap Plugin

A GDExtension plugin to use Ultraleap hand tracking in your project. See GIF below for very old status (it does more now!).

• 🌻 About
  • ✅ Features
• 🛠️ Build
  • 🍎 Mac extra instructions
  • 🍷 Windows extra instructions
  • 🤖 Android instructions
• ⏩ Quick start
• 👑 Credits

🌻 About

This GDExtension aim at exposing and abstracting the LeapC API to make Ultraleap's hand tracking available in Godot, for any kind of use. Out of the box Godot supports hand tracking through OpenXR, however that means it's only accessible for people who wants to use it for XR. Ultraleap's hand tracking is capable of tracking hands in two extra setups:

• Tabletop or "Desktop", flat on a desk pointing up
• "Screentop", on top of a monitor, pointing downward or with an angle of maximum 30 degrees

The priorities of this project is to target the most useful features of Ultraleap's hand tracking while exploring how hand tracking can be used in Godot. Because of this, there are demos and scripts that are not part of the extension but are part of the project. Some of the scripts implements new features or features related to the game engine rather than the hand tracking itself: displaying joints, rigging a skeleton etc... If possible, and when the project has matured enough, some of those extra features could be ported to the extension to make them more readily available and more performant.

✅ Features

The extension has the following features:

• Hand tracking events
• Multi-device support
• Camera images and distortion matrix
• Hand tracking interpolation
• Selecting tracking mode
• Pausing/resuming tracking
• Access to some obscure and unused parameters

One of the demos it comes with is an attempt at making a full-blown visualizer like Ultraleap's Control Panel but with more features and not using Unity. This is the main.tscn scene.

The other important scene is an obvious rip-off of Ultraleap's Blocks demo used to test rigging a skeleton on the hand data, and interacting with objects/gestures. This one is called scenes/RiggedHands.tscn

🛠️ Build

First thing first, pull all the sources and submodules:

git clone https://forge.lunai.re/rodolphe/godot-ultraleap-plugin
cd godot-ultraleap-plugin
git submodule update --init

If you want to build the plugin for Android, follow the specific instructions

To build you'll need to have Python and SCons installed. On Linux and Mac it can be installed using your favourite package manager (homebrew for Mac). On Windows you probably want to install it with pip. You'll also need a compiler, GCC on Linux, XCode Command Line Tools on Mac, and MSBuild on Windows.

From there, just launch scons:

scons

If you're building for Mac or Windows, make sure to follow the extra instructions to get the plugin to work in your project.

🍎 Mac extra instructions

After building the extension, copy /Applications/Ultraleap Hand Tracking Service.app/Contents/LeapSDK/lib/libLeapC.5.dylib in the demo/addons/godot_ultraleap_plugin/bin folder.

🍷 Windows extra instructions

After building the extension, copy C:/Program Files/Ultraleap/LeapSDK/lib/x64/LeapC.dll in the demo/addons/godot_ultraleap_plugin/bin folder.

🤖 Android instructions

There are a lot of extra-steps to build for Android. We'll start by assuming you already have installed the Android SDK and the NDK, plus the build tools and all the things required (you can refer to the Godot documentation). We'll also assume that all the submodules have been pulled and are up-to-date.

On top of that, you'll need to download and copy some files around by yourself:

• godot-lib.4.1.2.stable.template_release.aar: put it in godot-ultraleap-binder/ultraleap-binder/libs
• libLeapC.so (Android version from Ultraleap's repository): put it in android
• UltraleapTrackingServiceBinder.aar (from Ultraleap's repository): put it in godot-ultraleap-binder/ultraleap-binder/libs

You'll also need to find the file LeapC.h which should be wherever your SDK is installed, and place it in android. Now we can go on with building:

• Go into godot-ultraleap-plugin
• Build the service binder with ./gradlew build
• Go into the root of the project
• Build the plugin with ANDROID_NDK_ROOT=<path-to-your-ndk> scons platform=android architecture=arm64

If everything went well, all files should have been copied in their respective folders.

⏩ Quick start

These are very simple instructions and barely covers all the possibilities but it shows my willingness to try and document things to help people who want to adopt this plugin

If the build went well, the final result should be found in demo/addons/godot_ultraleap_plugin. Copy this folder over your addons folder in your project to add it to your project. Reload your project for it to take effect.

From there the most simple way to get hand data is to add an UltraleapHandTracking node and an UltraleapDeviceNode node:

• Add an UltraleapHandTracking node to your scene
• Add an UltraleapDeviceNode node to your scene
• Assign the UltraleapHandTracking node to the tracker field on your UltraleapDeviceNode
• Add a script on your UltraleapDeviceNode containing the following:

extends UltraleapDeviceNode

func _process(_delta):
	var frame : UltraleapFrame = get_last_frame()
	if frame != null and frame.is_right_hand_visible:
		print(frame.right_hand.palm.position)

• Hit play, make sure your hand tracking camera is plugged in, flat on your desk, and show your right hand. It should print the palm position in the editor's output.

👑 Credits

Glove by Quaternius