116 lines
6.6 KiB
Markdown
116 lines
6.6 KiB
Markdown
# :raised_hands: Godot Ultraleap Plugin
|
|
|
|
A GDExtension plugin to use Ultraleap hand tracking in your project. See GIF below for very old status (it does more now!).
|
|
|
|
- [:sunflower: About](#sunflower-about)
|
|
- [:white\_check\_mark: Features](#white-check-mark-features)
|
|
- [:hammer\_and\_wrench: Build](#hammer-and-wrench-build)
|
|
- [🍎 Mac extra instructions](#mac-extra-instructions)
|
|
- [:wine\_glass: Windows extra instructions](#wine-glass-windows-extra-instructions)
|
|
- [:robot: Android instructions](#robot-android-instructions)
|
|
- [:fast\_forward: Quick start](#fast-forward-quick-start)
|
|
- [:crown: Credits](#crown-credits)
|
|
|
|
|
|
|
|
## :sunflower: About
|
|
|
|
This GDExtension aim at exposing and abstracting the LeapC API to make Ultraleap's hand tracking available in Godot, for any kind of use. Out of the box Godot supports hand tracking through OpenXR, however that means it's only accessible for people who wants to use it for XR. Ultraleap's hand tracking is capable of tracking hands in two extra setups:
|
|
|
|
* Tabletop or "Desktop", flat on a desk pointing up
|
|
* "Screentop", on top of a monitor, pointing downward or with an angle of maximum 30 degrees
|
|
|
|
The priorities of this project is to target the most useful features of Ultraleap's hand tracking while exploring how hand tracking can be used in Godot. Because of this, there are demos and scripts that are not part of the extension but are part of the project. Some of the scripts implements new features or features related to the game engine rather than the hand tracking itself: displaying joints, rigging a skeleton etc... If possible, and when the project has matured enough, some of those extra features could be ported to the extension to make them more readily available and more performant.
|
|
|
|
### :white_check_mark: Features
|
|
|
|
The extension has the following features:
|
|
|
|
* Hand tracking events
|
|
* Multi-device support
|
|
* Camera images and distortion matrix
|
|
* Hand tracking interpolation
|
|
* Selecting tracking mode
|
|
* Pausing/resuming tracking
|
|
* Access to some obscure and unused parameters
|
|
|
|
One of the demos it comes with is an attempt at making a full-blown visualizer like Ultraleap's Control Panel but with more features and not using Unity. This is the `main.tscn` scene.
|
|
|
|
The other important scene is an obvious rip-off of Ultraleap's Blocks demo used to test rigging a skeleton on the hand data, and interacting with objects/gestures. This one is called `scenes/RiggedHands.tscn`
|
|
|
|
## :hammer_and_wrench: Build
|
|
|
|
First thing first, pull all the sources and submodules:
|
|
|
|
```bash
|
|
git clone https://forge.lunai.re/rodolphe/godot-ultraleap-plugin
|
|
cd godot-ultraleap-plugin
|
|
git submodule update --init
|
|
```
|
|
|
|
**If you want to build the plugin for Android, follow [the specific instructions](#robot-android-instructions)**
|
|
|
|
To build you'll need to have Python and SCons installed. On Linux and Mac it can be installed using your favourite package manager (homebrew for Mac). On Windows you probably want to install it with [pip](https://pypi.org/project/SCons/). You'll also need a compiler, GCC on Linux, XCode Command Line Tools on Mac, and MSBuild on Windows.
|
|
|
|
From there, just launch scons:
|
|
|
|
```bash
|
|
scons
|
|
```
|
|
|
|
If you're building for Mac or Windows, make sure to follow the extra instructions to get the plugin to work in your project.
|
|
|
|
### 🍎 Mac extra instructions
|
|
|
|
After building the extension, copy `/Applications/Ultraleap Hand Tracking Service.app/Contents/LeapSDK/lib/libLeapC.5.dylib` in the `demo/addons/godot_ultraleap_plugin/bin` folder.
|
|
|
|
### :wine_glass: Windows extra instructions
|
|
|
|
After building the extension, copy `C:/Program Files/Ultraleap/LeapSDK/lib/x64/LeapC.dll` in the `demo/addons/godot_ultraleap_plugin/bin` folder.
|
|
|
|
### :robot: Android instructions
|
|
|
|
There are a lot of extra-steps to build for Android. We'll start by assuming you already have installed the Android SDK and the NDK, plus the build tools and all the things required (you can refer to the [Godot documentation](https://docs.godotengine.org/en/stable/tutorials/export/exporting_for_android.html#download-the-android-sdk)). We'll also assume that all the submodules have been pulled and are up-to-date.
|
|
|
|
On top of that, you'll need to download and copy some files around by yourself:
|
|
|
|
* [`godot-lib.4.1.2.stable.template_release.aar`](https://github.com/godotengine/godot-builds/releases/download/4.1.2-stable/godot-lib.4.1.2.stable.template_release.aar): put it in `godot-ultraleap-binder/ultraleap-binder/libs`
|
|
* [`libLeapC.so` (Android version from Ultraleap's repository)](https://github.com/ultraleap/UnityPlugin/raw/develop/Packages/Tracking/Core/Runtime/Plugins/Android/libs/libLeapC.so): put it in `android`
|
|
* [`UltraleapTrackingServiceBinder.aar` (from Ultraleap's repository)](https://github.com/ultraleap/UnityPlugin/raw/develop/Packages/Tracking/Core/Runtime/Plugins/Android/libs/UltraleapTrackingServiceBinder.aar): put it in `godot-ultraleap-binder/ultraleap-binder/libs`
|
|
|
|
You'll also need to find the file `LeapC.h` which should be wherever your SDK is installed, and place it in `android`. Now we can go on with building:
|
|
|
|
* Go into `godot-ultraleap-plugin`
|
|
* Build the service binder with `./gradlew build`
|
|
* Go into the root of the project
|
|
* Build the plugin with `ANDROID_NDK_ROOT=<path-to-your-ndk> scons platform=android architecture=arm64`
|
|
|
|
If everything went well, all files should have been copied in their respective folders.
|
|
|
|
## :fast_forward: Quick start
|
|
|
|
**These are very simple instructions and barely covers all the possibilities but it shows my willingness to try and document things to help people who want to adopt this plugin**
|
|
|
|
If the build went well, the final result should be found in `demo/addons/godot_ultraleap_plugin`. Copy this folder over your `addons` folder in your project to add it to your project. Reload your project for it to take effect.
|
|
|
|
From there the most simple way to get hand data is to add an UltraleapHandTracking node and an UltraleapDeviceNode node:
|
|
|
|
* Add an UltraleapHandTracking node to your scene
|
|
* Add an UltraleapDeviceNode node to your scene
|
|
* Assign the UltraleapHandTracking node to the `tracker` field on your UltraleapDeviceNode
|
|
* Add a script on your UltraleapDeviceNode containing the following:
|
|
|
|
```
|
|
extends UltraleapDeviceNode
|
|
|
|
func _process(_delta):
|
|
var frame : UltraleapFrame = get_last_frame()
|
|
if frame != null and frame.is_right_hand_visible:
|
|
print(frame.right_hand.palm.position)
|
|
```
|
|
|
|
* Hit play, make sure your hand tracking camera is plugged in, flat on your desk, and show your right hand. It should print the palm position in the editor's output.
|
|
|
|
## :crown: Credits
|
|
|
|
[Glove](https://poly.pizza/m/l1zv4LaA4I) by [Quaternius](https://poly.pizza/u/Quaternius) |