FINALLY. Jeez…finally I feel ready to release the first version of Hand-Tracking Interaction in VR Experience! Blood, sweat and tears has been put into make this possible, but hell yeah its neat!
Beta v0.30 is being uploaded in a few minutes and will be available to you all for testing!
If you have any Leap Motion unit at home, this version should work out-of-the-box with your Pimax and Leap Motion. The hand tracking UI interaction is mainly made for Pimax (UltraHaptics) hand tracking units, that starts to ship anytime now, and I know nobody of you have any yet, but meanwhile, just try it with any Leap Motion if you have one.
Let me know what you think, and lets improve this together!
PS: I will answer all your new posts and reports later tonight (just been too busy the past days getting the Beta 0.30 done…)
Don’t connect an old LeapMotion controller to one of the external ports of a 5K+, 5K XR, 8K and 8K+. The LeapMotion software will think that you attached an UltraHaptics Pimax Module and try to automatically overwrite the firmware of the LeapMotion Controller (moving blue Leap Motion indicator instead of the normal green one) never coming to an end. It of course fails to update the firmware and no harm is done but you can get pretty nervous that you might just have bricked your LeapMotion Controller.
“wird aktualisiert” in German means “is being updated”.
Mind… blown!
I can confirm hand tracking works really well on my OG Leap Motion. Will post the thingiverse 3d model for leap motion mount tomorrow. Edit: this one Pimax Leap Motion Bolt-on Mount by JaZepi - Thingiverse
Horizontal tracking is about +/-40 degrees tops. But vertical is way better. Pinch mode is really, really sensitive, but that is ok. I need to relearn my motions. It feels really natural to use my hands instead of the index controllers! I’lll try more later today. It is almost 3 am in Sweden.
I will be getting hand tracking soon along with my backer box. Perhaps one hand can pinch to hold the scroll bar, while the other hand can pinch/press to select ? Thinking like holding a balloon with one hand to then the other hand to poke it ?
Is there a compatability list of games for hand tracking yet?
haha yep, although the hand tracking is already finished and starting to ship any time now (as far as I know)
Wow thats good to know, thanks Alex! I actually dont have any of the old LeapMotion modules (except the ones made for Pimax), but it sounds like its important to prevent auto-update of FW before connecting to Pimax.
Only a few minor bugs fixed, such as keyboard in the “Create New Profile” was not interacting with the input field (the Search bar took over the text hehe), but thats now fixed in 0.30. I recommend you keep the beta up to date, as theres always something fixed in every versioin.
Thankfully thats possible
Awesome Very happy to hear that!
Yes the sensitivity is too high, especally when pinching. Im trying to customize/tweak this a little using my own scripts, because without the tweaks, its even more sensitive. And believe it or not - by default with the Leap Motion UI plugin, clicks are made instantly upon press-down (retarded solution). I have no idea why Leap Motion made it this way, its totally useless for hand tracking UI interaction if you are not able to scroll properly. With their solution, you would click on things by accident all the time, and scrollviews would be totally unusable.
Also, Unity has changed loads of things when it comes to UI hierarchy since Leap Motion UI plugin for interaction was made, and it doesnt seem like the UI plugin is being updated anymore. Nowadays, UltraHaptics use a different leap Motion interaction engine that requires the UI to be built in a specific way (box colliders everywhere, restricted use of Raycas targets etc), and if I used the latest engine, it would totally mess up the entire VRExperience and require crazy amount of time to just re-made the UI almost completely. They really made things more complicated than before, without any advantage, since you can actually still use the old UI plugin and simultanously benefit from all new features with the new Interaction engine. All it required was a bunch of tweaks and changes in the script
Yes, that works. Both with pinch-mode and touch-mode you can interact with 2 hands simultanously and send dual actions without any conflict. Dragging scrollbars works like a charm, even if I prefer to just scroll inside the scrollview directly by dragging.
There will be one upon the hand-tracking unit release!
Ahh, I see. Better uncheck that option. Sorry if I knew that, I would have warned you…
O. My! I’ve been waiting for hand tracking for a looong time! (lol, not really sure why; to interact with sim cockpits, I guess) You are my new favorite person! Even without all the work you’re doing on Peetools!
I hate coming up with things that just increase the workload, but I’m thinking that you’re going to need a menu/ui for allowing user customization of things like movement and click sensitivity.
Now I know why changing the UI scale changes the position of the UI! Heh, I was gonna mention it as a bug. Makes sense, now!
You made a gesture that screamed for an action. Put both hands in front of you, pinch both and move your hands apart. Roughly a “pinch and spread”. This should trigger the UI scale to increase or decrease. When you did that, I was actually surprised that it didn’t change. Too natural. And seriously not important!
There has been a gentleman working on a hand tracked keyboard layout. Although he was working with a Quest, his work is worth checking out. I’m going to pass along a link to the YouTube of what you’ve got going to him, as well. Who knows, maybe he’ll buy a Pimax so that he can work with you!
The old Leap Motion connected to the bottom connector registers as High-speed 480MBps. All internal devices (including audio) registers as Full-speed 12MBps.
Awesome, looking forward to hear your thoughts and input
Wow this looks super interesting and its indeed a great idea! I will immediately look into this on Monday morning (busy family times this weekend…) Thanks a lot for this, and lets further discuss the possibilities on Monday!
Are there individual folders for each title? Can I change the icon panel (gamebanner.jpg or whatever) that shows up in PE for an individual game and if so where do I find those folders? I have several homebrew apps that just show up as a default steam logo instead and I would like to change them to a custom image.
To be honest, I havent even considered this before, but you might got me thinking to add a feature to replace the thumbnail of games to any preferred. This will require a GUI file explorer which Im actually working on already. If all goes well, we may have it ready end of next week.
Currently,. the thumbnails are not really cistomizable. They are all located in:
C:\Users\user\AppData\Local\Pimax\PiTool\games\
You can actually change the thumbnails of SteamVR games, Oculus and Viveport by just replacing the image (using same name), but custom imported games, Revive and non-Steam games in SteamVR library has a static image bound to the gallery button. I will change that as soon as the file explorer is ready to be used, and I will try to make sure you can customize the thumbnails for any game in all categories, which isnt much work after all, as long as the file explorer works.
The file explorer will also be used to import custom games (just like PiTool does already), and also other functionality, such as maybe importing new Skyboxes (backgrounds) etc…
I said You would probably be done before the hardware… 
