So, Pimax has now created their own Pimax experience software. I was thinking of Virtual Desktop and thought of a fantastic idea for the hand tracking. This would require Pimax to have their own sort of Virtual Desktop in the Pimax Experience. If you’ve ever used Virtual Desktop, one thing you may have noticed is that typing and moving your mouse can be difficult since you can’t see them when you’re in VR.
The idea is simple. We use the hand tracking to create a virtual equivalent of your table, keyboard, and mouse. Sounds difficult, right? Well, maybe not. There are only several prerequisites. The user must know the length of his letter keys, the length of his spacebar, and the length and width of his keyboard. These numbers can be easily acquired with a ruler, and will be given at the start of the setup.
First, we have the user place their hands flat on the desk where they will be using their mouse and keyboard. This step allows the software to determine the level that the mouse and keyboard will sit. Similar to when you set the floor in room setup. Then the user will touch the front edge of his desk. This step tells the software where the edge of the users desk is. Next, the user types out a small paragraph on their real keyboard. This paragraph would contain all letters of the alphabet, numbers 1-9 (and 0), along with some punctuation marks. Finally for the weird stuff we can just have the user press his shift key, or bracket keys. By doing this we can get the general layout of where the keys are located on the user’s keyboard.
Next Pimax’s software would create a simple rectangular 3D keyboard model with the same key layout and length as the users keyboard. Then, that keyboard model would be placed in the same location as they originally typed the paragraph. The final result is a basic, but fairly accurate model of the users keyboard in which the keyboard’s position is accurately reflected in real life.
For the mouse, setup is even easier. All the user has to do is place their hand on their mouse, and click their left and right mouse buttons. We have the user do this in several locations on their mouse buttons to get a general sense of how big their left and right mouse buttons are. Finally, a basic 3D model of the users mouse is placed on the virtual table. This clicking step is important since it allows us to not only determine the size of the mouse buttons, but also gain crucial information about how the user holds his mouse.
After doing setup, the user if left with a virtual table, and a virtual mouse. Since the software has seen how the user grips his mouse, we can do a sort of “fake positional tracking”. When the user’s hand and finger locations are similar compared to when he was gripping his mouse earlier in setup, we can assume that the user is holding his mouse. And since grip doesn’t change much when moving a mouse, we can positionally track the location of the mouse by tracking where the user moves his hand. Pretty cool, right? We are left with a keyboard that matches the basic layout of the users keyboard, as well as a generally accurate representation of the users mouse that is even positionally tracked.
This is a little complicated, but the final result is just what people have been wanting for a long time. Plus, it’s all done without tracking pucks or special keyboards. Virtually any normal keyboard and mouse can be used, regardless of their specific keyboard’s layout. This could really be a surprisingly killer feature of Pimax’s hand tracking module. I might even consider buying one if Pimax could pull this off.
