Eye Tracking Fix Update:

Pimel · May 28, 2022, 1:39pm

So I’m looking to get the tracker working in VRChat. I found a dll mod on your github, is that the one to get started? Do I need to create an avatar from scratch to map the eye tracker or can I use any eye tracking enabled avatar from there?

guppyexpress · May 29, 2022, 1:06am

Hello, so to get the tracker working in vrchat, the mod does not work with the Pi1. but there is an OSC for vrchat that should work with it made by @Toadstooliv. VRCFT makes an OSC but it sometimes has an error with the Pi1 unlike the one Toadstooliv made and ive been working with them to get it working on there as well.
you have to follow VRCFT’s instructions on setting up your avatar to how they have it

Pimel · May 29, 2022, 12:26pm

Thanks for the info, could you comment on how easy it is to get something working? I don’t have an avatar I want to use, any will do as long as it tracks my eyes

guppyexpress · May 31, 2022, 4:29am

its very very easy to get it working. its adding 3 parameters and running an OSC program
here is the vrcft program to look at how to set it up GitHub - benaclejames/VRCFaceTracking: VRChat OSC App to allow AV3 Avatars to interact with Vive SRanipal Eye and Lip Tracking SDK

Arend · June 4, 2022, 10:46pm

Hi! This is my first post to OpenMR.
Yesterday I installed the eye tracking mod to my Pimax 8kx headset, and I also installed Guppyexpress’ software fix. Everything was working (I guess), and I was very excited and even donated to Guppyexpress.
But today I can’t get it to work right anymore. Very frustrating!!!
It seems that some process or software crashes after a few seconds. I get the grayscale images for both eyes (I can see my fingers when I wave them in front of the lenses). Then the images go black. When I terminate the “forward.exe”, this is restarted automatically, the grayscale camera images reappear, but then the images go black again after 1 or 2 seconds. Calibration also does not work, neither does DFR in VR. I reran the whole installation process according to the instructions twice, but still no success. What can I do?
(In case it is relevant: Ryzen 5900X, RTX 3080)

guppyexpress · June 5, 2022, 12:53am

Hey Arend!
First of all
thank you so much for donating! <3 your donation helps the work to improve the eye tracker greatly

second of all. I would be more than happy to help debug this for you
Please shoot me a dm on discord @ guppyexpress#0001

Looking forwards to chatting with you and debugging this issue so you get the best eye tracking experience possible

Arend · June 5, 2022, 10:19am

Thanks for your kind reply. I just joined discord (complete noob ) and sent you a friend request.

guppyexpress · June 5, 2022, 2:15pm

Haha no worries it’s just easier to debug on there because your able to send and receive files

Arend · June 6, 2022, 7:49pm

Reporting back – Guppyexpress helped me track down the problem. We spent some time troubleshooting, awesome support! The issue is probably within my HMD: Using the provided short USB cable that links the eye tracker directly to the HMD, the setup kept crashing after a couple seconds. When I connected the device to the motherboard directly with a longer cable, it started working properly. So now there’s an extra cable to deal with, but at least I can use the thing.
(To be clear: The issue with my HMD and eye tracker was already there before I installed Guppyexpress’s mods.)

matsad · June 15, 2022, 9:11am

Hello,

Could anybody please confirm if the NGenesis/PimaxEyeTracker repo contains the newest version of the aSeeVR SDK? If not, how can I get it?

Edit.
I ran the utility library from the repo with Unity and I have found (same as @mbucchia) that most of the eye parameters are equal to 0. Has anybody been able to figure out why that is?

I need the record the raw eye-tracking data from the device. As far as I understand, @guppyexpress ’ eye-tracking update fixes how PiTool handles eye tracking, but won’t necessarily allow me to get the raw eye-tracking data. Is this correct?

guppyexpress · June 15, 2022, 2:08pm

Hey! No that doesn’t contain the sdk, that’s just a template code Ngenesis made on an interface that everyone uses.
If you would like the documentation to the code, i have it hosted here on GitHub aSeeVR UserSDK: Getting Started with aSeeVR UserSDK
If u need the tracking data you need to access it via the port it’s connected to. at the moment it is port 5777 but I’m going to change it later to reduce noise of other programs

mbucchia · June 15, 2022, 3:29pm

I never figured out why all 0. On the end, I only needed gaze ray, so i used the recommended eye gaze X,Y values (the 2 values that are not 0) and it worked out for me:

github.com

mbucchia/OpenXR-Toolkit/blob/baafab3606d66482066d3142239372b198e0f01d/XR_APILAYER_NOVENDOR_toolkit/eyetracker.cpp#L477


      
                  if (!eye_data) {
                      return;
                  }
          
          
        PimaxEyeTracker* tracker = reinterpret_cast<PimaxEyeTracker*>(context);
          
          
        int64_t timestamp = 0;
                  aSeeVR_get_int64(eyeData, aSeeVREye::undefine_eye, aSeeVREyeDataItemType::timestamp, &timestamp);
          
          
        aSeeVRPoint2D point2D = {0};
                  aSeeVR_get_point2d(eyeData, aSeeVREye::undefine_eye, aSeeVREyeDataItemType::gaze, &point2D);
          
          
        tracker->setEyeData(timestamp, point2D.x, point2D.y);
              }
          
          
    static void _7INVENSUN_CALL getCoefficientCallback(const aSeeVRCoefficient* data, void* context) {
                  PimaxEyeTracker* tracker = reinterpret_cast<PimaxEyeTracker*>(context);
                  tracker->setCoefficients(*data);
              }
          };

matsad · June 16, 2022, 8:22am

Ahh, now I get it! Thank you for your response.

I will try to use your version then. In what format/layout/encoding is the data written to the port?

Edit.
I have found a file called recv_example.cpp in aSeeVR_UserSDK\forward that seems to be reading eye-tracking data from a port. However, these are also only the projected(?) X,Y values. Would it be possible to hook up to the raw eye tracker data stream? Btw, are you doing the feature extraction on the eye camera images (what we can see in drawImage1/2 when fdimgline is enabled) and gaze geometry estimation yourself?

I’ll message you on discord too, so we can continue the conversation there if you see fit.

Thanks again!

matsad · June 16, 2022, 8:24am

Thanks for the update, @mbucchia

drowhunter · July 6, 2022, 8:02pm

Hi, I was just doing a fresh reinstall of the Pimax software due to going back to windows 10 from Windows 11.

I noticed that I had downloaded 3 different versions of the aSeeVR software.

1.0.0.8 (173 MB)
1.0.0.10 (175 MB)
2.0.0.5 (98 MB)

which of these is the best to install before applying your fix?

guppyexpress · July 7, 2022, 5:04am

Hey ya!
so all of those are older versions of the fix haha. 3.9.1 is the latest and most recent version.
the instructions are on the github here
you want to download and run the Pimax’s ET install at first due to additional files that I dont have access to that need to be installed. That is located here

hope this helps

drowhunter · July 7, 2022, 3:19pm

Yeah , I realized that after reading the instructions.

The aSeeVR download from within Pimax is flakey and constantly fails to download but I noticed what the installer within Pitool said that it was trying to download 98MB which is the same size as the 2.0.0.5 version i had stored on my computer. So I ran that and it worked.

One question though, after installing your latest build and running the AMCap software the preview only shows a green 320x250 window.

On my previous W11 installation I could actually see my eyeballs.

any idea what’s up with that?

Arend · August 23, 2022, 11:07am

Here’s something weird I’ve discovered. I had some issues with my system crashing to desktop from various VR games after a couple of minutes (MSFS 2020, War Thunder, Star Wars Squadrons) in the past few weeks. Today when I wanted to know why my drive was nearly full, I saw that 40 % was taken up by a file named aSeeVRClient.log. I will try to find out whether this issue caused the crashes or vice versa. (Sorry I can’t keep this file for troubleshooting purposes, need my drive for work!) See WizTree image of my drive attached below: The big orange space is the .log file, 180 GB. When I deleted it, I had to close forward.exe, which apparently I hadn’t done after the latest MSFS CTD.

drowhunter · September 4, 2022, 4:48am

I’m gonna assume all of the purple squares are …ahem…nature videos and tax forms

guppyexpress · September 16, 2022, 3:52am

Hello Arend!
so the aSeeVRClient.log is a bug that happens where every error it logs it on a file. but it tends to be the same error with a different timestamp on loop for eternity. Currently working on fixing that. But it is quite time consuming on updating this especially because I have to decompile and recompile everything.
just as that bug happens, delete it until I can get that fixed lol