Recommendation for future projects: better use eye tracking for retinal resolution

@Matthew.Xu, @bacon:

For future projects, I hope that Pimax doesn’t think of getting involved with two 8K panels for a Pimax 16K; it would take years to come out with a capable graphic card and CPU, and even if it were possible, would be a very, very huge waste of hardware. Better rely on a foveated system to increase the resolution, as does the company Varjo.

Varjo projects two small images from Sony Microled 0,7” 1920 x 1080 panels, on the focused spots (with eye tracking) of two standard Oculus Rift 1080 x 1200 panels, to accomplish human retinal resolution in all the headset FOV, in the prototype case, a Oculus Rift, 110º.

Varjo also offers augmented reality.

The real snag is the current cost of the microleds and that it doesn’t have a higher FOV, they also have the same problems of Pimax with the foveated rendering.

If it were associated with someone with experience in panels and lenses to get a large FOV of 220º, it would be my ideal product at the image level. A Pimax-Varjo-Adhawk colaboration?

The only way it could improve would be correcting the user’s visual problems.

Be careful Pimax, Varjo could already be thinking of doing something like that, just waiting for cheaper Microled panels.

My article in Spanish in Real o Virtual.

1 Like

Have you seen the screen shots of the Varjo prototype ? I can see that this may make sense for certain commercial applications, where function is all what counts, but it was pretty ugly if you imagine this to be worked into a VR HMD for playing games etc… You would have a box with awesome resolution, and then a brutal cut to standard resolution outside this box.

Hard to imagine you can develop any proper immersion with that sort of falling-of-a-cliff experience in terms of resolution.
The major difference here is, with foveated rendering you see full resolution anywhere on the screen, whatever point of the display your eyes are gazing at; even if it is on the edge of the display. With the Varjo you will be looking at high res if looking down the middle, but as soon as you leave it a bit more, suddenly you see a clear line where the resolution drops significantly.

No, wherever you look, it’s retinal resolution. It’s a Oculus Rift with retinal resolution.
In this case, it’s 110º because that is the FOV of a Oculus Rift.
Imagine this in a Pimax with 200º.

As far as I understood it, and just checked an article of RoadtoVR from the time, they suggest to have their high-res display in the center of the view for the 70x super-duper resolution, but then around it you would have a traditional VR display to get a proper FOV, and that would then only come at say 1x, 2x or 4x resolution. So you will inevitably get a border where the resolution drops dramatically:

“Varjo however claims an effective resolution that’s nearly 70 times greater than today’s headsets. They’re doing so with an interesting display layout which combines typical VR displays with higher density micro-displays to make part of the image hyper-sharp. The company says that portion of the image achieves retinal resolution, meaning that the pixel density is so great that you can’t tell individual pixels apart.”

1 Like

That was with the initial prototype, now they have one in which the eye tracking already works. They even invited the prince of Sweden to try it.
Let me find you the article.

Ok, was not aware of the newer developments. Yes, please share additional information on it.
I find it difficult to get clearer information on the subject. From a comment on an article it sounded as if they intend to make the micro-display, i.e. the hig-res display, movable ?! I cannot image that that would make sense, it would have to have lightning speed to keep up with your eye movements. And how else is it supposed to work, unless they have a 8K or 16K display after all…

1 Like

Sorry, is mine in spanish:


Prince William of Sweden was able to test the Alpha prototype during the Slush 2017, where it was shown with functional eye tracking.
The first prototype will come out during the first quarter of the year.

I found this one:

If I understand it correctly, the micro-display hig-res picture is projected to the pont where the user is gazing at. Wow. That sounds - pretty complicated to get it right. But if they did, it would of course be gorgeous. However, I am not entirely sure that such a technically compplex solution will be any cheaper than simply having a retina screen all over the FOV and apply the “traditional” foveated rendering to it… but it’s great to see that innovations are happening out there !

1 Like

The advantage is that you save dealing with so much data. With a relatively weak card you could have a 120hz screen refresh without problems and the graphics card would only have to deal with a resolution of twice the current one of the oculus rift, plus two full HD extra.

How they supply microdisplay with signal?
For example we (or i) have 1920x1080 per eye (100fov) and 4k micro display projected over some areas, where do i get 2 x 4k, what gpu do i need. Same field of problems currently PiMax working on? Bridge chip?

It is not known, but it is believed that it projects the fullHD microLED image with some kind of mirror, in a small point on the screen of 1080 x 1200, that for each eye.

In any case, with screens of much less density than those used by Pimax, we would see a much higher resolution, so it would not be necessary to use such a powerful graphic cards.

I would reccommend checking out Nvidia Cascsded Displays. @VRGIMP27 shared with us a awhile back. Its possible this could be used without going to 8k panels maybe with a frame doubler/clone?

Its an older article about potenial of using lcd for vr.

http://community.openmr.ai/t/what-about-the-pimax-5k/3016/22?u=heliosurge

1 Like

This is rather old, I barely remembered. The last thing I know about the developer is that he was working with Oculus.

Yep it is old. I am guessing one method to offset would be to rotate one display 180° so that the sub pixel pattern is overlapped ie the 4k has kinda rainbow arcs/swirls. But might be interesting to re evaluate the idea.

1 Like