This is what VR game developers need to add, to remove clipping/culling in peripheral view on Pimax

SweViver · March 24, 2020, 9:16am

No news really, but just want to remind you that the “clipping” or “culling” you see in peripheral view on certain games in Normal/Large FOV when not enabling Parallell Projection, can only be solved by the game developers with a few lines on code, at least in Unity engine:

void OnPreCull()
{
        //move the camera back 2m to cater for large fov
        Matrix4x4 worldToCameraMatrix = Matrix4x4.Inverse(Matrix4x4.TRS(m_Camera.transform.position - m_Camera.transform.TransformVector(2 * Vector3.forward), m_Camera.transform.rotation, new Vector3(1, 1, -1)));
        m_Camera.cullingMatrix = m_Camera.projectionMatrix * worldToCameraMatrix;
}

A German company & game developer I visited last month fixed that issue in their Unity game, within a few minutes.

So whats the point of this thread? Simply help out and make the developers aware of this fix!

I know many games like Half Life Alyx aren’t made in Unity, but I suspect their engines arent that different after all…

Original reddit post about this topic:

wilcovertegaal · March 24, 2020, 9:38am

Shouldn’t there be some “if (fov > 120)” statement? You don’t want to do this if you don’t have to, because the clipping/culling is there to speed up the rendering.

SweViver · March 24, 2020, 9:44am

Yes the best solution would be to enable this only when Pimax is detected, or a wide FOV is rendered.

risa2000 · March 24, 2020, 10:41am

Martin, the problem is not with the wide FOV per se. The total horizontal FOV of Pimax headset in particular configuration (Small, Normal, Large) is the same whether you turn the parallel projection on or off.

The problem is that when the PP is not on, the game/engine incorrectly calculates the culling matrix, because it is confused by non-parallel cameras. My bet would be they just take peripheral FOV of each camera and add it up to get the total FOV while forgetting that the cameras themselves are angled and thus end up to be 20° short. Which, for some reason I do not know (and can only guess), affects the right end of the peripheral view.

The proposed fix is more like a hack and I would not definitely recommend to anyone to use it in his code, because what it does is giving you a false perception of the wider FOV by moving (the supposedly) narrower FOV behind you. This can work to some extent, but eventually the wider FOV meets narrower FOV regardless how far you move it and the problem will still be there, plus it skews the culling in very close distance because it suddenly accepts even the objects right next to you or even behind you.

jojon · March 24, 2020, 3:51pm

Exactly the thought I immediately had, when the developer (I am assuming it is the same one) posted it on reddit, a few months ago. Something of a brute-force workaround, that helps with this particular target FOV now, as long as the thing we do not wish to cull is closer to the camera, than the point where the still-too-narrow-on-one-side-but-shifted-back culling frustum intersects the view one, but does not address the underlying issue.

drowhunter · March 25, 2020, 1:27am

you’re telling the wrong people. Post it on the steam forums, or better yet since you guys are “working closely” with valve maybe they can force it on or something

koochy_rat · March 25, 2020, 4:00am

That’s me. I’ve reposted my fix in the Pimax reddit

Yes, it is a hack, but I could not find a better or easier solution. Either Unity/SteamVR/Pimax has to fix something or we have to do it ourselves. It is not a perfect fix because the frustum still intersect some distance away, but it is usually too far to be noticeable.

risa2000 · March 25, 2020, 7:51am

Pimax and SteamVR are fine. The only thing which needs fixing is apparently Unity.

Ok, let’s try to do it ourselves .

If what you posted above:

m_Camera.cullingMatrix = m_Camera.projectionMatrix * worldToCameraMatrix

is your typical OnPreCull hook (not considering the fix now) then chances are you need to fix your Camera.projectionMatrix not the worldToCameraMatrix transform.

You only need to fix it when the camera views are not parallel (because with parallel camera views = when “parallel projection” is on, the culling works fine, so the projection matrix is calculated correctly).

To test for non-parallel cameras, you just need to check that rotation part of Camera.GetStereoViewMatrix is an identity matrix. If it is, the cameras are parallel, if it is not, the rotation defined by the matrix gives you the camera angle you need for your fix.

Once you have the camera angles, you need to calculate the new projectionMatrix (for the head camera) by using the frustums of the stereo cameras. This part might be a tricky one as I am not sure there are raw camera/frustum tangents available in Unity (I am not that versed in Unity). If not, the worst case, you would need to calculate them from projection matrices (Camera.GetStereoProjectionMatrix).
What you need in your “new” head projection matrix is:
- horizontally: left camera left FOV + left camera view angle + right camera view angle + right camera right FOV
- vertically: technically you need to transform the angled vertical FOV values from the stereo cams (which are angled) into the vertical FOV for one (head) camera looking ahead, which is not the same. But I would first try just using the original stereo camera values as the difference should not differ much.

You may optimize the camera angle calculation to single scalar equation (as the axis of rotation is known) and you may also optimize extracting the raw frustum values from the projection matrices (as you need only the peripheral tangent).

I do not have my 5k+ now, nor I am as proficient in Unity, but if you need a help with the math, let me know by PM.

SweViver · March 25, 2020, 9:23am

Thanks for the input guys! Wow this is waay over my head, to be honest Yeah it seems to be a workaround, but at least its a solution for now.

So if the FOV is incorrectly calculated having PP disabled, is there another way to fix this on our side, or is this just something the developers have to fix in the games?

risa2000 · March 25, 2020, 9:50am

The culling matrix (which is the problematic part as it seems) is calculated by the game/engine. There is nothing Pimax (or SteamVR) can do about it, except going to the devs and explaining to them, how they should fix it. I suspect the fix will be along the lines I posted above, but without knowing the implementation details (of the particular game/engine) this should be taken as a hint rather than a proven solution.

koochy_rat · March 26, 2020, 6:40am

Thanks for the detailed explanation! I took a look at the problem again and it looks like Unity never reports the actual fov, whatever it reports is about 130 degrees. There is some low level trickery going on at the lower level that somehow maps that to the full 170-180 deg fov. See this screenshot. Right eye is red, Left is green, Blue is head. This is far smaller from what is actually visible in the headset.

Here are the matrices. If you understand the math, please help

//Projection matrix
[
0.3611111, 0, -5.407921e-08, 0,
0, 0.7875, 0, 0,
0, 0, -1.000199, -0.1990233,
0, 0, -1, 0
]
//Left eye projection matrix = Right eye projection matrix
[
0.4859813, 0, -0.3457944, 0,
0, 0.7875, 0, 0,
0, 0, -1.00012, -0.1200072,
0, 0, -1, 0
]
//left view matrix
[
-0.9846469, -0.0194595, -0.1734711, 2.438025,
-0.02204913, 0.9996722, 0.01301362, -1.181422,
-0.173161, -0.01663871, 0.9847533, -13.00945,
0, 0, 0, 1
]
//right view matrix
[
-0.9844899, -0.02397672, 0.1737958, -2.232084,
-0.02204914, 0.9996722, 0.01301362, -1.181422,
0.1740509, -0.008979731, 0.9846959, -13.04576,
0, 0, 0, 1
]
//culling matrix
[
-0.3610233, -0.007963635, 5.948122e-05, 0.03775821,
-0.01736369, 0.7872418, 0.01024823, -0.93037,
-0.0004518941, 0.01300941, -1.000114, 13.01596,
-0.0004518042, 0.01300682, -0.9999154, 13.21235
]
*/

EDIT: Removed potential fix, it is not working properly and I got overexcited.

koochy_rat · March 27, 2020, 7:48am

Ok, thanks to help from risa2000, here is the new improved proper fix It is a general fix for all headsets with canted displays in non-parallel projection mode. The conclusion is that Unity calculates the projection and culling matrices without accounting for the eye rotation angle, causing it to come up 20 degrees short in the case of Pimax. I will be sharing this on various dev forums in the hopes other devs will pick this up.

Github link for this code: GitHub - koochyrat/SteamVRFrustumAdjust: For canted headsets like Pimax, calculate proper culling matrix to avoid objects being culled too early at far edges in Unity. Prevents objects popping in and out of view

using UnityEngine;
using Valve.VR;

//Place this script on the Camera (eye) object.
//for canted headsets like pimax, calculate proper culling matrix to avoid objects being culled too early at far edges
//prevents objects popping in and out of view
public class SteamVRFrustumAdjust : MonoBehaviour
{
    private bool isCantedFov = false;
    private float m00;
    private Camera m_Camera;

    void OnEnable()
    {
        m_Camera = GetComponent<Camera>();
        HmdMatrix34_t eyeToHeadL = SteamVR.instance.hmd.GetEyeToHeadTransform(EVREye.Eye_Left);
        if (eyeToHeadL.m0 < 1)
        {
            isCantedFov = true;
            float eyeYawAngle = Mathf.Acos(eyeToHeadL.m0);  //since there are no x or z rotations, this is y only. 10 deg on Pimax
            float eyeHalfFov = Mathf.Atan(SteamVR.instance.tanHalfFov.x);
            float tanCorrectedEyeHalfFov = Mathf.Tan(eyeYawAngle + eyeHalfFov);
            m00 = 1 / tanCorrectedEyeHalfFov;  //m00 = 0.1737 for Pimax
        }
        else
            isCantedFov = false;
    }

    void OnDisable()
    {
        if (isCantedFov)
        {
            isCantedFov = false;
            m_Camera.ResetCullingMatrix();
        }
    }

    void OnPreCull()
    {
        if(isCantedFov)
        {
            Matrix4x4 projectionMatrix = m_Camera.projectionMatrix;
            projectionMatrix.m00 = m00;
            m_Camera.cullingMatrix = projectionMatrix * m_Camera.worldToCameraMatrix;
        }
    }
}

SweViver · May 26, 2020, 7:48am

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.