PassThrough Camera for Pimax (Homemade)

Hi guys,
I designed this, its a pass through camera for Pimax headsets :


It is very light, weight 10 grams.
To be plugged on the lower usb C port, so light that it stand on the headset by itself once plugged in.
Here it is on my headset, can be used with or without the silicone protection.

Unfortunately it requires a third party app like OVRdrop to be displayed in the headset using the Camera app from Windows 10 (to have a windows to display the camera so OVRdrop add it in the headset)
@SweViver or @arminelec could probably easily add a feature in PE to display it but it would not work for SteamVR.
Using Unity it should not be that hard to make a software to run in like OVRdrop, just for the camera app of Windows 10 but I dont know if it worth it, maybe if a very lot of you are interested …
I will make a few more (4 or 5 but can make dozen if needed) that I will be selling once i’ve done every tests with it, mostly trying to find something else that the Camera app of windows 10 to have access to the camera settings but I am proud to show it already.
It is a 720p/30fps camera so it need light in the surrounding to see, of course an IR one would be better but much more expensive and maybe heavier.

ezgif-4-10ccbecd61e0
It can be useful in many situations ^^

18 Likes

Reality Blender in Steam this could be very handy for sims.

Very nice!

:beers::sunglasses::+1::sparkles:

3 Likes

Great work @Ludiks… The concept is certainly helpful. I hope you will succeed at it! :slight_smile:

I have added it to the backlog. @SweViver and I will discuss it together during our next prioritization phase (after the release of the Cloud Profiles).

4 Likes

Thank you @Heliosurge @arminelec :man_bowing:

3 Likes

:slightly_smiling_face:
this is near to what I was searching for. with a good software integration, one may reach a certain level of AR.
with two cameras one may even reach stereoscopic view and may grab / scan / measure / copy things from reality and take over in virtuality. so cool.

2 Likes

I like it but I’d rather not have to choose hand tracking OR camera. I’m wondering if you can modify the housing to handle both? The hand tracking module does not seem to have a pass thru usb port, so it would have to go on the bottom, I would think.

1 Like

@3D_FAN thanks ^^ yes using 2 cameras would probably be possible (using a micro hub inside the housing to use on a single usb port) but the hard part would be the software integration yes.

@Strydr Yes unfortunately cant use both on the low usb at the same time.
Using on one the top usb would require some new design but right now I dont see how we could attach it to the headset housing, but it is doable for sure.

3 Likes

about a year ago I had the same idea and immediately ordered a 3d camera,
which comes close to the resolution, but only delivers max 60fps to me.
with the reality blender this also works very well with “chroma key”. (purely for passthrough)
But for real AR applications you would need an intel real sense camera, as the image depth of individual objects can be calculated and thus AR can be made possible.
or maybe the combination of a 3d camera, the hand tracking module and the eye tracking module and of course a brilliant software solution works :wink:
Unfortunately, I also fail because of that … if you don’t have your hand at a certain distance from the camera lenses, for example, the 3d effect becomes blurred.
the focus must be controlled automatically by software in the background. maybe it works with the help of the leap motion sensors?

4 Likes

what a great idea!
It has bothered me for a long time that I can’t quickly see into real space.
I didn’t know the OVRdrop app, but have now found and tested the Daily OVR App.
Works fine, turn the controller and I see e.g. the camera or whatever.
I have to switch on my 3D printer and create a camera module.
Thank you for the really good idea, it put me on the right track …

Greetings
Uwe

1 Like

Nice !
Do you have a link to those cameras please to see if they are the same as the ones I can get.
I though they were too big but I am trying to find a single one for 60 fps use, they dont seem that big on your headset.
I actually wanted to do this for long now, but I had to deal with really hard time for more than a year now…

@UweP thank you !

Working on a few more ^^

1 Like

Yes, no problem, this is the model: ELP synchronization 3D USB 2.0 MJPEG 60 fps 1.3 MP UVC mini webcam dual lens stereo video USB camera
and I ordered it from amazon.

But there are even better options, because unfortunately this camera does not cover the entire field of view, 2 wide-angle cameras would be better,
but would also be a bit more expensive, the question is: What are you willing to invest?
You have to look, there are always advantages and disadvantages.
But you get a good, smooth picture with it and it is completely sufficient for the pure passthrough function.
Unfortunately I haven’t had time to make a video yet, I’ll be recording it on the weekend
I wish you every success

1 Like

Maybe you and Sweviver can convince Pimax to develop a Stereo path through camera for the Pimax Headsets and add it to the standard software (PiTool and StemaVR). or does the hand tracking module provide this functionality?

If not, my rating of usefulness of the Pimax Addons would be this:

  1. Wireless Module

  2. Passthrough (combined with hand tracking if possible)

  3. Hand Tracking

  4. Eye Tracking

Eye Tracking being 4th in the list only reflects the current state of support by drivers, APIs and games. That might also be the reason why nobody else provided it so far for end customers (B to C market). Same for Hand tracking

The more useful addons are lacking in my opinion as they would give an instant benefit to everyone and are not dependent on the support of the entire global players in the market (when done right of course).

@Ludiks: nice job!

3 Likes

As a possible customer, sorry, I have a few problems with this.

Hand Tracker compatibility most of all. I would need this to have a side-mount USB connector or something, to connect to a separate accessory USB extension cable.

Parallax seems to be another problem. Having briefly tried the LeapMotion camera demo thing, which basically does passthrough from the hand tracking IR cameras, I found the perspective from the bottom of the headset less than ideal. Of course the hand tracking itself compensates for this in software, however, for camera passthrough, I would rather have a perspective of ‘eyes’ mounted to the front of the headset.

Software needs to be convenient. I need to be able to say ‘construct/destruct passthrough’ to VoiceAttack, and have the camera passthrough just work. Ideally, a huge transparent overlay, somewhat like the original Vive offered.

At the end of the day, the convenience of the MAS today, and PimaxExperience, reduces the value of camera passthrough somewhat overall. I find I no longer even need to take the headset off much, and when I do, it no longer takes so long to re-align the lens sweet spot, and is no longer so difficult to avoid smudging the lenses. Having hinges set to the proper 161deg angle makes the ratcheting mechanism at the back of the MAS work amazingly fast with very little force.

Thank you :man_bowing:

Dont think so if you have HT you might not need this, maybe HT will provide such feature anytime soon.

Dont be

So do I.

I dont have HT, this is part of why I designed this, I had a free USBC port.

Of course, but this is not the purpose of this, this is not made to be a new AR device for Pimax but just an helper to see what is going on around, what is the cat/dog doing, where is the vape or the coffee cup etc…

This is one of the thing that is bothering me, I am totally aware of this but some people are using constant overlay inside their headsets.

I didnt say it was a game changer and a must have accessory.
I like make/create things, I made that and have a few spare, plus I didnt mention any price yet.

6 Likes

Thank you for understanding.

For me, even for that use, I don’t want parallax shift. Thus I do not really consider the HT cameras adequate either.

If one really wanted to go wild experimenting with correct camera placement, one could try the little trick from one of the patent applications from Jeri Ellsworth’s days at Valve, which would have you attach each camera to a little periscope, in order to add the Z distance from the front plate of the HMD to the user’s eyeballs… :9

I was thinking the same thing when I saw this. Even if this is not intended as an AR pass through device, having the cameras in ‘eye position’ could make applying an offset less crucial. This is more than just a vertical shift, as tilting your head rotates the view at a different offset from the pivot point(neck)

Does anyone know if any of the HMD sensors would be occluded with the board? The silicon sleeve does have some cutouts in this area, but it looks like some could be cosmetic.

1 Like

Yeah well at least I made something, ask Pimax for a real one.
I plan to sell the ones I have for around 40$, I made something simple and lightweight and cheap.

2 Likes

Zed Mini i think does that but is intended for ar use.

Actually I can make one that fit on the front side of the hmd with the camera set vertically, instead of a bar shaped it would be some kind of L or U, to plug on the lower usb port still.
I will see what I can do.
With ovrdrop you can flip/tilt the image so it can be used this way and since it is very small i don’t think it block the tracking.
I will design and try this when I find a moment.

4 Likes