You will see from the replacement that the plug is actually about 3-4cm long, not just into the back of the hmd, it slides right through the hmd.
You need quite a bit of steady even force right up at the plug near the end of the cable, not yanking on it, and heaps and heaps of wriggling side to side.
When the new one goes back in there is a positive click as it finally seats back in.