Hardware and content is still the big issue. The good porn games still suck in VR, and there's not a lot of them. The equipment is just too inconvenient.
Your hands are occupied, your positions are restricted, your tethered to the PC, and I don't want to get a thousand dollars of delicate hardware nutted on. It's just not there yet.
The guys using FOSS Ai would be the same guys using an operating system without an hostile Ai built in.