How did you settle on VR for the headset?
>After briefly thinking about an implementation with an LCD screen behind an eye piece cutout, connected to an SBC like Raspberry Pi which in turn would be wired up a rotary dial, we ditched the idea in favor of using a VR headset. Basically all of the above in one unit.
Seems like overkill compared to HDMI FPV Goggles? Something like these? "Bigeyes H3 Portable 2.5K Equivalent Screen Video Glasses with HDMI Input". It doesnt seem like you were using the VR elements of the goggles, and they caused some issues with their extra smarts.