Running your VM's from VirtualBox from a command line system (read Ubuntu Server or similar) is quite a challenge. The problem being is that systemd only likes to run in a graphical system (init5)
Here is how to make it run in init 3 --> https://rudd-o.com/linux-and-free-software/how-to-make-pulseaudio-run-once-at-boot-for-all-your-users
Here is another source of info, not sure if it is applicable or not --> https://askubuntu.com/questions/1071532/systemd-disable-pulseaudio-system-mode
Another way to run system-wide --> https://raven4.cz/wp/pulseaudio-in-system-wide-mode/
Arch Wiki's page on pulseaudio --> https://wiki.archlinux.org/index.php/PulseAudio
This is closer to what I want --> http://serendipity.ruwenzori.net/index.php/2015/06/01/sending-an-audio-stream-across-the-network-to-a-remote-raspberry-pi-with-pulseaudio-the-easy-way
This is the source of the above --> http://possiblelossofprecision.net/?p=1956
This is to do with bluetooth --> https://github.com/davidedg/NAS-mod-config/blob/master/bt-sound/bt-sound-Bluez5_PulseAudio5.txt
How does pulseaudio start --> https://unix.stackexchange.com/questions/204522/how-does-pulseaudio-start
In the end of the day, all I did was disable /usr/bin/start-pulseaudio-x11 (by sudo chmod -x) and then added a line to my users .bashrc systemctl --user start pulseaudio No need to add user to any groups other than vboxusers