This solved it for me.
Control Panel → Manage Audio Devices. Disable all devices except the one that I was using (Speakers High Definition Audio Device) and reset audio settings in gw2.
Usually this happened after every 2-3 big fights in WvW and always sometime during the Jormag fight. Now I haven’t had any issue for a whole day.