It keeps happening and I cannot pinpoint what exactly causes it to happen. I just accidentally quit the game rather than going to character select and upon relaunching it using the shortcut in the taskbar that points directly to the executable, it was windowed. Then I set it to windowed fullscreen, relaunched it and it was fine. I usually use either windowed fullscreen or fullscreen. When things work properly, the game launches as a window then becomes windowed fullscreen. It looks like sometimes it fails to become windowed fullscreen and stays as a window.
I have recently started using WindowBlinds, although I think the problem has been there for a longer time. I am on Windows 10.