I have tested this and the result is always the same no matter which character I play. Removing dies on the character screen without replacing them results in a crash to desktop (hereinafter, CTD).
I go to the character screen, pull up the die choices, remove die to go back to original colors, and as I attempt to exit from the character screen I get a CTD. It applies to street clothes and armor. Using “undo changes” will not stop the CTD. I get the CTD even if before exiting the screen I switch from armor to street clothes or vice versa. Replacing the colors in each slot with dies I have prevents the CTD.
Luckily, this bug is far from game breaking.