This argument boils down to “Well it’s happening already. So let’s just do whatever.” I agree that people harass other people and part of what we try and do is mitigate that. So, we wouldn’t make a change that would exacerbate that, unless it had a major pay off. This doesn’t fit that criteria.
A change that would make WvW so much more enjoyable for servers that are completely outnumbered should entirely fit the criteria you just mentioned.
You don’t have to make it such an overpowering buff that there would be an advantage in having less players. Something to help level the playing field a bit would be nice though.