Guild Wars 2

I have noticed that the buff does not show when we are outmaned.

Or maybe our server just sucks and our towers and keeps get taken by 10 man zergs.

How do you know 100% that you were Outmanned? Did you gather all your server WvW players and counted them, asking your opponent to do the same?
Maybe there were like 10 of your server doing a JP (in one of the homelands) or so..

Remember, there is more happening in a map than you can see on your screen.. :/