Zanko, your view of PvP seems to me limited to dueling and X on X fights. Realm v Realm, & world v world is certainly players fighting players however this game requires strategy, tactics and team coordination to win. This is much different than optimizing gear, your spec, and seeing how fast and coordinated your fingers are.
If your looking for a challenge in the gear prepping and reaction time of your fingers sure small man is the challenge you want, if your looking for a challenge in organization, planning, execution, communication and leadership then perhaps WvW is better. Both are entertaining but certainly each to his own. I personally find the WvW having a more complex and enduring challenge.