Guild Wars 2

Hopefully this doesn’t violate any rules or cause much friction. And TL;DR: It’s each of our ISPs, not NCSoft/Level 3.

I opened a support case for WvW Skill Lag. We all see it, at some point, often times when fighting in SMC blob vs blob. It’s something people who have been playing WvW for a lot longer than I have complained about with no resolution. I decided to open a support case on the issue. As many of you are aware, NCSoft is the hoster for GW2 and their servers are located in Texas (Dallas, if I recall correctly). This in and of itself isn’t the issue.

Level 3 is NCSoft’s ISP. This appears to be the core of the problem. My ISP is Comcast (Comcast Business class). That said, my guildies are using a variety of ISPs from all over the United States, Canada, and the world. We all experience skill lag (or outright lag) in a variety of ways. One of my guildies from overseas even only experiences lag in EBG, the map as a whole, under the right conditions. Borderlands are often lag free for this person.

But that said, ANet Support is of course our gateway to any issues regarding GW2. And this post isn’t a reflection on ANet Support’s response. They’re the voice of those who can provide a resolution when we’re discussing network or underlying service issues. ANet Support has been extremely supportive and patient looking into this and raising the problem to the right individuals within ANet/NCSoft. I do want to give them props for taking this particular ticket as far as they did; they did everything in their power to examine the issue with the data I was able to provide. Primarily, this consisted of providing rough timeframes for skill lag as well as traceroutes using their tooling of choice.

This ticket was open for almost 1.5 months and has come to a ‘conclusion’. ANet Support, per NCSoft engineers (service and/or network, I don’t personally know), have laid the blame at our ISPs, in as much as the email was almost templated as the first paragraph blamed Comcast, and subsequently went onto say “For the majority of you […] packet loss spikes are occurring at the point when Comcast routes your data onto our network, not after”. It went on to say “If you are not on Comcast, you are likely still being affected by similar problems on the part of our ISP, either right at the point when the data is being handed off from your ISP to ours, or even before then”. Again, this feels a bit templated, but I can understand why as NCSoft cannot blame one ISP as this occurs on ISPs across North America (and beyond).

Of course, the commonality between all of these users across all of these ISPs is the Level 3 hand off and of course NCSoft network/services. In fact, my WinMTR traces showed the packet loss occurring within Level 3’s network. Examples below, and they’ll be formatted poorly, so apologies in advance.

Columns are:

Resolved Address/IP | % | Send | Recv | Best | Avrg | Wrst | Last |

The percent is for packet loss percentage, while Best through Last are the round trip response time (RTT) in milliseconds.

Example #1:

|         <my rough location>.comcast.net -    2 |  263 |  260 |    0 |   20 |  169 |    0 |

|                              4.68.71.73 -   91 |  263 |   26 |    0 |   18 |   38 |   32 |

|                            4.69.210.141 -   97 |  263 |   10 | 24192 | 24870 | 25728 | 25278 |

|                             4.59.197.34 -    2 |  263 |  259 |   37 |   60 |  208 |   45 |
|                   64.25.32-9.ncsoft.com -    3 |  263 |  257 |   40 |   60 |  200 |   47 |

Example #2 (the ‘no response from host’ is the Level 3 handoff):

|         <my rough location>.comcast.net -    0 |   58 |   58 |   11 |   14 |   40 |   13 |
|                   No response from host -  100 |   58 |    0 |    0 |    0 |    0 |    0 |
|                            4.69.210.141 -   99 |   58 |    1 | 24633 | 24633 | 24633 | 24633 |
|                             4.59.197.34 -    2 |   58 |   57 |   65 |   74 |  164 |   67 |
|                   64.25.32-9.ncsoft.com -    0 |   57 |   57 |   65 |   74 |  172 |   75 |

Both of these examples are within the Level 3 network. Not on the handoff from my ISP, not on the handoff to NCSoft. Right in the middle.

But at this point, NCSoft via ANet Support has told me (and all of us who have ever experienced Skill Lag) that it is a problem with our ISP. It doesn’t matter what ISP you have or where you live, it is now up to you to negotiate with your tier 1, at best tier 2 support agents at your ISP of non-preference to get this ‘worked out’. As individual consumers, it is very difficult, if not impossible to not only get our issue across to the support agents at ISPs (reboot your modem, anyone?), but to actually get an ISP handoff issue resolved would be near impossible. Think of it like the peering between Netflix (Level 3) and Comcast. Imagine if I, as a consumer, attempted to contact Comcast asking them to improve that service at the peering/handoff. It simply would not happen. No, it will take the company, in this case NCSoft, to resolve that peering/handoff issue (even though it doesn’t appear as if that is the issue, again the problem lies within Level 3’s own network). NCSoft has significantly more leverage than we as individuals do with our own ISPs.

Apologies for the wall of text, but I thought I’d let everyone know the state of where NCSoft stands with regards to WvW Skill Lag as it has impacted a significant number of us at one time or another.

It’s pretty hard to be fed some numbers and to make sense of it. I do hope I got it right:
As far as I see it your hubs inbetween really are the problems and not the ncsoft servers. And anet can’t tell your povider which services they can use.
Packet loss makes the game play worse, yes, but it’s not the reason for the well known skill lags that occur when many people are on one spot.
And anet know the issue and they tried to work on it = lowered the map cap for some time and well, more didn’t happen. But they know that that very issue hasn’t got anything to do with packet loss. If you now show them some numbers which show packet loss on ‘your side’ then what should they say?

ETA: These are two ping plotter screenshots. My gameplay gets worse if no matter where packet loss occurs. In the first it’s my service that is the problem, in the second one of anet’s servers is lagging. The skill lag in wvw though occurs regardless of packet loss – its cause is that too many people are on the servers. You can however improve your game by getting another internet provider but still the skill lag will occur.

So the path everyone using North America servers will take is at least as follows. Your personal ISP -> Level 3 Communications -> NCSoft. There’s no way to avoid L3 as they are NCSoft’s ISP. The “lag” (latency) appears to be at the 2nd hop within L3’s own network, something you nor I have any leverage over as L3 is not our ISP.

That said, I’ve contacted Comcast and we’ll see if I get anywhere. I’m just not hopeful that we’ll come down to a resolution on an ISP backbone hand off. I pay $100/month for service… backbone handoffs are tens of thousands per month between ISPs (or free, depending on their agreement, with tens of thousands if not more invested in equipment).

It’s understandable that ANet/NCSoft would ‘blame’ your ISP if the latency occurred entirely within your ISP/network, it just doesn’t seem to be the case here. And the fact that it happens to so many people across North America just doesn’t sit too well with me. It’s certainly possible that the handoff between everyone’s ISP and L3 Communications is extremely poor, I wouldn’t dispute that, but then there’s not much we can do about it beyond NCSoft changing ISPs.

For what it’s worth, when I worked internal IT, we had a variety of ISPs for a significant amount of traffic and L3 Communications was the absolute worst of them in terms of performance.

And yes, this may not even be packet loss or latency issue whatsoever. But it is the only ‘evidence’, we as players, are able to collect. Heck, I’ve seen this same “issue” when standing at spawn no where near blob v. blob fights.

EDIT: I think it is also important to note, on a properly configured router/firewall, that ICMP 0 is, and should, be deprioritized compared to other traffic. Routers/firewalls will, under load, drop ICMP messages, either in part or entirely. 100% packet loss does not mean 100% of all packets are being lost — simply that it isn’t responding to ICMP 0.

Yeah. I have no idea about NA internet providers. I explained what my game does when there’s packet loss anywhere. And of course anet would say so if they’re right which they apperantly are in this case – futher explanation as ETA above btw – no idea if it helps.

yeah, the “standing at the spawn”: If there’s a blob anywhere on the map the whole map lags, so even you at the spawn. There’s a 2 year old thread somewhere in which a dev tried to get to the ground of it. He explained a lot actually. But I don’t really want to look for it right now – maybe someone else has got the patience to find it.

Hmm not sure I understand your standing at spawn comment. These “skill lagfests” only occur in close proximity to blob v blob (v blob), not when there just happen to be 2 or 3 blobs on the map.

I also edited my post with regards to ICMP from a net eng standpoint

Yeah. If there’s 2 blobs or 3 blobs fighting each other then the skill lags will happen for the whole map.
Actually try playing offhours sometimes, as a roamer – your game will always be a lot smoother than it is during prime time.

That’s kinda my point, the issue does not occur if one is some distance away from the blob v blob. We all die in this game and of course want to get right back into the middle of SMC. Respawning at spawn/keep and there’s no skill lag/lag what so ever until you’re approximately at the outer walls of SMC.

What should I say? Your experience versus mine.

I’m not disputing your experience, don’t get me wrong. I’m just relaying the behavior I’ve seen and those on TS have seen with regards to these large scale fights.

Yes, at the one hand you say it even happens to you when you’re standing at spawn, on the other that it only happens when you’re right next to blobs, so which is it?
I hope I clarified your initial issue. Whatever lags you might have might also be because your connection is lagging, so it’s very hard to tell what the reason for your specific lags are.

Ah I see, poor wording on my part. The latency/‘packet loss’ I’ve seen anywhere and everywhere. The skill lag I’ve only seen in close proximity to blob v blob, never when blob v blob activity is happening but one is some distance for it.

Just curious, why would you submit a ticket on a known issue that has been discussed, and responded to by the devs, many times?

Ni In.6578:

That’s kinda my point, the issue does not occur if one is some distance away from the blob v blob. We all die in this game and of course want to get right back into the middle of SMC. Respawning at spawn/keep and there’s no skill lag/lag what so ever until you’re approximately at the outer walls of SMC.

In my experience, skill lag occurs across the whole map when there’s a big fight going on. Trust me, when walking back to SMC from spawn, I use swiftness/leap skills to get back quickly, and these skills will often take several seconds to cast.

Swagger.1459:

Just curious, why would you submit a ticket on a known issue that has been discussed, and responded to by the devs, many times?

Why would you create a topic about what anet is working on wvw when it’s been known for some time since the last poll and mentioned by the devs a couple times since? And we’ve been told many times they won’t discuss future content.

If the OP is correct then somebody explain to me why I don’t get skill lag or other lag? Not even in blob situations. I run mid to high graphic settings, nothing low and run GW2 on an ASUS RoG laptop. My internet speed is 60mbps.

If the lag is the fault of ANet then why am I not getting it?

Balthazzarr, can I come live at your place?

Xenesis.6389:

Swagger.1459:

Just curious, why would you submit a ticket on a known issue that has been discussed, and responded to by the devs, many times?

Why would you create a topic about what anet is working on wvw when it’s been known for some time since the last poll and mentioned by the devs a couple times since? And we’ve been told many times they won’t discuss future content.

Well, there is obviously a big dry spell, so I was fishing for more information to help inspire the troops and improve attitudes of people on this section.

Is that an acceptable answer?

I have skill lag in EotM very very rarely. I think that all normal WvW maps are same kitten or something and that is reason why it lags.

Balthazzarr.1349:

If the OP is correct then somebody explain to me why I don’t get skill lag or other lag? Not even in blob situations. I run mid to high graphic settings, nothing low and run GW2 on an ASUS RoG laptop. My internet speed is 60mbps.

If the lag is the fault of ANet then why am I not getting it?

Define “Blob situations” – people have different meaning for this. Maybe you just have not met the typical “skill lag” (a 50vs50vs50 SM fight is pretty good chance for that, while 70vs70 is not – for some reason need 3 parties for true skill lag)

Ni In.6578:

That’s kinda my point, the issue does not occur if one is some distance away from the blob v blob. We all die in this game and of course want to get right back into the middle of SMC. Respawning at spawn/keep and there’s no skill lag/lag what so ever until you’re approximately at the outer walls of SMC.

That’s not true though.

If there’s a big fight in SMC it doesn’t matter where you are in the map, you’ll lag.

People who scout/roam can attest to this.

Hubal.8571:

Balthazzarr.1349:

If the OP is correct then somebody explain to me why I don’t get skill lag or other lag? Not even in blob situations. I run mid to high graphic settings, nothing low and run GW2 on an ASUS RoG laptop. My internet speed is 60mbps.

If the lag is the fault of ANet then why am I not getting it?

Define “Blob situations” – people have different meaning for this. Maybe you just have not met the typical “skill lag” (a 50vs50vs50 SM fight is pretty good chance for that, while 70vs70 is not – for some reason need 3 parties for true skill lag)

I’ve been in situations where there were 3 of our groups in the same spot at the same time, minimum 40+ each with no issues. I listen and watch the map chat as people scream about the lag and think to myself… thank goodness I am not wherever they are.

Jayne.9251:

Ni In.6578:

That’s kinda my point, the issue does not occur if one is some distance away from the blob v blob. We all die in this game and of course want to get right back into the middle of SMC. Respawning at spawn/keep and there’s no skill lag/lag what so ever until you’re approximately at the outer walls of SMC.

That’s not true though.

If there’s a big fight in SMC it doesn’t matter where you are in the map, you’ll lag.

People who scout/roam can attest to this.

Agreed, I find the smc fight affects my roaming and scouting, doesn’t matter where I am on map.

On more than one occasion ANet has come out and said it has nothing to do with networks but event propagation on their servers. If it were a network issue it’d be just local to certain players within vision of the fight; the often-cited complaint of major skill lag affects all players on the map regardless of location, and this explanation also explains the skill speedups that often come afterwards. If it was network-related, things wouldn’t speed up.

It’s very possible that some skill lag can be caused from network problems, but to say the infamous “skill lag” aspect of WvW blobbing is network-related just doesn’t seem to add up unless outgoing communication from ANet itself is being bogged down… but I sincerely doubt that considering all events happening at all points in the game on all servers in all map instances are also going outbound in the same way, and the population is way lower than it used to be, suggesting this is really, really unlikely.

The lag was noticably lower when they lowered the server cap and where stabi was not a pulsing/stacking boon yet.

Caedmon.6798:

The lag was noticably lower when they lowered the server cap and where stabi was not a pulsing/stacking boon yet.

Since HoT, there have been a lot more proc effects in the meta and many more players blobbing in one huge cluster due to the death of small-scale and havoc, on top of lower cooldowns and more cleave happening ant any given time. This would naturally cause a lot more congestion of events being fired.

Caedmon.6798:

The lag was noticably lower when they lowered the server cap and where stabi was not a pulsing/stacking boon yet.

Well the lag is noticably lower when 3 50-man zergs dont clash either, so theres that.

I have feeling that eb lags if there is big fight in borderlands. I have lag in eb when i using trebs in keep and there isn’t even any big swords in map.

Totally our fault when 50 of 50 Squadmates say “I can’t use any skills, god kitten EBG lag!”

Insert random “you serious?” meme

Wish I knew exactly what you’re talking about. I’m a layman, but it doesn’t change the fact that skill lag only ocurrs when there are big fights to everyone around the world, and not randomly when there are no big fights on the map. Whether it is with their programming, servers, or local networking, it’s Anets fault. I don’t know which, but it doesn’t matter. It’s on their end.

You say that this occurs at the stage in between NCSoft and everyone else, so they’re not responsible. Well, no, they still are. If that equipment is out of their hands, then it’s up to them to make a game that works within the limitations of the current state of internet technology. If this is something that they can have some control over via negotiations with the folks that send out and receive their data to the world, well, that’s still on them.

It’s just common sense.

Spurnshadow.3678:

Wish I knew exactly what you’re talking about. I’m a layman, but it doesn’t change the fact that skill lag only ocurrs when there are big fights to everyone around the world, and not randomly when there are no big fights on the map. Whether it is with their programming, servers, or local networking, it’s Anets fault. I don’t know which, but it doesn’t matter. It’s on their end.

You say that this occurs at the stage in between NCSoft and everyone else, so they’re not responsible. Well, no, they still are. If that equipment is out of their hands, then it’s up to them to make a game that works within the limitations of the current state of internet technology. If this is something that they can have some control over via negotiations with the folks that send out and receive their data to the world, well, that’s still on them.

It’s just common sense.

Well its not only common sense, Anet coders themselves has said that the skill lag is due to the serverside skill queue choking under pressure.

As a sidenote: NCSoft is neither the publisher or developer for GW2. So technically no, its not their fault.

Spurnshadow.3678:

You say that this occurs at the stage in between NCSoft and everyone else, so they’re not responsible. Well, no, they still are. If that equipment is out of their hands, then it’s up to them to make a game that works within the limitations of the current state of internet technology.

That would be an offline game, then. And I guess you still would blame anet if your operating system is bugged.

There’s 2 kinds of lags in this game – one because the connection has got packet loss and one because the game engine can’t handle too many players. The latter is anets issue and they admitted to it. What the OP complained about though was his internet connection, so it’s not surprising that anet pointed out that his connection is kitten. Different providers use different ‘sub providers’ – so while the one uses one that’s throtteling their servers, the other doesn’t. No idea how the laws in the USA are, maybe it’s perfectly legal for services to slow down the data streams of their customers. So what should anet do about that?

You’d think that after 4 years they would have adressed this game breaking issue.

Dawdler.8521:

Caedmon.6798:

The lag was noticably lower when they lowered the server cap and where stabi was not a pulsing/stacking boon yet.

Well the lag is noticably lower when 3 50-man zergs dont clash either, so theres that.

We had 3 way sm fights with no lag..So yeah theres that.

The thing about the 3 way EB lag is that it lags like hell, the game is unplayable, yet ping remains in 25-60 range for me. I’m no expert but it seems like a problem on their side, not mine.

I’m pretty sure that a few years ago they even admitted that it’s an issues with their servers and they cannot do anything about it.