Unfortunately the availability of the demo slot is tied to your computer’s clock. (Whether you can actually get in is up to the server.) . So check your computer’s clock. (This might explain why you saw the slots early and they disappeared.)
Sorry, we disabled a few too many servers, and when the reset hit, most people had nowhere to go as we are hitting peak time in NA. Fixed by re-enabling some hardware.
The raiding UI is on right now. It’s been on for about an hour. So far it has been stable.
BTW, you need 10 people; and you need to change the squad type to “raid” before it will let you in. If you need to become a commander, there is an NPC “Whispers Keeper” you can talk to.
Okay, there are still some crashes. We are going to turn it back off at 11:25 pacific time (or so). We encourage you to get a raid team together and get in and have some fun.
We have gathered more data from this experiment but it’s getting late. I don’t know (honestly) if we will try again tomorrow or not.
We had the system on for a few minutes – long enough for some people to get squads together and enter the raid map. But there were still crashes on the backend servers so we disabled again. We’re currently looking at those crashes to decide if we should try again tonight or give up.
The UI does affect other parts of the game because it is causing server crashes. We would leave it on in a ‘beta’ mode if we could. We suspect the basic problem has to do with the fact we’re doing pick-up groups for this event and there is a problem when too many people join a squad – that sounds simple to fix but actually it’s a parallel processing problem spread across lots of clients and servers.
… which is why it’s hard for us to be really specific about when things will be fixed or even if we will try again tonight. I know that is difficult and I’m sorry.
The number of players allowed in per team all follow the same rules on all instances of each WvW map.
The numbers you see on the World Transfer screen are based on the number of players who have signed up for that world: what we call the membership count – which is not the same thing as the number of players active at any one time for that world.
We’ve been tracking WvW server perf and as a result we have increased the map capacities by 10 for each team. We’ll keep tracking and look at further increases over the next week.
The WvW matchups are isolated right now but it’s only for our convenience in analyzing performance. Because of the way Megaservers works it doesn’t really matter in terms of CPU load.
BTW, the ping number we show in the client on the options screen is probably different than you see in other games. We should name it something different. Traditionally the ping time is the time it takes for a packet to get to a server and back. In our case, it is the time to get to the server, get processed by the game map you are in, and then sent back. So our misnamed “ping” is much more variable than most ping numbers because it includes game processing time.
One thing I have noticed is that sometimes people (including me) will adjust their graphics settings and not realize they have made permanent changes This is because the game continues to draw existing characters at the old setting and only new characters get the new higher (and potentially slower) setting. Then they map travel and suddenly the game is slower because all of the characters (players and NPCs) have been loaded at the new, higher graphics level setting. But it’s not obvious that the earlier setting change caused things to run slower later. We could “fix” this by doing a big graphics refresh every time you change a setting. That might annoy players but eliminate this problem. -prefereset will also ‘fix’ this problem by resetting your graphics settings which might be what you experienced.
This isn’t “good enough”. My focus has shifted temporarily to the Stronghold open beta for sPvP tomorrow. But even so I and other programmers have been looking at source code searching for reasons why some people standing right next to each other have lag and some don’t. Right now that’s the big mystery. As programmers we don’t like it when performance is inconsistent; if it was always lagging for everyone it would be easier to debug but alas that is not the case.
We have hardware monitoring on our servers. Just to be sure I visited our head of TechOps and we looked at all of the hardware monitoring before I moved the map.
The “latency indicator” is now behaving in Blackgate Borderlands as it should be.
I’ll play in this map when I have time today but please also let me know if it is behaving a better for you. So far it looks better to me.
Okay – we looked at the hardware and it seems fine and is passing traffic fine so the next step is to do the experiment of moving this map (Blackgate Borderlands). There will be a couple of in-game announcements first.
Definitely strange goings-on in Blackgate Borderlands. The map was relatively empty but my special “latency indicator” graph was bouncing around like crazy, even though nothing was going on nearby. It could be hardware. We can move the map to different hardware but that requires kicking everyone because everyone has to disconnect and reconnect to the new hardware. It’s an easy test to do just inconvenient for people playing (about the same as if a new game build comes out). I will consult with my colleagues at work whether we should move everyone to different hardware.
The “rubber band” variety like you report means the client and server are not communicating and if it gets bad enough you get disconnected. This is the problem I’m working on understanding.
In the short run, just in case you are getting traffic shaping from your ISP, try, as an experiment, using “/clientport 443” on your command line. This moves you away from our standard port 6112 which some ISPs associate (incorrectly) with torrents. What happens with port 6112 is your traffic will be fine until you hit a zerg, then your internet traffic takes a big jump, and then (in some cases – not necessarily what is happening to you) your ISP or another one along the line will drop a lot of your traffic and/or disconnect you because they think you are serving torrents.
ALSO! It will be very helpful if you can post error codes when you are disconnected. When the error message is on your display you can press control-c and it will copy the error to the clipboard which makes it easy to paste in here. I just created the error below by force-closing my connection to the game server with a utility program and then copy/pasted the error here:
The game client lost its connection to the server. Please wait a few minutes before restarting the client and trying again. (Code=7:11:3:191:101)
Sometimes skill lag is across the entire map and that shows up in the charts I’m tracking. At other times, and I saw this for myself over the weekend, some people are fine and some people have giant pings and other people are DCing in the same battle.
BTW, when you run a long way on auto-run and have a problem you’ve probably already disconnected but didn’t know it until you tried to use a skill or do a special movement command.
This is the more interesting thing to me now – why are some people DCing but others in the same map are fine? My current course of action is to dig through the connection logs and look for a pattern.
Might be the people that DC have been in the middle of the fight spamming their skills ?
I considered this as a possibility so I tried it on myself and failed to DC. However I can imagine that someone like a Commander (who was a person I heard was DCing a lot) would be firing skills as fast as possible in the heat of battle and that caused a problem. It shouldn’t be a problem but perhaps combined with lag there is a problem. So one thing y’all can try at home if you notice that you’re getting skill lag try easing off on the keyboard. This is just something experimental to try.
I’ve increased the map caps on all of the Borderland maps. I continue to explore why people get disconnected in zergs (as I said, it has never happened to me, even when I visited a zerg where other people told me it happened in that exact zerg).
Okay – back to the issues. My graphs say “server induced lag” is greatly reduced (not perfect but very playable) but many people here are telling me they get a lot of skill lag and disconnects. I’m going to call that “communications induced lag” and it sounds like a very serious problem. Sadly I’ve never run into it. That doesn’t mean it doesn’t happen – I believe you all have a valid complaint! It’s just much harder to diagnose than a problem that happens to me. It could be our servers fail to send updates for some reason, or the client fails to process them for some reason, or some other communications channel problem. Yesterday in-game a person contacted me and said their guild chat and guild status was broken. I suggested using “/clientport 443” to see if they got a better connection and they did and their guild problems were solved. To me that means they were a victim of traffic shaping. I understand nobody wants me to blame the internet – so I won’t. I’ll assume this is a problem on our side that gets exercised in big battles.
As for zero ping being interesting to me … I go by something I call “Pat’s Dictum” which I learned from Pat Wyatt. If something seems a bit off in asynchronous programming you should check it out! It could be a simple UI bug or it could be that there is a broken communication path to the game! I won’t know until I or one of the programming staff figures it out. It might be nothing or it might be very important.
So I’ll be turning my attention to “communications induced lag” and trying to find a way to measure it across the player base since it doesn’t happen to me. I have no idea how long this will take but it’s very important to me to figure it out.
Hi all. “Studio Technical Director” at ArenaNet means I’m responsible for the engineering of the game and related tools (hardware and software). “Head of Engineering” is more descriptive of the role. I’ve been at ArenaNet nine years; I started as a server programmer on Guild Wars 1. That’s why I pay attention to server perf among many other things.
Well – here’s a thing. My ping is reported at 0 right now – in fact, since I joined this map. That makes me not trust that number very much! I’ll have to look into that. (Just remembered to screenshot it because no one at work tomorrow will believe it!)
Pretty interesting. I just watched a zerg fest in t5 and I had no skill lag. But players chatting with me who were closer to the battle said they had skill lag. They said I need to get into the battle to experience it. I would, but I die in 2 seconds. The big difference between me and those in the battle is the amount of data being reported- closer to battle means more data sent to client so it knows what to draw. Hmm. Interesting.
This requires looking at a whole new set of perf graphs.
Right now, I see that t3 EB EU has “server induced” lag.
I’ll keep repeating this: I’m looking at one specific problem this weekend. As near as I can tell from your posts (which I appreciate) is that the vast majority of lag problems y’all are experiencing is not from this problem.
Some other time (maybe even later today!) I’ll look at the huge variation in ping that people standing right next to each other have and how some people get such a bad ping they disconnect. But that’s a different problem, technically speaking.
I’m interested in that problem too, just not at this exact moment.
I believe you have lag, Nieny, but it’s not from the problem I’m looking at. Right now (meaning as I post this and for the last 10 minutes), nobody has lag from server lag. I was just talking to some peeps and one person with a 100 mps fibre connection was getting a 900 ping, then 20, then 900. That’s not the game. The people standing nearby had consistent 40 ping. I had 90. That’s not map lag. I don’t know what it is. People get mad when I suggest it is there ISP or their WiFi or someone in the house streaming Netflix. So I won’t suggest that.
briggah – I’ll look at your server. BTW, what actually happens is that you get DC’d but the client holds out hope more packets will arrive. You see increasing lag and then client decides its really not going to get an update and you get the DC message. “Lagged until you DC’d” translates to “DC’d when the lag started but the client was hoping for more messages to arrive before giving up”.
Hi; when I said “no lag” I meant at that exact moment when I posted. I’m looking for maps that are lagging so I can join them at the same time as I examine internal graphs. I didn’t mean to imply there is no lag anymore! I generally see heavier battles in EU but really these are just generalizations. Which map has the biggest battle varies greatly and can have a zerg fest at any time.
The t1 EB game is booking along at 90% cpu without lag (as tested by me hitting a skill) so maybe the timer tweak helped. Will move on to another server where there is more lag. (Maybe Far Shiverpeaks.) Wait! Epick battle just started in front of me!
Good Morning. This morning I’m observing Seafarer’s Rest WvW battles.
I also tweaked our map scheduling timer a tiny bit about an hour ago. We’ll see if it makes any difference.
There was a question about queue fairness. From my charts it looks fair.
I’m going to do an experiment on myself next time there is lag to see if I can induce the disco behavior. I had an idea about that last night. So far no lag though on my map. I see it at tier 5 though.
My “lag graphs” and my own experience in the map tells me that we’re looking at a different problem now for the disconnects (or discos as I call them). Do you get any other error code than 7:11 ?
Thanks. I’ve been whispering with BATMAN in-game to compare his experience to the various graphs I can see. So far this is going way better than EU peak time, but he says there are a lot of disconnects. That would be a different problem – is anyone else seeing a lot of that in T1 NA?
Yo – turns out I can temp move myself into Blackgate and experience the lag level for myself! Don’t worry – I won’t actually play for points. I’m “Asuradly So”, a little Asura engi in-game hanging at Blackgate Hill, if you want to say “hi”.
Of course, now I have forums, graphs, and actual game to watch at the same time. Should be interesting! I think I need more monitors.
I’m not going to argue technical trade-offs in a thread like this. We fixed a bug in our game scheduler and as a result we’ve found another bug in WvW that was previously hidden. It’s as simple as that.
