Guild Wars 2

Would be fantastic for other players to copy their character over to test out what could be bugged and what is working right ect ect. At least this way when it does release on live servers there would be that much less for them to “fix”

So far bugs are still being fixed from the new patch.. yet 90% of them could have been avoided if Arena Net had some kind of test server.

I had made this suggestion in a thread a little while back, but I guess they didn’t see it..

https://forum-en.gw2archive.eu/forum/game/gw2/anet-s-lack-of-transparency/4496966

Actually based on the nature of the bugs about 90% of them would most likely not have been detected at a testing server at all. Mainly due to the fact that it is quite likely that most of them turned up when patch was added to the live servers, which a test server can never fully mimic.

ArenaNet actually does have testing servers, they are just not public.

I have played multiple games with Public Testing Servers and yet every single one of those games still have rather large bugs when the patch is pushed to live.

It is in most cases simply a waste of resources, since most people would just use it as a way to play content before everyone else.

Or just not use it at all. The words “we’re not your unpaid testers” were used quite a bit when I played games with public test servers.

And then there was what happened to the UO test server – it became its own animal entirely, not a place to test patches.

I’m not convinced a test server would make much difference. I started playing Elder Scrolls Online (a game that has a test server) not long before their last big update and in spite of many people playing it a lot the patch still had a lot of bugs when it went live, including some that were apparently never seen on the test server. Apparently it’s the same with WoW.

I think in part it’s just the nature of software. I helped test a new version of the database we use at work. The same 7 people who use it every day tested it, on the same computers we’d normally use it on and we all agreed it was working. As soon as it was actually released (in other words saved on a different location on the server and linked to the full dataset instead of a sample) new bugs appeared. It took us months to get rid of them all.

Anet already have their own QA department to test things. Adding player testing, like the upcoming Heart of Thorns beta is great for getting player feedback (and it can’t hurt the bug hunting) but I’m not convinced it’d make a big difference for things like the recently released patch. That needs a focused group of testers, not random players running around the server.

Tobias Trueflight.8350:

Or just not use it at all. The words “we’re not your unpaid testers” were used quite a bit when I played games with public test servers.

And then there was what happened to the UO test server – it became its own animal entirely, not a place to test patches.

I loved the UO test servers. I had no idea they were for actually testing new releases and patches for the game, I genuinely thought they were for testing character builds and other ideas, since you could level up any skill instantly, got a huge pile of gold etc.

I was one of those people who would just use them to build castles we could never afford on the normal servers, set all skills absurdly high and just run around killing everything etc. etc. (In my defence I was 12 at the time.)

A properly run test server will root out bugs, especially the blatantly obvious ones like the green downed state health bar, and so on.
Smaller things like a broken sigil or rune might slip through because you need a lot of people to figure that out, but really guys, stop making excuses for Arenanet on this one.

Neural.1824:

A properly run test server will root out bugs, especially the blatantly obvious ones like the green downed state health bar, and so on.
Smaller things like a broken sigil or rune might slip through because you need a lot of people to figure that out, but really guys, stop making excuses for Arenanet on this one.

So, I need to ask, do you just assume their test server isn’t catching bugs because some visibly slip through? I mean, that’s what it looks like.

A properly run test server will root out bugs and make a nice database of them. Of course, then it’s up to people other than the testers to fix that crap. Which is why things like a green downed state health bar . . . which affects very little gets dropped to the bottom of the list while “faces render inside out” gets moved a bit higher.

I don’t think people understand that there’s a good chance that QA already is aware of the bug before it hits live. The problem lies in how serious the bug is. In Game Development, QA has to rate bugs on a Triage scale. Is it Critical, Serious, Medium, or Low-risk? Programmers have to fix the Critical and Serious bugs first, which can often times leave Medium and (especially) low-risk bugs (like text errors, slight graphic clippings) to go into live. They fix Med and Low bugs only if they have time to.

Critical and Serious bugs would be something that would damage your character or could be exploited. Medium bugs would be like the Guild Chat problem (especially when there is a work around) and hearing random voices go off (yes, it’s annoying, but it doesn’t disrupt gameplay).

Even if you got players who are dedicated to finding bugs instead of playing new content, there’s still a high chance that a lot of those bugs will still go in to the live patch.

Which do you all believe is the better method of the two?

Having testing done Inhouse, or public outhouse beta?

My concern with the way testing been done with GW2, is that most major changes were tested by Inhouse testers without word from the public, that could have spotted major issues far better than the Inhouse team.

Example of this, is the recent Trait change backlash. Inhouse testers didn’t see anything wrong with it, but clearly the public and the Inhouse testers don’t see eye to eye.

Same situation with the NPE.

I would argue that public is better as spotting these design flaws than the inside testers are IMO. I believe GW2 should have a public test server like Rift and World of Warcraft.

But what’s your thoughts on this?

A lot of companies have public beta servers that get the new patches before they go live, but after inhouse testing.

Thats usually a good idea to get relevant feedback as most people working for these companies have honestly no clue about things like balancing.

Then again Anet claims that the NPE/trait changes were based on feedback so I am not sure if more feedback from a public test server would have changed anything.

Problem with most Public Test Servers is that players rather explore the new content, choose to gossip about that and will put up with the bugs rather than constructively report them.

Knighthonor.4061:

Example of this, is the recent Trait change backlash. Inhouse testers didn’t see anything wrong with it, but clearly the public and the Inhouse testers don’t see eye to eye.

Same situation with the NPE.

I would argue that public is better as spotting these design flaws than the inside testers are IMO. I believe GW2 should have a public test server like Rift and World of Warcraft.

Calling the NPE and the trait revamp “design flaws” is mighty rich of you.

The thing is, testers are supposed to find bugs. Their job isn’t to comment on whether they like some feature or not.

Public test servers would also be disastrous for the economy, as anyone on the PTR would have advance knowledge of new recipes or changes to existing ones.

And personally, I think that with services like Steam and Kickstarter literally selling beta access as “Early Access”, public testing isn’t going to work. While there are some people who genuinely want to play the early versions to iron out bugs and glitches, there’s a far larger audience who simply want to play everything earlier. And those people will not stop complaining about bugs in the betas.

Knighthonor.4061:

Example of this, is the recent Trait change backlash. Inhouse testers didn’t see anything wrong with it, but clearly the public and the Inhouse testers don’t see eye to eye.

But what’s your thoughts on this?

Not seeing eye to eye with the public isn’t the same as introducing bugs. People might not like the trait changes but those aren’t bugs. At best they’re unpopular.

Behellagh.1468:

Problem with most Public Test Servers is that players rather explore the new content, choose to gossip about that and will put up with the bugs rather than constructively report them.

Pretty much this. Many public testers go in to play the game, and not test. Sure, they’ll offer some feed back of a bug or two, a broken event, or something that doesn’t seem fun, and yes, companies need that feedback too.

But the minority of public beta testers are spending time trying to ‘break’ the game, or ‘break’ something, because they know “wait, this is beta, i’m ALLOWED to find a cheat, exploit, unintentioned combination, etc as long as I report it so the devs fix it to make a better game”. I don’t blame many people in betas for not thinking like this, playing the game live, its hammered into your head to NOT do this stuff or ELSE, so people don’t go looking for it.

So I hope that the people getting into the stress test (which yes is seen as how much the game can hold with so many people, and further tests usually mean that larger amounts of people are allowed in) I hope those testers STILL try to break the game or find flaws or something so we get a better product in the end.

Public Test servers are not merely used to find bugs.
They are absolutely used to get early feedback on (possibly) upcoming features.

And yes that helps against obvious head in the cloud decisions like the trait changes.

10,000 people running around Tyria are going to spot bugs that professional testers and automatic regression tests aren’t going to have found. But that doesn’t make public testing ‘better’ for the game.

In case it’s not obvious, once the worst bugs of the original patch are fixed, there are going to be a lot of very uncomfortable conversations at ANet about what the heck happened that so many things that worked fine in the live environment failed after the patch, that some seemingly obvious bugs weren’t caught before the release, and so on.

There are all sorts of ways stuff like this can happen that have nothing to do with the quality of the QA team. For example, it’s possible ANet’s change & release management systems aren’t sufficient for the game’s aggressive update schedule (a symptom would be that the stuff released into live doesn’t precisely match what was tested by QA).

Further, setting up public testing introduces additional expenses and creates new issues: you have to maintain multiple code environments (increasing the likelihood that something that is a viable fix in one breaks something in another), additional virtual environments, and additional support staff (or take away from existing teams). Those in the testing environment will gain advantages over those who aren’t, including practice time for PvP and economically beneficial ideas.

In short, just because public testing would spot more bugs doesn’t mean that public testing is better, overall, for the game and the community.

Instead, what would be better is for ANet to figure out why stuff like this happens and take steps to prevent us from seeing it. If they are able to do that, great: problem solved. And if they aren’t, then it won’t matter how many resources they throw at testing; we are still going to see issues in the live environment.

It’s not an “either, or” situation. I’m sure ANet does extensive testing on their own, but no MMO ever goes without bugs or oversights. Sometimes there is no way to know how certain features are going to play out before the savage mob of the internet can tear into them. Even if people don’t report bugs directly, there are a lot of statistics that can be gathered simply by running a beta, seeing where people are burning through content due to unforeseen exploits, melting servers, etc.

I don’t think any MMO should have “only in-house testing,” or “only public beta testing.” I think a healthy mix of both is the smart decision.

Neural.1824:

A properly run test server will root out bugs, especially the blatantly obvious ones like the green downed state health bar, and so on.
Smaller things like a broken sigil or rune might slip through because you need a lot of people to figure that out, but really guys, stop making excuses for Arenanet on this one.

I don’t think the green downed state bar was a bug. That was an intentional change that was just not very well received so they called it a bug and reverted it. Either that or it was a bug resulting from fixing Necros.

Why not have both?

Lazaar.9123:

Why not have both?

See above: having public testing doesn’t prevent all of the problems we are seeing and introduces its own set of issues. It’s often not cost effective, in terms of resources and even reputation.

The better thing is, in my strong opinion, for ANet to figure out what went wrong this week and make substantive process changes to prevent things from getting this far again. Without robust procedures, introducing public testing will make matters worse, by introducing the need for much more coordination and distracting the development teams.

(I have to admit that before working with software developers, I would have agreed that more testing could only help.)

Illconceived Was Na.9781:

Instead, what would be better is for ANet to figure out why stuff like this happens and take steps to prevent us from seeing it. If they are able to do that, great: problem solved. And if they aren’t, then it won’t matter how many resources they throw at testing; we are still going to see issues in the live environment.

Well….they can’t figure out why stuff happens, if no one points it out. And having more eyes around TO point out stuff is a benefit. I mean, yes, there is a thing as too much, but there is also a thing as too little.

So yes, public betas are good for the game and community when GOOD feedback is involved. Sadly, not all feedback will be good. If someone just comes up and says “Dis sux, bad, doesn’t work, not fun” and doesn’t elaborate, yeah I categorize them as just a person testing server load. But you do get people in public betas that can say “This is bad because…., this is glitched when…., What would make this more fun to me is….” then those people I’d rather listen to.

So, getting those people to speak, offer good feedback, can help anet implement the measures to be more preventive the next time. But players must also remember one other thing; devs are people too, are prone to mistakes like overlooking something or making a bad call, and just not perfect. Yes, you can hold them to your high standards, but holding them to perfect standards is wrong.

Behellagh.1468:

Problem with most Public Test Servers is that players rather explore the new content, choose to gossip about that and will put up with the bugs rather than constructively report them.

Thats one thing .. the other is that bugs that are reported on the testservers often
still are released .. and also that there are even bugs after the release that never
before occured on the testserver at all.

So in the end the testservers mostly are used to know the upcoming stuff before
release and maybe also know what bugs to exploit for better farming or in GW2
what mats to buy to make a profit after release.