Guild Wars 2

Justin, here is your recent reply to a player who had badly lost two games:

That wasn’t your imagination, you were predicted to lose both times.

1) Forest, was 2-1-1-1 vs 2-1-1-1 You had a slight rank and MMRs disadvantage though.
2) Foefire, was 2-2-1 vs 1-1-1-1-1, ranks were even, but you had MMR advantage. Apparently wasn’t enough to overcome roster disadvantage.

The second one is interesting, and I think some changes I’ve been working on will help.

Now then…here a player reported to you what actually happened and you have rightly compared to the Prediction and the MMR “balance” of the match.

It is only through this sort of analysis that you can find out if the matching algorithm and Prediction are accurate. Otherwise you could just be twiddling knobs forever and never really making it better.

IDEA: “Secret Shoppers”. People who understand the game and can play specific matches to report back to you what actually happened and why so you can fix flaws in the matching and the Predictor.

It’s a key reality check.

it’s not a bad idea but how to select those players? especially considering how a low skill player (because of bad pc/connection/poor mechanical skills) might be on the other hand extremely knowledgeable and have great match awareness. The opposite cna also be true, a great player might have kitten-poor match understanding and carry teams with his mechanical goodness alone.

It occurs to me that you wouldn’t need to actually play in the game.

You could spectate it and do a “matchmaker/prediction accuracy report”.

A requirement would be that the analyzer would have to fundamentally understand Conquest strategy and be able to spot where the matchup failed to meet the prediction and what the predictor missed in its calculation.

I’m pretty sure the matchmaking algorithm does this already. And if it doesn’t, it’d be simple to collect the actual match result and compare it to the prediction. Ta da: accuracy reporting.

Obviously, reporting qualitative things is difficult. But, in a way, that doesn’t matter. If you’re against a team who is simply employing better tactics, that should be reflected in a lowering of your MMR and an increasing in the opponents MMR.

I think a lot of people are forgetting that the more you play the more accurate the matchmaking will become.

You may also get put in bad match up if there simply aren’t any better alternatives. If you’re a group of 5 queued, and there’s only one other group of 5 queued, you’re probably going to go against them regardless of your MMR.
Like when my team faced The Abjured, minus one. We’re no where close to their ranks, but I suspect it was the closest match there was at the time.

William,

No the matchmaking algorithm cannot analyze a match as a person could.

The idea here is to observe things that were not predicted from the MMRs. A reality check on why a prediction failed.

An MMR is adjusted after a match based on the accuracy of the existing MMRs in predicting the match.

This is circular logic. It makes me suspect self-reinforcing errors, and a somewhat inaccurate system.

Your MMR is not based primarily on your performance. It is inferred from you team’s score. And there can be many factors that mess up that process. Especially for a random solo team.

Maybe one team knows each other better and have established roles. Maybe they are using voice communications. Maybe the team makeup is better and MMR does not take that strongly enough into account.

All these things could make your MMR inaccurate.

The amount of random factors could drastically increase the time needed to determine your real MMR. You’d have to be in certain situations many, many times. Maybe it increases the number of matches needed exponentially. Maybe it could take 10,000 matches!

Laserbolt.6731:

William,

No the matchmaking algorithm cannot analyze a match as a person could.

The idea here is to observe things that were not predicted from the MMRs. A reality check on why a prediction failed.

An MMR is adjusted after a match based on the accuracy of the existing MMRs in predicting the match.

This is circular logic. It makes me suspect self-reinforcing errors, and a somewhat inaccurate system.

Your MMR is not based primarily on your performance. It is inferred from you team’s score. And there can be many factors that mess up that process. Especially for a random solo team.

Maybe one team knows each other better and have established roles. Maybe they are using voice communications. Maybe the team makeup is better and MMR does not take that strongly enough into account.

All these things could make your MMR inaccurate.

The amount of random factors could drastically increase the time needed to determine your real MMR. You’d have to be in certain situations many, many times. Maybe it increases the number of matches needed exponentially. Maybe it could take 10,000 matches!

I understand your point, but I still don’t think having people reporting back to Justin saying “Oh, well one team was in TeamSpeak so that’s why they won” would help. As far as I know, there’s no way for GW2 to check if people are using VoIP so no way to change the match prediction because of it.

The same applies to things like one team rotating badly. The system can’t really identify this, and yes an actual person easily could, but what would the result of that be? How would you try and incorporate this into MMR and match predictions?

It’s basically just saying “That team was better because of x, y and z” which is fine, but in the end, it’s irrelevant to the MMR algorithm unless it’s a factor which can be incorporated.

Regarding whether MMR is based on performance, or inferred from team score, I’m not entirely sure. What makes you think it’s based on team score not kills/caps/decaps/point defenses?

Also, the answer to your ‘circular logic’ argument has already been answered:

Justin ODell.9517:

Silentshoes.1805:

What do you think about this:

Your MMR changes based on whether your team performs better or worse than expected.

The expectation is based primarily on the MMRs of each team!

So…if the MMRs are inaccurate, the prediction is wrong, and the MMRs are adjusted incorrectly.

Glicko-2 doesn’t predict anything. A better analogy would be that it reacts. We think A is this and B is that, so if A beat B then it must be closer to that and vice versa. It improves its assessment of players by adjusting based on the result of each game. It is the outcome of these games over time that is the driving force.

“What makes you think it’s based on team score not kills/caps/decaps/point defenses?”

Here you go:

http://wiki.guildwars2.com/wiki/PvP_Matchmaking_Algorithm

Silentshoes.1805:

“What makes you think it’s based on team score not kills/caps/decaps/point defenses?”

Here you go:

http://wiki.guildwars2.com/wiki/PvP_Matchmaking_Algorithm

What, you mean this bit?:

The metrics used during [the match making second phase] include: rating, rank, party size, profession, ladder position, and dishonor.

Other than that, I can’t find anywhere that it explicitly states that previous match scores are used, other than is working out leaderboard points, which are entirely separate from MMR.

William C.6195:

Silentshoes.1805:

“What makes you think it’s based on team score not kills/caps/decaps/point defenses?”

Here you go:

http://wiki.guildwars2.com/wiki/PvP_Matchmaking_Algorithm

What, you mean this bit?:

The metrics used during [the match making second phase] include: rating, rank, party size, profession, ladder position, and dishonor.

Other than that, I can’t find anywhere that it explicitly states that previous match scores are used, other than is working out leaderboard points, which are entirely separate from MMR.

It’s in the first section, about Ratings. That’s your MMR rating. It is not based on your personal score in a match. It uses the Glicko2 System:

http://en.wikipedia.org/wiki/Glicko_rating_system