Guild Wars 2

Inb4 the first “no MMR hell, git gud” post.

Nice analysis btw.

Nicely done. Thanks for the effort. Now if anet listens…

Galandil.9641:

Inb4 the first “no MMR hell, git gud” post.

Nice analysis btw.

git gud! jk

So what’s the solution?

Very well done.

I suggested once on simulation would reveal everything on this simple matter and you put it into practise.

TPMN.1483:

So what’s the solution?

Remove MMR.

inb4 everyone who says nice post is also stuck in “mmr hell”.

Go back to season1 mechanics, allow lopsided matches instead of forcing 50/50, award more pips for winning in a high mmr match (meaning: if both teams are pro league niveau, they should get 5 pips per win, but only loose 1-2 pips if they loose).
Or instead of awarding more pips for higher mmr matches, run placement matches during off-season, solo queue only… teamscore divided through matches shows where the player belongs.

Winstreak awards luck, comeback pip awards loosing. No consideration for close matches. Has been pointed out quite often now. Nice to have shiny pictures to go with it

Nice work, btw.

/edit:
Btw, not stuck in mmr hell anymore. Playing with teammates and during primetime helped definitely getting out of negative mmr rating. And LOTS of wins.

I have to mention that I told you on forum that don’t wait 1-2 week than play, because the best players progressed very fast, you were good player too, but if all best players end up against you, and your good team, then bye bye, you won’t progress… I played every day, now I’m legendary, wuahhhhh shaka lakka!
https://www.youtube.com/watch?v=3uSTOHa4Im4
https://www.youtube.com/watch?v=8qkSe4YM7EY

According to your model, what’s the mechanism through which players become stuck? For instance, I can spot players in your simulation with a true skill rating of 2000 but a MMR pinned to 0; what is going on there to get them stuck? What division did they end up in and what do their matches look like? Even if they have a terrible win rate, their real expected points per game should exceed MMR’s expectations, and MMR should slowly (very slowly) correct itself. Why isn’t that happening?

The point of these exercises is to identify potential mechanisms through which players can get stuck or end up in negative feedback loops. Something in your model is producing those, so the next, important step is to identify what is responsible and whether that’s also something that could be happening in the real world. I’m particularly concerned about how you seem to have hard floors at 0 for both ‘real skill’ and MMR, neither of which makes sense (the scale for both MMR and true skill is arbitrary and none of the values mean anything out of context – and while there is an ‘AFK MMR’, it’s not something that’s measured in the system). When you stick a floor in your model like that I immediately think of the effects that edge will have on the system’s dynamics – akin to the ratchet from division floors.

RubberDougie.2750:

A close game where team MMR is identical changes MMR by five times the amount of a game where the team rating difference is 600. This means as far as Glicko is concerned, games between teams with vastly different ratings are not meaningful.

This is as far as I can tell the biggest problem with the current system. While it works pretty well if your initial MMR position is adequate and thus you get launched to the ‘proper’ pip position, if your MMR is not indicative and you’re in the wrong ladder rank it can take a long time for things to fix themselves.

RubberDougie.2750:

- Most players are pretty consistent between games. Their performance in each game is a Gaussian random variable with a log-normal deviation. A 100 deviation means 95% of performances will fall within 600 rating points.

Be careful with this – Glicko assumes game-to-game performance is normally distributed (well, strictly speaking, distributed in line with a logistic CDF), not log-normally distributed. I don’t have a good intuition on the impact of this on the model; however, I do see some bifurcated distributions in some of your plots, and an incongruence between the match win model and the Glicko model could be behind that.

Nice post.
I did´t do analysis. It was quite clear that this system causes grief in a mixed environement.

Excellent analysis, at least those of us, who have background in computer science and mathematics can appreciate it.

I am now waiting for Arenanet comments. Evan?

LordOtto.2650:

I have to mention that I told you on forum that don’t wait 1-2 week than play, because the best players progressed very fast, you were good player too, but if all best players end up against you, and your good team, then bye bye, you won’t progress… I played every day, now I’m legendary, wuahhhhh shaka lakka!
https://www.youtube.com/watch?v=3uSTOHa4Im4
https://www.youtube.com/watch?v=8qkSe4YM7EY

another useless pointless post!

skill should not be dependent on when you start in the league!
good players should be able progress no matter when they start! on the skill

another elitist thread there’s a change!

@godsie.2864
So who gets the legendary title is an elite? Oh yeah!
What is your elite division godsie? Don’t tell me emerald or sapphire?!
I hate this MMR system too, but I could climb, I was for a week in MMR hell, I got out, so practice make team ups, be better!
In season 1 in season 2 everybody crys, I cryed too, but I got diamond in season 1, now legendary, so yeah, I got gut!

try reading pls – my point is that your skill and rank SHOULD NOT be determined by when you start playing the season – I am where I deserve to be
good that you were able to climb!
I have practiced in unranked teamed up as well done everything
you escaped good for you – I have not despite all I can do and all the suggestions made
I have got better myself and perform better than I did at the start of the season but my season was over long time ago I admit I cannot carry 4 other players – so I still have to improve don’t know how – but I wont give up !

and gratz to you for getting "gut!! I know you will enjoy it !

LordOtto.2650:

I have to mention that I told you on forum that don’t wait 1-2 week than play, because the best players progressed very fast, you were good player too, but if all best players end up against you, and your good team, then bye bye, you won’t progress… I played every day, now I’m legendary, wuahhhhh shaka lakka!
https://www.youtube.com/watch?v=3uSTOHa4Im4
https://www.youtube.com/watch?v=8qkSe4YM7EY

I actually waited quite a while to get into the season (probably at least 2 weeks) and I swept through the early ranks easily. I had 3 legitimate losses up until about halfway through Ruby. Games at the end of Ruby became challenging, but still easier overall than last season. I’m now in diamond slowly working my way through with a few games a day, as my schedule allows.

Very interesting. This is refreshing to see more and more ppl addressing the issues of S2 with more than just opinions to back their claims one way or the other.

How do you think predetermined MMR from S1 affected the MMR hell?

I would be interested to know that as well

R O C.6574:

How do you think predetermined MMR from S1 affected the MMR hell?

Well, in my case, knowing that there was no other consequence than loosing a match I didn’t mind playing riskier builds last season. When the rules changed I sooooo regretted having done that.

Next time, warn people before you make such changes or reset the metric you want to use.

Sirbeaumerdier.3740:

R O C.6574:

How do you think predetermined MMR from S1 affected the MMR hell?

Well, in my case, knowing that there was no other consequence than loosing a match I didn’t mind playing riskier builds last season. When the rules changed I sooooo regretted having done that.

Next time, warn people before you make such changes or reset the metric you want to use.

My problem was I learned to play in S1 ranked…

It’s ugly

Evan Lesh.3295:

This is a really great analysis. I appreciate all the work you put into this. We recognize the problems that the season 2 matchmaking is causing and are looking into solutions that will provide a good experience and satisfy our goals for the league system.

abolish mmr and just match people randomly within division? Can’t be worse than it is now

It was evident from the first week. Even some of the minor corrections would take the majority of the season (which is almost over).

But the solution to season 2 was to say ‘Prestige!!!’ and that you achieved what you wanted in that.

Akachi.1352:

Evan Lesh.3295:

This is a really great analysis. I appreciate all the work you put into this. We recognize the problems that the season 2 matchmaking is causing and are looking into solutions that will provide a good experience and satisfy our goals for the league system.

abolish mmr and just match people randomly within division? Can’t be worse than it is now

They need a combination of S1 and S2 and maybe smarter mmr, as opposed to no mmr.

Could this all be the unwanted by-product of trying to achieve faster q times…?

Man, wish you had chipped in when Ensign and I were comparing simulations yesterday. We had another math thread going, might be worth a look for you.

On a side note that helps to explain the differences between our two results since we had different methods of determining who would play against who.

LanfearShadowflame.3189:

Akachi.1352:

Evan Lesh.3295:

This is a really great analysis. I appreciate all the work you put into this. We recognize the problems that the season 2 matchmaking is causing and are looking into solutions that will provide a good experience and satisfy our goals for the league system.

abolish mmr and just match people randomly within division? Can’t be worse than it is now

They need a combination of S1 and S2 and maybe smarter mmr, as opposed to no mmr.

Why do you need any MMR any way? If you are a strong player you will ascend along with other strong players and eventually settle where your matches will be as close as can be. And on top of it you will do so without tarnishing your prestige because having an even easier climb than you would have had otherwise.

MMR is fun as a statistic to inform you about your team (if you have a regular team) but using it to build soloq teams is problematic.

BTW: Ty for replying here Evan. It is nice to have some feedback from you guys about matters like this.

Oh, and Evan, a polite request: Since this problem actually affects the accuracy of MMR in the system, please consider a soft reset or increasing the volatility numbers for everyone once you change the system.

Sirbeaumerdier.3740:

LanfearShadowflame.3189:

Akachi.1352:

Evan Lesh.3295:

This is a really great analysis. I appreciate all the work you put into this. We recognize the problems that the season 2 matchmaking is causing and are looking into solutions that will provide a good experience and satisfy our goals for the league system.

abolish mmr and just match people randomly within division? Can’t be worse than it is now

They need a combination of S1 and S2 and maybe smarter mmr, as opposed to no mmr.

Why do you need any MMR any way? If you are a strong player you will ascend along with other strong players and eventually settle where your matches will be as close as can be. And on top of it you will do so without tarnishing your prestige because having an even easier climb than you would have had otherwise.

MMR is fun as a statistic to inform you about your team (if you have a regular team) but using it to build soloq teams is problematic.

BTW: Ty for replying here Evan. It is nice to have some feedback from you guys about matters like this.

I’m not a huge fan of the “just throw ‘x number’ of players together” at random ala GW1 FA, JQ or RA. I don’t necessarily want to be teamed with just 4 other ppl that queued at the same time as me so long as they fall in the right pip range.

I got a lot of enjoyment from the 50/50 system because a lot more matches were close and exhilarating The issue with MMR is its simply not smart, nor is it truly “individual.” (If I understand how it presently works)

LanfearShadowflame.3189:

Sirbeaumerdier.3740:

LanfearShadowflame.3189:

Akachi.1352:

Evan Lesh.3295:

This is a really great analysis. I appreciate all the work you put into this. We recognize the problems that the season 2 matchmaking is causing and are looking into solutions that will provide a good experience and satisfy our goals for the league system.

abolish mmr and just match people randomly within division? Can’t be worse than it is now

They need a combination of S1 and S2 and maybe smarter mmr, as opposed to no mmr.

Why do you need any MMR any way? If you are a strong player you will ascend along with other strong players and eventually settle where your matches will be as close as can be. And on top of it you will do so without tarnishing your prestige because having an even easier climb than you would have had otherwise.

MMR is fun as a statistic to inform you about your team (if you have a regular team) but using it to build soloq teams is problematic.

BTW: Ty for replying here Evan. It is nice to have some feedback from you guys about matters like this.

I’m not a huge fan of the “just throw ‘x number’ of players together” at random ala GW1 FA, JQ or RA. I don’t necessarily want to be teamed with just 4 other ppl that queued at the same time as me so long as they fall in the right pip range.

I got a lot of enjoyment from the 50/50 system because a lot more matches were close and exhilarating The issue with MMR is its simply not smart, nor is it truly “individual.” (If I understand how it presently works)

If they decide to go toward a reward track I’m backing you 100%. It’s if they insist on going toward a ladder that what I say matters.

Sirbeaumerdier.3740:

LanfearShadowflame.3189:

Sirbeaumerdier.3740:

LanfearShadowflame.3189:

Akachi.1352:

Evan Lesh.3295:

This is a really great analysis. I appreciate all the work you put into this. We recognize the problems that the season 2 matchmaking is causing and are looking into solutions that will provide a good experience and satisfy our goals for the league system.

abolish mmr and just match people randomly within division? Can’t be worse than it is now

They need a combination of S1 and S2 and maybe smarter mmr, as opposed to no mmr.

Why do you need any MMR any way? If you are a strong player you will ascend along with other strong players and eventually settle where your matches will be as close as can be. And on top of it you will do so without tarnishing your prestige because having an even easier climb than you would have had otherwise.

MMR is fun as a statistic to inform you about your team (if you have a regular team) but using it to build soloq teams is problematic.

BTW: Ty for replying here Evan. It is nice to have some feedback from you guys about matters like this.

I’m not a huge fan of the “just throw ‘x number’ of players together” at random ala GW1 FA, JQ or RA. I don’t necessarily want to be teamed with just 4 other ppl that queued at the same time as me so long as they fall in the right pip range.

I got a lot of enjoyment from the 50/50 system because a lot more matches were close and exhilarating The issue with MMR is its simply not smart, nor is it truly “individual.” (If I understand how it presently works)

If they decide to go toward a reward track I’m backing you 100%. It’s if they insist on going toward a ladder that what I say matters.

Just the way they have set forth how the back pieces are to be done, it’s just a glorified reward track.

If they want to turn it into an actual, competitive ladder, then the back piece, among other things, need to be removed. Titles fine, unique minis fine, hell unique skin fine. 1 of only 2 ways to get the actually legendary tier item…nope.

“Personal” scores based on the team’s result where you have 20% influence. Yo, makes sense.

RubberDougie.2750:

The nature of the MMR hell (Matlab analysis)
To those who are unfamiliar, MMR hell refer to the long streak of losses many players experienced in season 2. It’s caused by reducing MMR lead to reduced teammate quality which leads to further reduced MMR. This also applies to win streaks. Some people do not believe this happens, and think everyone who has this problem is bad (ROFL).

I have written a Matlab Simulation of the Season 1 (and unranked) and Season 2 algorithms to find out exactly why this happened and what factors were involved. I made some educated guesses on the distribution of player population and implementation of the matchmaker. The purpose of this is not to publish in Nature, but to reproduce the symptoms people are experiencing. If you are interested in the assumptions, please see the figures at the end of the post and comments in the source code. I also made the source code available to anyone interested in running this simulation. (http://www.ssnt.org/personal/pvp/pvp_sim.rar)
For the complete post with inline figures, see:
(http://www.ssnt.org/personal/pvp/pvp_analysis.pdf)
For Full Sized figures, see:
(http://www.ssnt.org/personal/pvp/figs.rar)

So without further delay, the pretty pictures

http://i.imgur.com/eFxtmkR.png

Unlike real players, true skill levels of simulated players are assigned. This allowed me to have a look at how matchmaking captures the real skill. Both algorithms were terrible, with S2 being a little better with the added “benefit” of vastly inflating or deflating rating of some players. In this simulation there were about 4.5% of stuck players, and that number did not decrease as season progressed.

So why is this happening? Let’s look at a few factors.

Login Randomness
Eliminating the login randomness not only kills the MMR hell, it also allow S2 algorithm to vastly out-perform S1. If everyone who played less frequently only logged in while people who played more often are online (In the other words, the number of players online is a perfect sinusoid), there is no snowball hell. This is probably why Anet thought season 2 algorithms was better on paper. But reality doesn’t work like that. People have jobs.

http://i.imgur.com/O1uJLQI.png

This happens because people can’t find even games during off-hours. Looking over the data for the stuck models, I realize they overwhelmingly had very poor number of close games. But before we get to that, there is another factor to consider.

Within-team MMR Tolerance

http://i.imgur.com/eZmb03K.png

Removing the limit for difference between MMR of teammates improved the situation dramatically. It did not eliminate the MMR hell (about 3.5% player were stuck) for S2, but it did allow the MMR to have some resemblance of actual skill. The reason is an increased number of off-peak games, which increased the chance of finding even games during these hours. This leads us to the heart of the problem.

Rating Error and Close games

http://i.imgur.com/O0T1eDm.png A
http://i.imgur.com/eEz1DP8.png B
http://i.imgur.com/0RX0EEw.png C

In every case (A-baseline, B-Nonrandom Login, C-No MMR limit within team), the Maximum Rating Error is a decreasing function of number of CLOSE games (team MMR difference less than 300). This can be seen from the Glicko equations. A close game where team MMR is identical changes MMR by five times the amount of a game where the team rating difference is 600. This means as far as Glicko is concerned, games between teams with vastly different ratings are not meaningful. The team with the higher rating cannot gain MMR, and the team with the lower rating cannot win.

The job of Glicko algorithm is to estimate a player’s true skill from CLOSE games. Without them, Glicko cannot rate players. While these streaks of wins or losses are subjectively satisfying or frustrating, they don’t provide any meaningful information about the players’ true skill. The job of the Matchmaker is to provide Glicko with these games from which true skill of players can be inferred from wins and losses.

By removing the limit on the MMR difference of red and blue teams, Season 2 matchmaker is wasting everyone’s time with strings of pointless games. It’s less efficient without any benefits. In less technical terms, season 2 algorithm is dumber than two bags of rocks. Relaxing teammate MMR requirement for the old algorithm would do a better job.

Assumptions

http://i.imgur.com/R2GV77G.png

This is the distribution of player models used. It makes the following assumptions:
- Most players are casual and plays a couple hours a day, but a small number of players …
- Online time modeled as sum of a sinusoid with a random variable, and is strongly correlated (R squared ~=.5) with a pure sinusoid.
- Player skill is normally distributed and is strongly (R squared ~=.45) correlated with play time.
The more you play the better you get.
- Most players are okay, but there aren’t many past 2500 ratings, and there are 2% player with zero true skill ratings, which represent AFKs, brand new players and people who intentionally throw matches.
- Most players are pretty consistent between games. Their performance in each game is a Gaussian random variable with a log-normal deviation. A 100 deviation means 95% of performances will fall within 600 rating points.
- The ratings were adjusted in each game assuming each player played a series of game in the time interval against every player on the other team. Email communication with Dr. Glickman indicates the algorithm was never intended to be used in a sport where teams can rearrange between matches. I also tested the alternative interpretation where the teams were adjusted as a single player. But it made no difference. Please check team_adjust in the source code for this implementation.
- Teammate MMR requirement was 100 for baseline and 1000 for the relaxed case.

Quick question, where did you get the data on the population? (like population size, because I do think GW2 PvP suffers from a lack of players)

Is it in the .rar file? I can’t open those because my computer doesn’t have a rar extractor.
If you could post a picture or an excel file (preferably an excel file), that would be great! I’d also like to try analyzing the numbers, but with R since I use R (also would be good practice for me!)

Thanks in advance

LanfearShadowflame.3189:

Sirbeaumerdier.3740:

LanfearShadowflame.3189:

Akachi.1352:

Evan Lesh.3295:

This is a really great analysis. I appreciate all the work you put into this. We recognize the problems that the season 2 matchmaking is causing and are looking into solutions that will provide a good experience and satisfy our goals for the league system.

abolish mmr and just match people randomly within division? Can’t be worse than it is now

They need a combination of S1 and S2 and maybe smarter mmr, as opposed to no mmr.

Why do you need any MMR any way? If you are a strong player you will ascend along with other strong players and eventually settle where your matches will be as close as can be. And on top of it you will do so without tarnishing your prestige because having an even easier climb than you would have had otherwise.

MMR is fun as a statistic to inform you about your team (if you have a regular team) but using it to build soloq teams is problematic.

BTW: Ty for replying here Evan. It is nice to have some feedback from you guys about matters like this.

I’m not a huge fan of the “just throw ‘x number’ of players together” at random ala GW1 FA, JQ or RA. I don’t necessarily want to be teamed with just 4 other ppl that queued at the same time as me so long as they fall in the right pip range.

I got a lot of enjoyment from the 50/50 system because a lot more matches were close and exhilarating The issue with MMR is its simply not smart, nor is it truly “individual.” (If I understand how it presently works)

You’re assuming the pip system alone is not an MMR system. Without the hidden MMR, you will be more likely to find a spot where you get 50/50, but only if they also take the bias out of the pip reward system. No more rebounds, safety walls, and lose 2 pips for 3 game losing streaks.

The hidden system is dual redundant and can only work against the first. Season 1 carried newbs by pairing them with pro players, which held the pro players back. Season 2’s system was just completely divergent based on luck and timing.

Both pips and hidden MMR alone do the same thing. They give you score value when you win or lose. Having 2 systems working at the same time will cause errors and fail to represent actual ranking.

Evan Lesh.3295:

This is a really great analysis. I appreciate all the work you put into this. We recognize the problems that the season 2 matchmaking is causing and are looking into solutions that will provide a good experience and satisfy our goals for the league system.

So pleased to see a post like this!

If you’re looking for solutions could I suggest you take a look at the new MOBA called Paragon (not openly released yet so $20 for early access).

It’s not a substitute at all for GW2, and how it fairs as a game is too early to say, but the way they handle their matchmaking and divisions has E-sports excellence written all over it.

The matchmaking challenges are similar to GW2: 5 vs 5 class based PVP, majority Solo Queing.

They have a companion website for their players: agora.gg
Go there and put in my account name: Stackalee

Once you’ve finished laughing at how bad I am, have a good look at what they are doing there. Expand the games, see how they match the average MMR of the teams, take into account other important (for that game) statistics Kill/Deaths etc. Remember their player base is much smaller than GW2 at the moment but they’re managing to create good games with very reasonable queue times – max I’ve had is about 7 mins, usually 2-3 mins.

Importantly see how beautifully transparent it all is, and what an amazing resource that is for players. I’m able to see so much about who I played with and against, find new people to invite to teams or add to my friends list. Also you are able to look at the builds of your team mates and opponents while in-game.

Something akin to this for GW2 would propel it in the right direction.
Reward tracks can exist ‘In-Game’ (like Season 1) but an external E-sport focused transparent resource site like this, that is a pure MMR ladder would create a real, meaningful competitive environment – and of course THEN matchmaking can happily try to make even, fair matches between similar MMR teams. Everyone gets what they want:

- Fair games
- Reward track progression in-game (grind-able… put in the effort, reap the rewards)
- Division ranking that’s meaningful (on external site) – e.g. You’re in ‘Gold Division’ with 2600 MMR then Yeah you’re good – it’s not arguable.

This isn’t a quick fix but GW2 needs something that isn’t a quick fix. To be taken seriously it needs transparency and Division placement that the player base TRUST is a reasonable reflection of player skill.

Dirtyrascal.1023:

Once you’ve finished laughing at how bad I am, have a good look at what they are doing there. Expand the games, see how they match the average MMR of the teams, take into account other important (for that game) statistics Kill/Deaths etc. Remember their player base is much smaller than GW2 at the moment but they’re managing to create good games with very reasonable queue times – max I’ve had is about 7 mins, usually 2-3 mins.

So they match 10 players within an elo range and sort them into teams so the average elo is around the same?

This looks exactly what GW2 should do, I doubt it will happen though because they don’t appear to want transparency. And there is the problem of the ascension achievements.

OTOH I was flip flopping on trying Paragon, I don’t like mobas that much, however your post has convinced me to try it cause I know how committed Epic are to providing a good gaming experience.

Nice post OP. Nice to see ANet recognizing your work.

And no I’m not in MMR hell, I’d have to still be playing PvP for that.

morrolan.9608:

Dirtyrascal.1023:

Once you’ve finished laughing at how bad I am, have a good look at what they are doing there. Expand the games, see how they match the average MMR of the teams, take into account other important (for that game) statistics Kill/Deaths etc. Remember their player base is much smaller than GW2 at the moment but they’re managing to create good games with very reasonable queue times – max I’ve had is about 7 mins, usually 2-3 mins.

So they match 10 players within an elo range and sort them into teams so the average elo is around the same?

This looks exactly what GW2 should do, I doubt it will happen though because they don’t appear to want transparency. And there is the problem of the ascension achievements.

OTOH I was flip flopping on trying Paragon, I don’t like mobas that much, however your post has convinced me to try it cause I know how committed Epic are to providing a good gaming experience.

“So they match 10 players within an elo range and sort them into teams so the average elo is around the same?”

- Yup, seems the case and it’s a really nice experience. Had some amazing games. It’s really interesting after a game to see how the match was formed. Also I’m satisfied that my ranking is entirely fair and that when (ahem) my MMR increases it will indicate that I have genuinely improved as a player.

I’m definitely not suggesting Paragon is better/worse as a game than GW2 – it’s a completely different genre but GW2 could definitely learn/copy from the way they are handling matchmaking, E-sports and MMR transparency.

Whatever Anet decide to do – a HARD reset of everyone’s MMR before the next season is needed.

Evan Lesh.3295:

This is a really great analysis. I appreciate all the work you put into this. We recognize the problems that the season 2 matchmaking is causing and are looking into solutions that will provide a good experience and satisfy our goals for the league system.

Is that it? Whole season ruined, you are going to wait for it to finish to serve us another bullkitten for the next one? And then again “mmm interesting, we are looking into it”.

Way to go.

LanfearShadowflame.3189:

I’m not a huge fan of the “just throw ‘x number’ of players together” at random ala GW1 FA, JQ or RA. I don’t necessarily want to be teamed with just 4 other ppl that queued at the same time as me so long as they fall in the right pip range.

Yeah I’m not so sure this would work out either. After all theoretically this S2 system should have worked and its worse. There may be issues with mmr but basing matches on averaging it out led to closer matches and no problems with long loss streaks.

I do not have a mathlab-license anymore. Could you do me a favour and ease some assumptions in order to get the maximum possible percentage of players stuck in “mmr-hell”?

@Evan: It is nice that you admit that something went wrong. The consequence would be to terminate the league, put the resources on finding a solution and restart the leagues.

Evan Lesh.3295:

This is a really great analysis. I appreciate all the work you put into this. We recognize the problems that the season 2 matchmaking is causing and are looking into solutions that will provide a good experience and satisfy our goals for the league system.

Goals? What Goals? – You have one – get as many players playing the game and buying items from the gem store as as possible – everything else is irrelevant.

Prestige? – what a joke.

Tenebria.7239:

On a side note that helps to explain the differences between our two results since we had different methods of determining who would play against who.

The much more detailed model of who plays against who is pretty cool. It looks like his model is finding the edge effects of having a small effective population (small player population, even smaller from playing off-peak), which are kind of interesting. That’s something I’d want to play around with.

My main concern is the visible edges and the bifurcation in some of these distributions. The former is a result of mis-specification; the latter likely from an incongruence between the match win probability model and Glicko. Both of these problems are matchmaking-agnostic; you’d see them even if you made teams totally at random. When you compound them with the small sample convergence problems that his (her?) model wants to investigate it’s really hard to break apart what is going on.

This is important because, statistically, the problem with the S2 matchmaker is a convergence problem. You have some complicated dynamics between the pip matcher, the MMR matcher, and player behavior that can make recovering from a MMR deviance impossibly slow (like hundreds and hundreds of games slow). As the number of games goes to infinity it doesn’t make a difference how you match your teams – eventually everyone gets to the right place, if the MMR predictor is accurate. I think OP has a really cool model of some of those dynamics that I’d like to see more of.

It’s just that if there’s a wrench in there, I feel like I have to talk about the wrench and not the cool thing he wants to talk about.

SneakyTouchy.6043:

LanfearShadowflame.3189:

Sirbeaumerdier.3740:

LanfearShadowflame.3189:

Akachi.1352:

Evan Lesh.3295:

This is a really great analysis. I appreciate all the work you put into this. We recognize the problems that the season 2 matchmaking is causing and are looking into solutions that will provide a good experience and satisfy our goals for the league system.

abolish mmr and just match people randomly within division? Can’t be worse than it is now

They need a combination of S1 and S2 and maybe smarter mmr, as opposed to no mmr.

Why do you need any MMR any way? If you are a strong player you will ascend along with other strong players and eventually settle where your matches will be as close as can be. And on top of it you will do so without tarnishing your prestige because having an even easier climb than you would have had otherwise.

MMR is fun as a statistic to inform you about your team (if you have a regular team) but using it to build soloq teams is problematic.

BTW: Ty for replying here Evan. It is nice to have some feedback from you guys about matters like this.

I’m not a huge fan of the “just throw ‘x number’ of players together” at random ala GW1 FA, JQ or RA. I don’t necessarily want to be teamed with just 4 other ppl that queued at the same time as me so long as they fall in the right pip range.

I got a lot of enjoyment from the 50/50 system because a lot more matches were close and exhilarating The issue with MMR is its simply not smart, nor is it truly “individual.” (If I understand how it presently works)

You’re assuming the pip system alone is not an MMR system. Without the hidden MMR, you will be more likely to find a spot where you get 50/50, but only if they also take the bias out of the pip reward system. No more rebounds, safety walls, and lose 2 pips for 3 game losing streaks.

The hidden system is dual redundant and can only work against the first. Season 1 carried newbs by pairing them with pro players, which held the pro players back. Season 2’s system was just completely divergent based on luck and timing.

Both pips and hidden MMR alone do the same thing. They give you score value when you win or lose. Having 2 systems working at the same time will cause errors and fail to represent actual ranking.

You cannot take out the safeties from a random pug group game, you could if all we had was premade 5v5-s then the team itself has a rating (like in wow arena) you cannot penalize a soloQer that much for what he doesn’t have control over (what the other 4 players are doing)

OP, what would it look like if matchmaking was based purely on divsion/piprange?

They need a new metric that can give individual rating based on contributing to the win. i.e. capping/decapping points, dps on points, stomps on points, secondary objectives that swing the momentum. I’m sure the math would be far more complex, but that’s why devs get the big bucks right? I remember gaining pips on loses in S1, what was mechanic for that?

That was an impressive work!

R O C.6574:

They need a new metric that can give individual rating based on contributing to the win. i.e. capping/decapping points, dps on points, stomps on points, secondary objectives that swing the momentum. I’m sure the math would be far more complex, but that’s why devs get the big bucks right? I remember gaining pips on loses in S1, what was mechanic for that?

I don’tthink Anet has ever taken personal performance into consideration because some tactics like evasive kiting and leaving portals on far can sway the entire match without being detectable number wise.