[Season 5] MEASURING SKILL & MATCHMAKING

[Season 5] MEASURING SKILL & MATCHMAKING

in PvP

Posted by: Ithilwen.1529

Ithilwen.1529

Based on what I’ve read, there seems to be a strong intent to talk the community into reversing on solo/duo after this season. I hope that’s not the case.

I’d still like to know about preventing class stacking.

Also, what’s to stop people from sandbagging the placement matches to get an easy win streak?

Mesmerising Girl

[Season 5] MEASURING SKILL & MATCHMAKING

in PvP

Posted by: Yannir.4132

Yannir.4132

Based on what I’ve read, there seems to be a strong intent to talk the community into reversing on solo/duo after this season. I hope that’s not the case.

I’d still like to know about preventing class stacking.

Also, what’s to stop people from sandbagging the placement matches to get an easy win streak?

Because winstreaks don’t give you any benefits in the new system. Why would anyone sandbag their rating for basically nothing?

Yannir for males. (guard,thief,war,ele)
Sonya for females. (necro,rev,ranger,mes,engi)
All classes lvl 80.

[Season 5] MEASURING SKILL & MATCHMAKING

in PvP

Posted by: Exciton.8942

Exciton.8942

I would say the willingness to change your build or class is part of your skill.

If you want to win and are not confident in winning with your main in certain team composition, then it is time to switch up.

Also, don’t forget you always have the option to duo queue. You can queue with one dps and one support class and with that you should be able to reach the rank where your skill level belongs to.

Stop finding excuses before season even started and focus on improving yourself if you want a high rank.

[Season 5] MEASURING SKILL & MATCHMAKING

in PvP

Posted by: Ithilwen.1529

Ithilwen.1529

@Yannir

Members of the community team in unranked, simply to get laughs from easy wins against PUGs. Why would they not sandbag to get classed in with new people they could easily trounce?

You’d end up with a high win rate. This would advance you quickly. Some find entertainment in winning a match with very little chance of losing.

Mesmerising Girl

[Season 5] MEASURING SKILL & MATCHMAKING

in PvP

Posted by: Yannir.4132

Yannir.4132

@Yannir

Members of the community team in unranked, simply to get laughs from easy wins against PUGs. Why would they not sandbag to get classed in with new people they could easily trounce?

You’d end up with a high win rate. This would advance you quickly. Some find entertainment in winning a match with very little chance of losing.

And yet, as Evan explained earlier, a high win rate may not affect your rating in any way. It’s only as you defeat matches you are supposed to lose you get a better rating. Tanking your rating does nothing except temporarily placing you on a lower rating. Which will start to climb back to your old rating as soon as you start winning again.

Yannir for males. (guard,thief,war,ele)
Sonya for females. (necro,rev,ranger,mes,engi)
All classes lvl 80.

[Season 5] MEASURING SKILL & MATCHMAKING

in PvP

Posted by: Shaogin.2679

Shaogin.2679

Honestly I don’t know why 50/50 matchmaking gets so much hate. Maybe because of the pip system and how it was implemented in past. But in an ideal matchmaking system, you would have 2 teams of equally skilled players matched up against each other. By simple statistics you will average a 50% win rate in that scenario.

The hate came from 50/50 matchmaking going against how the pip system worked, and the fact that it felt like the system was forcing wins or losses on players due to just about every match being a complete blowout. Blowouts, whether you win or lose, just aren’t fun. So yeah, pretty easy to understand that frustration when people are trying to gain pips and they just keep getting alternating blowout wins/losses. From the sound of it though, the 50/50 matchmaking should work out better this season.

Doc Von Doom – Asuran Necromancer
Gate of Madness
Contribute to the Wiki MetaBattle Builds

[Season 5] MEASURING SKILL & MATCHMAKING

in PvP

Posted by: AIJ.3784

AIJ.3784

Okey, still slightly salty that I can`t play ranked together with several of my guildies at the same time… however I do like the sound of the new changes to the system and are excitied to test them out. Well done so far Anet!

[Season 5] MEASURING SKILL & MATCHMAKING

in PvP

Posted by: Entenkommando.5208

Entenkommando.5208

What you have to understand about cs is that you have a ton more players and still maybe 1/10 the variables gw2 has so statistics work a lot more reliable.
I don’t have exact numbers but I bet the number of active players differs probably around the factor of 100.

Also you don’t have things like class swapping and class synergies in a team because classes simply don’t exist there.

R.I.P Kodasch Allianz [KoA]

All we wanted was a GvG.

[Season 5] MEASURING SKILL & MATCHMAKING

in PvP

Posted by: skcamow.3527

skcamow.3527

Based on what I’ve read, there seems to be a strong intent to talk the community into reversing on solo/duo after this season. I hope that’s not the case.

I’d still like to know about preventing class stacking.

Evan did comment on this item

1) There will be the expected periodic balance updates, but I cannot speak to the extend of them.
2) Class stacking has not been a priority because of the leagues rework, and negative feedback regarding character locking when queuing. This might be something worth polling.
3) The vendor will always be around, but the new currency is only obtainable from the league pip system. You won’t be getting all the skins in a single season.
4) We’ve added the double currency requirement for ascended items to sink out shards of glory and will use the league tickets for more rare, longer term rewards as we make them.

Kortham Raysplitter (Yak’s Bend)

[Season 5] MEASURING SKILL & MATCHMAKING

in PvP

Posted by: Exedore.6320

Exedore.6320

Based on what I’ve read, there seems to be a strong intent to talk the community into reversing on solo/duo after this season. I hope that’s not the case.

Why? The old solo queue was pretty terrible and was closest to the new system than Season1 thru Season4. It’s a team-based game; stopping teams from playing it competitively is silly.

I’m pretty certain there’s a design flaw in how the algorithm matches full teams with players, and that’s the cause of a lot of complaints.

I’d still like to know about preventing class stacking.

It’s not going to happen any time soon. First, you’d have to remove swapping characters at the start of a match. That makes a team with better role distribution a guaranteed win because no one can swap to create a well-rounded team. Second, it increases queue times and leads to worse matches since you have to skip over duplicate professions.

It’s more productive to balance the problems which make class stacking so powerful. Class stacking was actually discouraged for years prior to HoT because a diverse team was better.

Kirrena Rosenkreutz

[Season 5] MEASURING SKILL & MATCHMAKING

in PvP

Posted by: shion.2084

shion.2084

Well now that we will have visible ratings, we will start talking to the community about averages, ranges, deltas, everything MMR. Average/starting rating is 1200, and has a bell curve distribution.

Hi Evan, 2 things.

First a Hypothesis. The 50/50 hate may be generated by the perception that the following can happen.
1) I am winning several matches in a row.
2) There is a small population of players queueing.
3) My reward for doing well is to be placed with worse and worse players to average out for my relatively high MMR. So with 10 players in queue my reward might be Me (MMR1) and MMR7 – MMR10 vs. MMR2 -MMR6. (where MMR1 is the highest of the 10 in the match)

Essentially the perception that rather than teaming me up with better allies and against better quality opponents (which less people would object to as its more thrilling matches) , my reward for doing well could end up being matches where I’m teamed with crappier and crappier teammates to account for my doing well. Essentially seeing how far I can carry worse and worse people until I break. (Un-fun).

Is the above possible given certain populations of player queues?

NEXT:

You are starting new players at 1200, an expected average rating? So if I really am 1200, I’ll be more likely to be teamed with new folks, and play against new unknown folks? (assuming a healthy dose of new folks keep coming in through out the season). Does this not seem somewhat problematic to you?

Now lets say you get teamed with 4 random blokes and the other 5 rando new blokes at 1200 are better, so you sink in MMR and will play with folks whom also happened to randomly lose. Would that not be frustrating if the bulk of people you keep getting teamed with are totally random unknown players and you keep getting 1 or 2 who sink things for you? Lets say you manage to climb back to 1200…. do you not then get the rando lottery of being likely to be paired with a bunch of untried folks again?

[Season 5] MEASURING SKILL & MATCHMAKING

in PvP

Posted by: Supreme.3164

Supreme.3164

Why starting MMR so high? makes no sense whatsoever …

[Season 5] MEASURING SKILL & MATCHMAKING

in PvP

Posted by: Exedore.6320

Exedore.6320

Why starting MMR so high? makes no sense whatsoever …

In Elo and Glicko, you start everyone at the midpoint in the rating scale. If everyone starts at 1200, then the max is 2400 (assuming min of 0).

Kirrena Rosenkreutz

[Season 5] MEASURING SKILL & MATCHMAKING

in PvP

Posted by: Kicker.8203

Kicker.8203

What’s up with profession-based MMR? It used to be separate for each class you play. Is it taken out of the equation?

[Season 5] MEASURING SKILL & MATCHMAKING

in PvP

Posted by: Exedore.6320

Exedore.6320

What’s up with profession-based MMR? It used to be separate for each class you play. Is it taken out of the equation?

Because you’re not locked into your profession choice once the match is created, there’s no reason to use profession in matchmaking.

Kirrena Rosenkreutz

[Season 5] MEASURING SKILL & MATCHMAKING

in PvP

Posted by: Soothsayer.9206

Soothsayer.9206

Evan Lesh – "Sure, we could have zero matchmaking, but two things will happen: First, your teammates will have random ratings and your opponents will have random ratings. Did you lose because you are less skilled than your opponents, or because your teammates are less skilled than your opponents? There is no easy way to tell without a lot of complicated math. “Make a system to figure it out, then!” you say. Well, Glicko does this for us already. Matchmaking assumes its data is correct by pairing opponents that appear to be evenly matched, then Glicko does its job to adjust ratings in case the outcome wasn’t what was expected. Second, random matches make your rating update verrry slowly. Every match won against a much less skilled opponent won’t make you go up because of course you were suppose to beat them, your rating must already be accurate. Every match lost against much higher rated opponents won’t make you go down because of course you were suppose to lose, your rating must already be accurate."

First, I do not think that it necessarily follows that your MMR rating will be random, it will, in part, be a reflection of the quality of the team(s) with yourself being a part. The only drawback is that if your team(s) are bad your MMR will also be bad regardless of your personal skill level.

Second, the system has no way of determining your individual skill level since sPvP is team oriented your MMR is a compilation of team data not your individual contribution (which would only 20% of a team – ideally speaking). The question of whether you won or lost is not based on skill but on a faulty assumption that MMR is entirely up to the individual player when it is not. Matchmaking “assumes” the data is correct, but has no way of verifying if that is indeed the case as a result it “appears” to match TEAMS evenly.

Third, no matter how else it is stated, matches should never be “supposed to win” or “supposed to lose” no matter how miniscule the odds may be of winning/losing they should never be predetermined by any data, especially data collected that doesn’t reflect the individual but rather is taken on assumptions thus eliminating the most basic element of competition – an opportunity to win however remote that may be. Sport’s history is replete with examples of teams that were “suppose” to win and lost, or teams that were suppose to “lose” and found a way to win – that’s why they play the game.

Kyrie eleison…Dies irae, dies illa…Quando judex est venturus

(edited by Soothsayer.9206)

[Season 5] MEASURING SKILL & MATCHMAKING

in PvP

Posted by: hackks.3687

hackks.3687

Personal Score

We have never been quite happy with how personal score attempted to portray a player’s contribution in a match, so we’ve reworked the personal score system as you currently know it in favor of raw statistics. All stats have been condensed into more meaningful categories: Damage Dealt; Healing Given; Kills; Deaths; Revives; and two game-mode specific stats, Offense and Defense, which track effort toward gaining team score. Top stats are now team based, and you’ll be able to view who got which awards the end of a match.

This change alone makes next season worth checking out. I’m highly skeptical of any “skill rating” until they prove the matchmaking is in fact working. But at least there’ll be a real scoreboard for once.

Also, blocking teams from queuing together in a team game seems pretty silly.

Hackkz/Riggamaroll
I’ve stayed at this party entirely too long

[Season 5] MEASURING SKILL & MATCHMAKING

in PvP

Posted by: shion.2084

shion.2084

Well now that we will have visible ratings, we will start talking to the community about averages, ranges, deltas, everything MMR. Average/starting rating is 1200, and has a bell curve distribution.

Hi Evan, 2 things.

First a Hypothesis. The 50/50 hate may be generated by the perception that the following can happen.
1) I am winning several matches in a row.
2) There is a small population of players queueing.
3) My reward for doing well is to be placed with worse and worse players to average out for my relatively high MMR. So with 10 players in queue my reward might be Me (MMR1) and MMR7 – MMR10 vs. MMR2 -MMR6. (where MMR1 is the highest of the 10 in the match)

Essentially the perception that rather than teaming me up with better allies and against better quality opponents (which less people would object to as its more thrilling matches) , my reward for doing well could end up being matches where I’m teamed with crappier and crappier teammates to account for my doing well. Essentially seeing how far I can carry worse and worse people until I break. (Un-fun).

Is the above possible given certain populations of player queues?

NEXT:

You are starting new players at 1200, an expected average rating? So if I really am 1200, I’ll be more likely to be teamed with new folks, and play against new unknown folks? (assuming a healthy dose of new folks keep coming in through out the season). Does this not seem somewhat problematic to you?

Now lets say you get teamed with 4 random blokes and the other 5 rando new blokes at 1200 are better, so you sink in MMR and will play with folks whom also happened to randomly lose. Would that not be frustrating if the bulk of people you keep getting teamed with are totally random unknown players and you keep getting 1 or 2 who sink things for you? Lets say you manage to climb back to 1200…. do you not then get the rando lottery of being likely to be paired with a bunch of untried folks again?

Evan, I wouldn’t mind a response to how your algorithm handles this. You never know I could actually be potentially helpful with ideas, I have a career in working out cognitive solutions to a myriad of problems with more complex interactions.

[Season 5] MEASURING SKILL & MATCHMAKING

in PvP

Posted by: Spartacus.3192

Spartacus.3192

I really like the end summary screen showing who won top stats etc and your % values. I was duo queuing last night with a guildie and for a few matches between us we got all the top stats, none for our teammates.

Your typical average gamer -
“Buff my main class, nerf everything else. "

[Season 5] MEASURING SKILL & MATCHMAKING

in PvP

Posted by: Evan Lesh

Previous

Evan Lesh

PvP Gameplay Programmer

Next

Well now that we will have visible ratings, we will start talking to the community about averages, ranges, deltas, everything MMR. Average/starting rating is 1200, and has a bell curve distribution.

Hi Evan, 2 things.

First a Hypothesis. The 50/50 hate may be generated by the perception that the following can happen.
1) I am winning several matches in a row.
2) There is a small population of players queueing.
3) My reward for doing well is to be placed with worse and worse players to average out for my relatively high MMR. So with 10 players in queue my reward might be Me (MMR1) and MMR7 – MMR10 vs. MMR2 -MMR6. (where MMR1 is the highest of the 10 in the match)

Essentially the perception that rather than teaming me up with better allies and against better quality opponents (which less people would object to as its more thrilling matches) , my reward for doing well could end up being matches where I’m teamed with crappier and crappier teammates to account for my doing well. Essentially seeing how far I can carry worse and worse people until I break. (Un-fun).

Is the above possible given certain populations of player queues?

No, the system is not out to get you. If you have the highest MMR in the queue, I believe the absolute worst outcome is 1,4,6,8,10 vs 2,3,5,7,9.

NEXT:

You are starting new players at 1200, an expected average rating? So if I really am 1200, I’ll be more likely to be teamed with new folks, and play against new unknown folks? (assuming a healthy dose of new folks keep coming in through out the season). Does this not seem somewhat problematic to you?

Now lets say you get teamed with 4 random blokes and the other 5 rando new blokes at 1200 are better, so you sink in MMR and will play with folks whom also happened to randomly lose. Would that not be frustrating if the bulk of people you keep getting teamed with are totally random unknown players and you keep getting 1 or 2 who sink things for you? Lets say you manage to climb back to 1200…. do you not then get the rando lottery of being likely to be paired with a bunch of untried folks again?

The matchmaker uses number of games played to avoid this. Moving the starting rating will just change which players get affected. Its best to start players where they are likely closest to their true rating.

Bluxgore (80 Warr), Xilz (80 Necro), Ivo (80 Eng)
Bra (80 Guard), Fixie Bow (80 Ranger), Wcharr (80 Ele)
Xdragonshadowninjax (80 Thief)

[Season 5] MEASURING SKILL & MATCHMAKING

in PvP

Posted by: Link.1049

Link.1049

Shion,

I’ve seen a similar thing…hundreds of people playing and I’m teamed up with the same 3-4 people 4-6 matches in a row. It can either work for good, or terribly bad, like when teamed up with a pair that would keep running back to the same point over and over, get owned, and repeat. You’d think with so many people playing, you’d get team variety.

I must have the worst mmr in all of my matches, because if I win it’s 13, if I lose its 13-17…always. As I mentioned in a prior post the system has some fatal flaws, so I’m just going for the dye chests (and gold).

[Season 5] MEASURING SKILL & MATCHMAKING

in PvP

Posted by: Aveigel.2601

Aveigel.2601

Here is what gets people: Having a low or negative team mmr telling them they are bad when they themselves think they are or are better than what the team puts them to be.

So in the example above of where you’d have a 1,4,6,8,10 vs 2,3,5,7,9 type of setup, you’re bound to have at least 2 unhappy players with good reason, assuming there is a wide enough gap in true skill and not mmr.

Either the 8 & 10 from team 1 if they lose, or the 7 and 9 from team 2 if they lose. That’s 40% of the losing team who feels justified in being frustrated because they don’t feel that they are adequatly represented by the matrch outcome. all htat because too often the 1,4 (6 maybe too) or the 2,3 (5 probably too) screws the match up for them.

and being carried or carrying should not be a big part of league play. unranked…sure people got to play to get good and there is not the incentive to be good in the same manner that league is percieved, since it is a competitive environment and people can now see what the current system thinks of them, however scewed by randomness it is at the moment because of lack of matches played.

While I understand a need to have “x” matches played to have an accurate rating for your system, I am 100% sure that over 80% people’s expectations is to have something they feel representative by the end of the qualifying matches. And the movement thereafter being also rng aka vs what mmr team you’re facing seems to not favor self image in the slightest.

Perhaps higher rewards for wins to reach a plateau would be better since people could test their true skill faster would ease this feeling. It would seem expected that a skill would plateau @ their actual skill level, and that even if mmr would be measured on a smaller moving average, that true skill would occillate within a certain range. and that would be the actual true skill level of a player given the team makeups he got till then.

(edited by Aveigel.2601)

[Season 5] MEASURING SKILL & MATCHMAKING

in PvP

Posted by: Pyriall.1683

Pyriall.1683

No, the system is not out to get you. If you have the highest MMR in the queue, I believe the absolute worst outcome is 1,4,6,8,10 vs 2,3,5,7,9.

If you are using MMR to pair teams, why are 1,4,2,3 even being teamed with 8,9,10’s? That’s why this system continues to frustrate people.

I want teammates who know how to play and I want to play against other people who know how to play. Constantly getting people who don’t know how to play is a never ending cycle of frustration. You guys need to do a better job of teaching new people how to play.

(edited by Pyriall.1683)

[Season 5] MEASURING SKILL & MATCHMAKING

in PvP

Posted by: Evan Lesh

Previous

Evan Lesh

PvP Gameplay Programmer

Next

If you are using MMR to pair teams, why are 1,4,2,3 even being teamed with 8,9,10’s? That’s why this system continues to frustrate people.

Because it’s a 10 player game. Those 10 players could have the exact same rating in the best case scenario.

Bluxgore (80 Warr), Xilz (80 Necro), Ivo (80 Eng)
Bra (80 Guard), Fixie Bow (80 Ranger), Wcharr (80 Ele)
Xdragonshadowninjax (80 Thief)

[Season 5] MEASURING SKILL & MATCHMAKING

in PvP

Posted by: Pyriall.1683

Pyriall.1683

If you are using MMR to pair teams, why are 1,4,2,3 even being teamed with 8,9,10’s? That’s why this system continues to frustrate people.

Because it’s a 10 player game. Those 10 players could have the exact same rating in the best case scenario.

Am I misunderstanding what you are saying. 1-10 denotes skill or simply the number of players?

[Season 5] MEASURING SKILL & MATCHMAKING

in PvP

Posted by: Honest John.4673

Honest John.4673

No, the system is not out to get you. If you have the highest MMR in the queue, I believe the absolute worst outcome is 1,4,6,8,10 vs 2,3,5,7,9.

If you are using MMR to pair teams, why are 1,4,2,3 even being teamed with 8,9,10’s? That’s why this system continues to frustrate people.

It’s just an example.

You have to use numbers to tell who’s higher or lower than each other. It’s just so you don’t have to spend the time coming up with examples like:

1520, 1535, 1545, 1555, 1565 vs 1525, 1530, 1540, 1550, 1560

[Season 5] MEASURING SKILL & MATCHMAKING

in PvP

Posted by: Evan Lesh

Previous

Evan Lesh

PvP Gameplay Programmer

Next

If you are using MMR to pair teams, why are 1,4,2,3 even being teamed with 8,9,10’s? That’s why this system continues to frustrate people.

Because it’s a 10 player game. Those 10 players could have the exact same rating in the best case scenario.

Am I misunderstanding what you are saying. 1-10 denotes skill or simply the number of players?

Just ordering of skill. The skill range could be big or small. (hopefully small)

Bluxgore (80 Warr), Xilz (80 Necro), Ivo (80 Eng)
Bra (80 Guard), Fixie Bow (80 Ranger), Wcharr (80 Ele)
Xdragonshadowninjax (80 Thief)

[Season 5] MEASURING SKILL & MATCHMAKING

in PvP

Posted by: jcbroe.4329

jcbroe.4329

If you are using MMR to pair teams, why are 1,4,2,3 even being teamed with 8,9,10’s? That’s why this system continues to frustrate people.

Because it’s a 10 player game. Those 10 players could have the exact same rating in the best case scenario.

Am I misunderstanding what you are saying. 1-10 denotes skill or simply the number of players?

Just ordering of skill. The skill range could be big or small. (hopefully small)

Hey Evan,

Just wanted to take a quick moment to ask that if in the future, you would consider implementing a “hard cap” on initial tier placements (can’t place higher than gold3, plat1, etc).

While it’s true I don’t know your exact algorithm and 10 placement matches is a generally accepted industry standard for a multitude of good reasons, as a player, judging from my own experience and observations as well as some others; the perception is that the placement matches have over or under-inflated peoples placements.

With a cap, it just seems like you could still place people within proper thresholds accurately, but then people wouldn’t be able to play the “place and beat the decay” game, at least, not if they wanted those higher tiers, which would “force” people into participating in more matches than a lot currently have (granted we’re only 2-3 days in).

This would help the system gather more statistics and place people more accurately and with a higher indication of their actual skill tier than the way it appears currently, not to mention there being a larger pool of players to create matches from since people would still have the incentive to play for more than just beating MMR decay (I hear reports from friends in higher tiers of games being scarce, and I assume they’d also be less evenly matched since the longer the matchmaker works, the more the rating threshold opens up to try to create a match).

Granted I didn’t really need to explain it all, but I did so in case you do actually decide to respond and do one of those “separate perception from reality” things that you do.

Jroh | Former SOAC Ranger Podcaster | Platinum Division Top 100 Player
www.twitch.tv/itsJROH For stream, stream schedule, other streamers, builds, etc
https://www.youtube.com/user/JRoeboat

[Season 5] MEASURING SKILL & MATCHMAKING

in PvP

Posted by: Evan Lesh

Previous

Evan Lesh

PvP Gameplay Programmer

If you are using MMR to pair teams, why are 1,4,2,3 even being teamed with 8,9,10’s? That’s why this system continues to frustrate people.

Because it’s a 10 player game. Those 10 players could have the exact same rating in the best case scenario.

Am I misunderstanding what you are saying. 1-10 denotes skill or simply the number of players?

Just ordering of skill. The skill range could be big or small. (hopefully small)

Hey Evan,

Just wanted to take a quick moment to ask that if in the future, you would consider implementing a “hard cap” on initial tier placements (can’t place higher than gold3, plat1, etc).

While it’s true I don’t know your exact algorithm and 10 placement matches is a generally accepted industry standard for a multitude of good reasons, as a player, judging from my own experience and observations as well as some others; the perception is that the placement matches have over or under-inflated peoples placements.

With a cap, it just seems like you could still place people within proper thresholds accurately, but then people wouldn’t be able to play the “place and beat the decay” game, at least, not if they wanted those higher tiers, which would “force” people into participating in more matches than a lot currently have (granted we’re only 2-3 days in).

This would help the system gather more statistics and place people more accurately and with a higher indication of their actual skill tier than the way it appears currently, not to mention there being a larger pool of players to create matches from since people would still have the incentive to play for more than just beating MMR decay (I hear reports from friends in higher tiers of games being scarce, and I assume they’d also be less evenly matched since the longer the matchmaker works, the more the rating threshold opens up to try to create a match).

Granted I didn’t really need to explain it all, but I did so in case you do actually decide to respond and do one of those “separate perception from reality” things that you do.

We talked about capping placement and opted to have less restrictions. One thing I would consider is lowering the default rating deviation so people don’t fly around as much and soft reset has more meaning.

Bluxgore (80 Warr), Xilz (80 Necro), Ivo (80 Eng)
Bra (80 Guard), Fixie Bow (80 Ranger), Wcharr (80 Ele)
Xdragonshadowninjax (80 Thief)

[Season 5] MEASURING SKILL & MATCHMAKING

in PvP

Posted by: jcbroe.4329

jcbroe.4329

If you are using MMR to pair teams, why are 1,4,2,3 even being teamed with 8,9,10’s? That’s why this system continues to frustrate people.

Because it’s a 10 player game. Those 10 players could have the exact same rating in the best case scenario.

Am I misunderstanding what you are saying. 1-10 denotes skill or simply the number of players?

Just ordering of skill. The skill range could be big or small. (hopefully small)

Hey Evan,

Just wanted to take a quick moment to ask that if in the future, you would consider implementing a “hard cap” on initial tier placements (can’t place higher than gold3, plat1, etc).

While it’s true I don’t know your exact algorithm and 10 placement matches is a generally accepted industry standard for a multitude of good reasons, as a player, judging from my own experience and observations as well as some others; the perception is that the placement matches have over or under-inflated peoples placements.

With a cap, it just seems like you could still place people within proper thresholds accurately, but then people wouldn’t be able to play the “place and beat the decay” game, at least, not if they wanted those higher tiers, which would “force” people into participating in more matches than a lot currently have (granted we’re only 2-3 days in).

This would help the system gather more statistics and place people more accurately and with a higher indication of their actual skill tier than the way it appears currently, not to mention there being a larger pool of players to create matches from since people would still have the incentive to play for more than just beating MMR decay (I hear reports from friends in higher tiers of games being scarce, and I assume they’d also be less evenly matched since the longer the matchmaker works, the more the rating threshold opens up to try to create a match).

Granted I didn’t really need to explain it all, but I did so in case you do actually decide to respond and do one of those “separate perception from reality” things that you do.

We talked about capping placement and opted to have less restrictions. One thing I would consider is lowering the default rating deviation so people don’t fly around as much and soft reset has more meaning.

That would be an interesting (and I think viable) implementation. I’ll be excited to see the discussion/progress on the development/direction of ideas like this.
Thanks for the response.

Jroh | Former SOAC Ranger Podcaster | Platinum Division Top 100 Player
www.twitch.tv/itsJROH For stream, stream schedule, other streamers, builds, etc
https://www.youtube.com/user/JRoeboat

[Season 5] MEASURING SKILL & MATCHMAKING

in PvP

Posted by: Aveigel.2601

Aveigel.2601

Seriously Evan if you guys plan anything ask yourselves this question: Would the current system upset anyone in it’s current form? and why? If you have your own reservations about your own system, you could even mention what they are and goal/intentions, when you expose yourself like this you’ll probably find that people will be more lenient/understanding and willing to provide positive feedback. I’d hate having to read negative things about something I worked hard on, I bet you do too.

That said, here is what I think:

Even if you have 0.5% of the people placed in top division @ startup it’s wrong if most people feel they don’t deserve to be where they belong because of some normal distribution. And while being able to see your placement vs leaderboard and friends, if you played lots with them and you know where your true skill level should be at (guesstimate) you’re more enclined to feel cheated by having a rating that is not conform with your self image.

Startup should not be a normal, and everything earned, no freebies. Narrow to 1 standard deviation at most and have it so that within that range mmr may fluctuate rapidly enough that it might reach a decent point.

But you still have the issue of not adressing individual skill and starting by putting together 10 random people and hoping that something significant will come out of this. That’s just not gonna happen. To have anything at least remotely significant you’re looking @ min of 13 matches with a decent population where you can identify/classify people @ the start and you don’t even have that when you start everyone @ about a same skill rating. So how can you hope to be fairly representative at all knowing this?

My answer is what I’m trying to get people to explore in my thread about Economy of Worth in PvP, aka determining a single player’s worth in every match through measuring positive behaviours and negative ones to establish a more accurate baseline, since then you’ll not be comparing teams formed up of random people but comparing actual people to people.

I’m not trying to diss anything just stating reasons why upset people are speaking up. Love that you guys wanted to make a change, I think that’s in the right track, this just might not yet be the right track, either you can tweak it to something that actualy works or you might need to consider other options.

Lets keep an open mind.

Could you shed more light on the innerworkings of match placement? Did you use previous pvp data to do trial runs vs various other team setups to validate/invalidate placement?

aka
from 10 matches.
if I had previous mmr of 1800, place me with equivalent previous mmr players and then test us against:

  • Game 1: 1000 mmr team, if win add +200, if loss -100
  • Game 2: 1200 mmr team, if win add +200, if loss -100
  • Game 3: 1400 mmr team, if win add +200, if loss -100
  • Game 4: 1600 mmr team, if win add +200, if loss -100
  • Game 5: 1800 mmr team, if win add +200, if loss -100
  • Game 6: 1700 mmr team, if win add +100, if loss -50
  • Game 7: 1800 +100 win, -50 loss
    and so on, narrowing and testing limit again until you have a clearer placement with no default assignement…

Could you elaborate? ty.

(edited by Aveigel.2601)

[Season 5] MEASURING SKILL & MATCHMAKING

in PvP

Posted by: reikken.4961

reikken.4961

Does unranked queue apply any kind of matchmaking? It doesn’t seem like it does.

anecdote: yesterday I queued up for unranked with a party of fairly casual players for a few rounds of pvp. 2nd match in we get utterly crushed, barely even able to get any points at all. Then next round we get matched against the same team. with similar results. Half my party logs off for the day.

Whatever your matchmaking is or isn’t doing, please adjust it to prevent things like that from happening (large skill gaps and facing the same people repeatedly). Good experiences in unranked are important too. Even moreso now that you can’t do ranked with a full party.

(edited by reikken.4961)

[Season 5] MEASURING SKILL & MATCHMAKING

in PvP

Posted by: BlaqueFyre.5678

BlaqueFyre.5678

Seriously Evan if you guys plan anything ask yourselves this question: Would the current system upset anyone in it’s current form? and why? If you have your own reservations about your own system, you could even mention what they are and goal/intentions, when you expose yourself like this you’ll probably find that people will be more lenient/understanding and willing to provide positive feedback. I’d hate having to read negative things about something I worked hard on, I bet you do too.

That said, here is what I think:

Even if you have 0.5% of the people placed in top division @ startup it’s wrong if most people feel they don’t deserve to be where they belong because of some normal distribution. And while being able to see your placement vs leaderboard and friends, if you played lots with them and you know where your true skill level should be at (guesstimate) you’re more enclined to feel cheated by having a rating that is not conform with your self image.

Startup should not be a normal, and everything earned, no freebies. Narrow to 1 standard deviation at most and have it so that within that range mmr may fluctuate rapidly enough that it might reach a decent point.

But you still have the issue of not adressing individual skill and starting by putting together 10 random people and hoping that something significant will come out of this. That’s just not gonna happen. To have anything at least remotely significant you’re looking @ min of 13 matches with a decent population where you can identify/classify people @ the start and you don’t even have that when you start everyone @ about a same skill rating. So how can you hope to be fairly representative at all knowing this?

My answer is what I’m trying to get people to explore in my thread about Economy of Worth in PvP, aka determining a single player’s worth in every match through measuring positive behaviours and negative ones to establish a more accurate baseline, since then you’ll not be comparing teams formed up of random people but comparing actual people to people.

I’m not trying to diss anything just stating reasons why upset people are speaking up. Love that you guys wanted to make a change, I think that’s in the right track, this just might not yet be the right track, either you can tweak it to something that actualy works or you might need to consider other options.

Lets keep an open mind.

Could you shed more light on the innerworkings of match placement? Did you use previous pvp data to do trial runs vs various other team setups to validate/invalidate placement?

aka
from 10 matches.
if I had previous mmr of 1800, place me with equivalent previous mmr players and then test us against:

  • Game 1: 1000 mmr team, if win add +200, if loss -100
  • Game 2: 1200 mmr team, if win add +200, if loss -100
  • Game 3: 1400 mmr team, if win add +200, if loss -100
  • Game 4: 1600 mmr team, if win add +200, if loss -100
  • Game 5: 1800 mmr team, if win add +200, if loss -100
  • Game 6: 1700 mmr team, if win add +100, if loss -50
  • Game 7: 1800 +100 win, -50 loss
    and so on, narrowing and testing limit again until you have a clearer placement with no default assignement…

Could you elaborate? ty.

You realize that you have been with and against players close to your Mmr since they told us how they did the Soft reset, (Previous Mmr+1200)*.5

The reason I see that they soft reset is probably due to the horrible abuse of the previous system.

[Season 5] MEASURING SKILL & MATCHMAKING

in PvP

Posted by: Crowley.8761

Crowley.8761

I got unlucky with teams in placement and I know I’m gonna be bronze for the next month. I get a good game, then 3 warrs on my team, a good game, then 3 rangers, a good game etc.

I realize I’d have to be extremely lucky to get out of bronze. How well I play etc, doesn’t affect my rating. If my team loses, I lose rating. I can play like a god, I still lose rating.

It’s just about powering through it and hope the next season placement will be luckier for me. It is pretty discouraging and demotivating though but oh well.

They’ve made a ranked system that’d worked for 5 queue teams and are using it for pugs so can’t expect much tbh.

[Season 5] MEASURING SKILL & MATCHMAKING

in PvP

Posted by: OriOri.8724

OriOri.8724

@Evan Lesh – I’m a bit confused as to how the game determines your initial MMR. I know that you play 10 placement games to find it, but does the game calculate your MMR from the first game or does it just use the default MMR for all 10 games before catapulting you higher or lower in terms of MMR depending on how you did?

[Season 5] MEASURING SKILL & MATCHMAKING

in PvP

Posted by: BlaqueFyre.5678

BlaqueFyre.5678

@Evan Lesh – I’m a bit confused as to how the game determines your initial MMR. I know that you play 10 placement games to find it, but does the game calculate your MMR from the first game or does it just use the default MMR for all 10 games before catapulting you higher or lower in terms of MMR depending on how you did?

Look at my post above and Evans post from the +9\-15 thread.

Evan stated that your initial Mmr for placement matches is (previous Mmr +1200)*.5 then is adjust by your wins and losses in the 10 placement matches.

[Season 5] MEASURING SKILL & MATCHMAKING

in PvP

Posted by: Aveigel.2601

Aveigel.2601

You realize that you have been with and against players close to your Mmr since they told us how they did the Soft reset, (Previous Mmr+1200)*.5

The reason I see that they soft reset is probably due to the horrible abuse of the previous system.

Well this little bit is nice to know.

Are they going to use the data about individual matches for player contributions in % to try to make any relevant assessment of skills?

Like I mentioned, current system, at this time needs some tweaking, I’d be somewhat relieved if it was a v.1.0 say and that a vision of v.2.0 or whatever evolution of the system was the goal.

[Season 5] MEASURING SKILL & MATCHMAKING

in PvP

Posted by: Phil.8901

Phil.8901

If you are using MMR to pair teams, why are 1,4,2,3 even being teamed with 8,9,10’s? That’s why this system continues to frustrate people.

Because it’s a 10 player game. Those 10 players could have the exact same rating in the best case scenario.

Am I misunderstanding what you are saying. 1-10 denotes skill or simply the number of players?

Just ordering of skill. The skill range could be big or small. (hopefully small)

Hey Evan,

Just wanted to take a quick moment to ask that if in the future, you would consider implementing a “hard cap” on initial tier placements (can’t place higher than gold3, plat1, etc).

While it’s true I don’t know your exact algorithm and 10 placement matches is a generally accepted industry standard for a multitude of good reasons, as a player, judging from my own experience and observations as well as some others; the perception is that the placement matches have over or under-inflated peoples placements.

With a cap, it just seems like you could still place people within proper thresholds accurately, but then people wouldn’t be able to play the “place and beat the decay” game, at least, not if they wanted those higher tiers, which would “force” people into participating in more matches than a lot currently have (granted we’re only 2-3 days in).

This would help the system gather more statistics and place people more accurately and with a higher indication of their actual skill tier than the way it appears currently, not to mention there being a larger pool of players to create matches from since people would still have the incentive to play for more than just beating MMR decay (I hear reports from friends in higher tiers of games being scarce, and I assume they’d also be less evenly matched since the longer the matchmaker works, the more the rating threshold opens up to try to create a match).

Granted I didn’t really need to explain it all, but I did so in case you do actually decide to respond and do one of those “separate perception from reality” things that you do.

We talked about capping placement and opted to have less restrictions. One thing I would consider is lowering the default rating deviation so people don’t fly around as much and soft reset has more meaning.

You basically say everything, can you tell us how much min/max can fly the soft rank after 10 games placement?

+ 200 -200?

Es. soft rank 1700 you can go to 1900 or 1500?

[Season 5] MEASURING SKILL & MATCHMAKING

in PvP

Posted by: Wolfric.9380

Wolfric.9380

I got unlucky with teams in placement and I know I’m gonna be bronze for the next month. I get a good game, then 3 warrs on my team, a good game, then 3 rangers, a good game etc.

I realize I’d have to be extremely lucky to get out of bronze. How well I play etc, doesn’t affect my rating. If my team loses, I lose rating. I can play like a god, I still lose rating.

It’s just about powering through it and hope the next season placement will be luckier for me. It is pretty discouraging and demotivating though but oh well.

They’ve made a ranked system that’d worked for 5 queue teams and are using it for pugs so can’t expect much tbh.

Loosing is OK but i fear that the more matches you play the more you are stuck. You can improve or be much better then your place but you will crawl upwards even if yor actuall skill is 500 points higher. So the placement matters largely. Bad luck or a bad day and you need 100 matches more to get where you actuelly belong. Or do i see that wrong?
And no you can´t carry most matches. You increase your odds by being better and maybe do 60/40 not 50:50. So you are right when you say gettign or beeing better will move you up but the queston is how stony will it be. The lucky one with 9:1 can hold top level with a match a day the unlucky but skillful needs 500 to fight up ?
I like the system but there are open questions and improvments visible.

(edited by Wolfric.9380)