On August 08 2015 22:10 TokO wrote: I never had an issue understanding Sc2 ranks.
No one does. E.g. if you're in gold league in SC2, you're in the top 40% to top 72%. This is 16 times less precision than know you're rank 20 in HotS which is top 40% to top 42%.
You just have to realise that there is significant overlap in leagues. To advance in league, you need to play consistently better than roughly 50% of the next league.
Why is this good? Why isn't the transparent HotS system that Dustin Browder advocates better?
The system requires stability in winrate because that's the best way to ensure you're correctly placed, to precede a promotion. It was just a solution they used to avoid players bouncing between two leagues, because that would screw with the idea of stable divisions in which a player could compare themselves to others.
Divisions are meanginless, so who cares if they are stable. The leagues are wrong. You can have higher MMR than your league and not get promoted, or lower MMR and not get demoted. This is one reason why HotS scraps leagues and replaces with an accurate, fluid ranking system based on a continuum of points, not buckets that are discrete, arbitrary and wrong.
Well, the reason I think it is better is that there is more justification for anyone's rank. Looking at games played that season and placements in division, I have a pretty good idea about another player's skill that season. Obviously, I haven't played the latest iterations with 20-30 games placement in Heroes, but when I played, my team would almost always consist of a shotgun shot of different ranks, and I had no idea of how to translate that into anything comprehensive. You kinda have this image of 50 ranks system being a perfect indicator of skill, when there are so many factors that could distort the indication, and distort it quite easily.
I think a lot of people level a lot of hate towards the SC2 system because they aren't getting a promotion when they believe that they "deserve" it. The opposite is actually the case. I think promotion in SC2 is a much stronger indicator because the rigidity means that the person actually deserved his promotion when he finally achieves it. You only move in SC2 when you're consistent, and when the amount of games played is high, it becomes a consistent indicator. On the other hand, in systems where you fluctuate a lot because of unlucky teams, being carried, etc. the ranking means a lot less.
On August 08 2015 23:25 TimeSpiral wrote: I've alway thought matchmaking in SC2 was pretty good--significantly better than many other games, in other genres.
They're improving ranking accuracy in Heres because matchmaking is horribly, horribly bad, atm. But, it's also a wildly different system. Not even apples to apples. More like, apples to hockey, lol.
Improving the SC2 matchmaking should--imho--have only one goal: increase enjoyment for rec-level players. It is not fun to lose a match because you never had a chance. If possible, the vast majority of games should be up against opponents of similar skill. I believe the SC2 matchmaking already does this fairly well, but could probably be improved.
So the SC2 league system appears to be clunky, and perhaps less accurate because there are a small number of divisions. These are just shiny objects to get you to play the game more, lol. I strongly suspect your league badge has very little to do with the matches you play. But *shrugs* that's just me.
The other issue is the number of players and the quality of time playing. Is it better to have longer waits and closer matches, or shorter waits and total stomp-fests? This is a user experience issue that must be addressed, and it can be a downward spiral. Less people playing means the rec-level players will have a harder time having fun with the game.
At the rec-level, it needs to be fun first. If you don't think you're recreational, then none of this matters. You're super tryhard and dedicated, and your MMR should do a decent job at keeping your W/L at 50%. But what the hell do I know?
No one is complaining about the matchmaking. A good matchmaker is only possible with a good estimate of skill (MMR). Therefore, the existence of good matchmaking implies the existence of a good estimate of skill. The problem is that Blizzard is not using this good estimate of skill for ranking. Instead, they deliberately distort ranks, deliberately making them inaccurate, when they could just use that good estimate of skill for ranking as Dustin Browder now wants to do in HotS (and that's the right decision).
On August 08 2015 22:10 TokO wrote: I never had an issue understanding Sc2 ranks.
No one does. E.g. if you're in gold league in SC2, you're in the top 40% to top 72%. This is 16 times less precision than know you're rank 20 in HotS which is top 40% to top 42%.
You just have to realise that there is significant overlap in leagues. To advance in league, you need to play consistently better than roughly 50% of the next league.
Why is this good? Why isn't the transparent HotS system that Dustin Browder advocates better?
The system requires stability in winrate because that's the best way to ensure you're correctly placed, to precede a promotion. It was just a solution they used to avoid players bouncing between two leagues, because that would screw with the idea of stable divisions in which a player could compare themselves to others.
Divisions are meanginless, so who cares if they are stable. The leagues are wrong. You can have higher MMR than your league and not get promoted, or lower MMR and not get demoted. This is one reason why HotS scraps leagues and replaces with an accurate, fluid ranking system based on a continuum of points, not buckets that are discrete, arbitrary and wrong.
Well, the reason I think it is better is that there is more justification for anyone's rank. Looking at games played that season and placements in division, I have a pretty good idea about another player's skill that season. Obviously, I haven't played the latest iterations with 20-30 games placement in Heroes, but when I played, my team would almost always consist of a shotgun shot of different ranks, and I had no idea of how to translate that into anything comprehensive. You kinda have this image of 50 ranks system being a perfect indicator of skill, when there are so many factors that could distort the indication, and distort it quite easily.
I think a lot of people level a lot of hate towards the SC2 system because they aren't getting a promotion when they believe that they "deserve" it. The opposite is actually the case. I think promotion in SC2 is a much stronger indicator because the rigidity means that the person actually deserved his promotion when he finally achieves it. You only move in SC2 when you're consistent, and when the amount of games played is high, it becomes a consistent indicator. On the other hand, in systems where you fluctuate a lot because of unlucky teams, being carried, etc. the ranking means a lot less.
How do you justify a wrong rank? Even if leagues require "consistency" (ExcaliburZ says that the requirement of low MMR uncertainty to be promoted has been scrapped, so your claim here is probably false), there are many reasons why someone is in a league. They could have just been put there after 5 placement games, rather than due to "consistency", they could badly deserve to be demoted but are locked in, they could deserve a promotion but Blizzard forgot to adjust the league boundary (it seems, they have to calculate this manually each time) so they don't get the promotion that they do actually deserve. You also have bonus pool, intrinsically making your rank decline every single hour. Moreover, a league can be 32% of players. So really, when you know the league, you don't know anything.
As you point out, a problem with promotions is that people think they deserve one and the promotion criteria is not transparent, and gives very little indication of skill with up to 16 times less precision than HotS ranks. That's a reason to scrap leagues as has been done in HotS.
You frequently get matched to people in different leagues in SC2. That's because players' distorted ranks are different from the MMR skill rating. That's the problem that Dustin Browder says he will now fix in HotS by making ranks based off what is essentially the percentile of MMR once placement matches have been done.
What are these factors that "distort the indication [of skill], and distort it quite easily"? Performance at landing skill-shots? The meaning of ranks is clear, it's suppose to measure your ability to win games, so that's correctly taken into account. Bad teammates? That's equally random for both sides, so it increases the standard error but not bias. Compare that to distortions like no demotion in SC2 and bonus pool. That doesn't increase standard error, it increases bias. Systemic bias.
On August 08 2015 22:10 TokO wrote: I never had an issue understanding Sc2 ranks.
No one does. E.g. if you're in gold league in SC2, you're in the top 40% to top 72%. This is 16 times less precision than know you're rank 20 in HotS which is top 40% to top 42%.
You just have to realise that there is significant overlap in leagues. To advance in league, you need to play consistently better than roughly 50% of the next league.
Why is this good? Why isn't the transparent HotS system that Dustin Browder advocates better?
The system requires stability in winrate because that's the best way to ensure you're correctly placed, to precede a promotion. It was just a solution they used to avoid players bouncing between two leagues, because that would screw with the idea of stable divisions in which a player could compare themselves to others.
Divisions are meanginless, so who cares if they are stable. The leagues are wrong. You can have higher MMR than your league and not get promoted, or lower MMR and not get demoted. This is one reason why HotS scraps leagues and replaces with an accurate, fluid ranking system based on a continuum of points, not buckets that are discrete, arbitrary and wrong.
Well, the reason I think it is better is that there is more justification for anyone's rank. Looking at games played that season and placements in division, I have a pretty good idea about another player's skill that season. Obviously, I haven't played the latest iterations with 20-30 games placement in Heroes, but when I played, my team would almost always consist of a shotgun shot of different ranks, and I had no idea of how to translate that into anything comprehensive. You kinda have this image of 50 ranks system being a perfect indicator of skill, when there are so many factors that could distort the indication, and distort it quite easily.
I think a lot of people level a lot of hate towards the SC2 system because they aren't getting a promotion when they believe that they "deserve" it. The opposite is actually the case. I think promotion in SC2 is a much stronger indicator because the rigidity means that the person actually deserved his promotion when he finally achieves it. You only move in SC2 when you're consistent, and when the amount of games played is high, it becomes a consistent indicator. On the other hand, in systems where you fluctuate a lot because of unlucky teams, being carried, etc. the ranking means a lot less.
How do you justify a wrong rank? Even if leagues requires "consistency" (ExcaliburZ says that the requirement of low MMR uncertainty to be promoted has been scrapped, so this might not even be true), there are many reasons why someone is a league. They could have just been put there after 5 games, rather due to "consistency", they could so bad they deserve to be demoted but are locked in, they could deserve a promotion but Blizzard forgot to adjust the league boundary (they have calculate this manually each time, it seems) so they don't get one. Moreover, a league can be 32% of players. So really, when you know the league, you don't know anything.
As you point out, a problem with promotions because people think they deserve one and the promotion criteria is not transparent, and leagues given very liltte indication of skills, with up to 16 times less precision than HotS ranks. That's a reason to scrap leagues as has been done in HotS.
You frequently get matched to people in different leagues in SC2. That's because player's distorted ranked is different from the MMR skill rating. That's the problem that Dustin Browder says he will now fix in HotS by making ranks based off what is the percentile of MMR after placement matches.
That's why you need to look at the amount of games played. The more games, the stronger the indication. Fewer games means the rank is a worse indicator of skill. This is going to be the case regardless of system.
I don't really accept your argument of ranks in Heroes being 16 times more precise. If the placements lack proper justification, and can more easily due to factors outside of player skill, then the ranks cease to be an accurate indicator. It seems that you've made an assumption that Heroes ranking system is always accurate when it places you, and that SC2 is always inaccurate, because of leagues. You also argue with the assumption that all gold-leaguers are the same, all platinum-leaguers are the same, etc. But they are not. If we compare the two systems fairly, there is actually 700 ranks different ranks in SC2 that players can tend towards, making it a lot more precise.
Ranks will always be distorted to a certain degree, due to skill decay, rank decay, changes in skill of playerbase composition. That's why you meet people of different leagues and ranks in SC2. It's really naive to believe that Heroes will successfully have each of their 2% brackets perfectly represent the distribution of playerbase's skill in Heroes. As long as some people don't play until they are accurately placed, play deliberately worse, stop playing, only play once in a blue moon, etc. there will be inconsistencies established in Heroes. The same things that you are attacking the SC2 system for.
EDIT: Yeah, I'd definitely want a ranking system with smaller standard deviation and revealed bias than a system with a larger standard deviation.
if you showed mmr then there is no need for ranks. I think most feel this would be ideal.
since mmr is hidden, and the system is designed to give you 50% win rates..you need ranks to show improvement otherwise you feel like you're running in place.
On August 08 2015 22:10 TokO wrote: I never had an issue understanding Sc2 ranks.
No one does. E.g. if you're in gold league in SC2, you're in the top 40% to top 72%. This is 16 times less precision than know you're rank 20 in HotS which is top 40% to top 42%.
You just have to realise that there is significant overlap in leagues. To advance in league, you need to play consistently better than roughly 50% of the next league.
Why is this good? Why isn't the transparent HotS system that Dustin Browder advocates better?
The system requires stability in winrate because that's the best way to ensure you're correctly placed, to precede a promotion. It was just a solution they used to avoid players bouncing between two leagues, because that would screw with the idea of stable divisions in which a player could compare themselves to others.
Divisions are meanginless, so who cares if they are stable. The leagues are wrong. You can have higher MMR than your league and not get promoted, or lower MMR and not get demoted. This is one reason why HotS scraps leagues and replaces with an accurate, fluid ranking system based on a continuum of points, not buckets that are discrete, arbitrary and wrong.
Well, the reason I think it is better is that there is more justification for anyone's rank. Looking at games played that season and placements in division, I have a pretty good idea about another player's skill that season. Obviously, I haven't played the latest iterations with 20-30 games placement in Heroes, but when I played, my team would almost always consist of a shotgun shot of different ranks, and I had no idea of how to translate that into anything comprehensive. You kinda have this image of 50 ranks system being a perfect indicator of skill, when there are so many factors that could distort the indication, and distort it quite easily.
I think a lot of people level a lot of hate towards the SC2 system because they aren't getting a promotion when they believe that they "deserve" it. The opposite is actually the case. I think promotion in SC2 is a much stronger indicator because the rigidity means that the person actually deserved his promotion when he finally achieves it. You only move in SC2 when you're consistent, and when the amount of games played is high, it becomes a consistent indicator. On the other hand, in systems where you fluctuate a lot because of unlucky teams, being carried, etc. the ranking means a lot less.
How do you justify a wrong rank? Even if leagues requires "consistency" (ExcaliburZ says that the requirement of low MMR uncertainty to be promoted has been scrapped, so this might not even be true), there are many reasons why someone is a league. They could have just been put there after 5 games, rather due to "consistency", they could so bad they deserve to be demoted but are locked in, they could deserve a promotion but Blizzard forgot to adjust the league boundary (they have calculate this manually each time, it seems) so they don't get one. Moreover, a league can be 32% of players. So really, when you know the league, you don't know anything.
As you point out, a problem with promotions because people think they deserve one and the promotion criteria is not transparent, and leagues given very liltte indication of skills, with up to 16 times less precision than HotS ranks. That's a reason to scrap leagues as has been done in HotS.
You frequently get matched to people in different leagues in SC2. That's because player's distorted ranked is different from the MMR skill rating. That's the problem that Dustin Browder says he will now fix in HotS by making ranks based off what is the percentile of MMR after placement matches.
That's why you need to look at the amount of games played. The more games, the stronger the indication. Fewer games means the rank is a worse indicator of skill. This is going to be the case regardless of system.
No. Looking at the number of games played is not informative. The player could deserve a demotion after many games, and is locked in. Or the player could deserve to be demoted after playing small number of games because the initial placement was an overestimate. Or the player could have only a few games played but have very low uncertainty in the MMR due to lots of games played last season. Or the player could deserve to get promoted even with very few games played, but hasn't been promote because Blizzard forgot to manually adjust the league boundaries. It's also needlessly complicated to take so many things into account when you could just look at 1 number, like in HotS (behind the scenes, everything is based on 2 numbers).
I don't really accept your argument of ranks in Heroes being 16 times more precise.
They are 16 times more precise because they use groups of size 2%, which is 16 times more precise than groups of size 32%, by definition. Suppose SC2 used groups of size 50%. Does this not give less information? You could be the worst player or the median player, both are equal.
If the placements lack proper justification, and can more easily due to factors outside of player skill, then the ranks cease to be an accurate indicator. It seems that you've made an assumption that Heroes ranking system is always accurate when it places you, and that SC2 is always inaccurate, because of leagues.
You also argue with the assumption that all gold-leaguers are the same, all platinum-leaguers are the same, etc. But they are not. If we compare the two systems fairly, there is actually 700 ranks different ranks in SC2 that players can tend towards, making it a lot more precise.
Ranks will always be distorted to a certain degree, due to skill decay, rank decay, changes in skill of playerbase composition. That's why you meet people of different leagues and ranks in SC2. It's really naive to believe that Heroes will successfully have each of their 2% brackets perfectly represent the distribution of playerbase's skill in Heroes. As long as some people don't play until they are accurately placed, play deliberately worse, stop playing, only play once in a blue moon, etc. there will be inconsistencies established in Heroes. The same things that you are attacking the SC2 system for.
You are confused between the meaning of precision and accuracy.
"It seems that you've made an assumption that Heroes ranking system is always accurate when it places you, and that SC2 is always inaccurate, because of leagues." I have not made such an assumption.
Your argument is that factors X, Y and Z affect ranks, so they are wrong. This is basically the fallacy that because we don't know everything, we know nothing.
Some factors distort ranks (e.g. skill decay), some factors, contrary to your claims, don't distort ranks (e.g. changes in playerbase). These factors equally affect both games. I'm saying that there are far more factors that distort ranks in SC2 than in HotS: e.g. bonus pool and leagues. These distortionary factors don't exist in HotS. These SC2 distortionary factors are particularly bad because they increase bias, whereas bad teammates in HotS only increases standard error, but not bias. There are no distortionary factors in HotS that don't exist in SC2 in a like-for-like comparison that would even up the score. And Dustin Browder agrees, that's why he's removing these pointless distortionary factors to create accurate ranks using MMR.
On August 09 2015 00:23 Ignorant prodigy wrote: if you showed mmr then there is no need for ranks. I think most feel this would be ideal.
since mmr is hidden, and the system is designed to give you 50% win rates..you need ranks to show improvement otherwise you feel like you're running in place.
Yep.
This is exactly the reason Dustin Browder has given to refuse to expose MMR in SC2: because after around 10 to 20 games, your MMR stabilizes, and then you'll get stuck there, unchanging, unless you get more skill.
But now he has suddenly flip-flopped. Now he accepts this and says, if you want a higher rank, you can't count on grinding out many games like you can now (which works up to a point), instead you must get more skilled, otherwise, your rank will stagnate:
On August 08 2015 22:10 TokO wrote: I never had an issue understanding Sc2 ranks.
No one does. E.g. if you're in gold league in SC2, you're in the top 40% to top 72%. This is 16 times less precision than know you're rank 20 in HotS which is top 40% to top 42%.
You just have to realise that there is significant overlap in leagues. To advance in league, you need to play consistently better than roughly 50% of the next league.
Why is this good? Why isn't the transparent HotS system that Dustin Browder advocates better?
The system requires stability in winrate because that's the best way to ensure you're correctly placed, to precede a promotion. It was just a solution they used to avoid players bouncing between two leagues, because that would screw with the idea of stable divisions in which a player could compare themselves to others.
Divisions are meanginless, so who cares if they are stable. The leagues are wrong. You can have higher MMR than your league and not get promoted, or lower MMR and not get demoted. This is one reason why HotS scraps leagues and replaces with an accurate, fluid ranking system based on a continuum of points, not buckets that are discrete, arbitrary and wrong.
Well, the reason I think it is better is that there is more justification for anyone's rank. Looking at games played that season and placements in division, I have a pretty good idea about another player's skill that season. Obviously, I haven't played the latest iterations with 20-30 games placement in Heroes, but when I played, my team would almost always consist of a shotgun shot of different ranks, and I had no idea of how to translate that into anything comprehensive. You kinda have this image of 50 ranks system being a perfect indicator of skill, when there are so many factors that could distort the indication, and distort it quite easily.
I think a lot of people level a lot of hate towards the SC2 system because they aren't getting a promotion when they believe that they "deserve" it. The opposite is actually the case. I think promotion in SC2 is a much stronger indicator because the rigidity means that the person actually deserved his promotion when he finally achieves it. You only move in SC2 when you're consistent, and when the amount of games played is high, it becomes a consistent indicator. On the other hand, in systems where you fluctuate a lot because of unlucky teams, being carried, etc. the ranking means a lot less.
How do you justify a wrong rank? Even if leagues requires "consistency" (ExcaliburZ says that the requirement of low MMR uncertainty to be promoted has been scrapped, so this might not even be true), there are many reasons why someone is a league. They could have just been put there after 5 games, rather due to "consistency", they could so bad they deserve to be demoted but are locked in, they could deserve a promotion but Blizzard forgot to adjust the league boundary (they have calculate this manually each time, it seems) so they don't get one. Moreover, a league can be 32% of players. So really, when you know the league, you don't know anything.
As you point out, a problem with promotions because people think they deserve one and the promotion criteria is not transparent, and leagues given very liltte indication of skills, with up to 16 times less precision than HotS ranks. That's a reason to scrap leagues as has been done in HotS.
You frequently get matched to people in different leagues in SC2. That's because player's distorted ranked is different from the MMR skill rating. That's the problem that Dustin Browder says he will now fix in HotS by making ranks based off what is the percentile of MMR after placement matches.
That's why you need to look at the amount of games played. The more games, the stronger the indication. Fewer games means the rank is a worse indicator of skill. This is going to be the case regardless of system.
No. Looking at the number of games played is not informative. The player could deserve a demotion after many games, and is locked in, distorting his rank. Or the player could deserve to be demoted after playing small number of games because the initial placement was an overestimate.The player could have only a few games played but have very low uncertainty in the MMR due to lots of games played last season. Or the player could h very low uncertainty in the MMR due to lots of games played. So the number of games is not useful. It's also needlessly complicated to take so many things into account, when you could (and the system does) just look at the MMR. HotS uses 1 number.
I don't really accept your argument of ranks in Heroes being 16 times more precise.
They are 16 times more precise because they use groups of size 2%, which is 16 times more precise than groups of size 32%, by definition. Suppose SC2 used groups of size 50%. Does this not give less information? You could be the worst player or the median player, both are equal.
If the placements lack proper justification, and can more easily due to factors outside of player skill, then the ranks cease to be an accurate indicator. It seems that you've made an assumption that Heroes ranking system is always accurate when it places you, and that SC2 is always inaccurate, because of leagues.
You also argue with the assumption that all gold-leaguers are the same, all platinum-leaguers are the same, etc. But they are not. If we compare the two systems fairly, there is actually 700 ranks different ranks in SC2 that players can tend towards, making it a lot more precise.
Ranks will always be distorted to a certain degree, due to skill decay, rank decay, changes in skill of playerbase composition. That's why you meet people of different leagues and ranks in SC2. It's really naive to believe that Heroes will successfully have each of their 2% brackets perfectly represent the distribution of playerbase's skill in Heroes. As long as some people don't play until they are accurately placed, play deliberately worse, stop playing, only play once in a blue moon, etc. there will be inconsistencies established in Heroes. The same things that you are attacking the SC2 system for.
You are confused between the meaning of precision and accuracy.
"It seems that you've made an assumption that Heroes ranking system is always accurate when it places you, and that SC2 is always inaccurate, because of leagues." I have not made such an assumption.
You're argument that factors X, Y and Z affect ranks, so they are wrong. This is basically the fallacy that if we don't know everything so we know nothing.
I'm some factors distort ranks (e.g. skill decay), some factors, contrary to your claims, don't distort ranks (e.g. changes in playerbase). I'm saying that there is more factors that there are far more factors that distort ranks in SC2 that do exist in HotS: e.g. bonus pool, leagues. And Dustin Browder agrees, that's why his removing them.
When I am looking at the information that the SC2 system gives me, games played definitely gives me a relevant indicator of how accurate the given player's position is. If you think that a systems ability to accurately place a player after 5 games is the same as it is after 500 games, then obviously there is no reason of discussing anything. Obviously previous seasons matter, but other things being equal, many games recently tend to give a better indication of recent skill.
Do you understand that the system gives you more information than that a player is in gold? That there is a point count and a rank within each league? You evaluate the system as if the only feedback the system gave you was in which league you're in. A league can't really be compared directly to Heroes' system. So the 16 times more accurate statement is not true at all.
All I'm trying to argue is that the more rigid system makes the ranking evaluation more informative. You move relatively faster between ranks in Heroes so every rank conveys less information. There are a lot of factors that distorts both systems, maybe there are more systemic distortions in SC2, but as long as those are revealed and well understood, it doesn't make SC2 ranking that much worse all.
In doesn't really make a difference, as long as the system used matches you up with people who are the same MMR. That wasn't my experience in Hero League when I played (When you would move around being awarded between 100 and 200 points out of 300 per match). Obviously, as the ranks become more rigid and advancement slower, ranks in Heroes will become more informative. It just seems like your critique is very severe as a result of a really simplified understanding of the SC2 system. It's not that bad as you would have it.
from your source, timestamps 28:02 to 29:26 before 28:02 they talk about how the matchmaker matches teams (or more accurately, tries not to match teams) of different party sizes together, not about how well your rank represents your skill. after 29:26 they stop talking about matchmaking and start talking about when the next season might be coming up.
28:02 "it really sounds like the ranked system now comes a bit closer to, like, the hidden mmr"
"it'll be, yea it'll be much more realistic, so basically while we are determining your hidden mmr we are also determining your rank and we should be able to get you, then after that if you want to go up you are gonna have to really improve your skills to get there. so that will hopefully be a lot more transparent to players, this is something a little bit more similar to what we had in starcraft and so we are sort of going back a little bit more in that direction so far it feels really good to us we've had good feedback from the people we have talked about it with but you know its, its a work in progress if we get different feedback, we're not shy we are not ashamed, we will change it again if we absolutely need to."
"that's great" "yea"
"when it comes to the core matchmaking, obviously we want to keep doing improvements there as well, we've got engineers dedicated to nothing but that, going forward we've recently introduced a few changes that have made it a little bit better, one of the concerns we had from players was 'hey, y'know I've been playing for 500 games I know I'm not that good but I just got matched with someone who've played 50 games, maybe he's coming from another game in the genre, he's actually almost as good as I am but, he doesn't know the game nearly as well as I do and I get very frustrated playing with him cause I know how the map objectives work, I know how the talent system works, I know how to fight, maybe as a team better than he does, this guy drive me a little crazy, can you, can you get him by himself with other players, you know that are new to the game' so we made that change recently and it's upped wait times by about 20 or 30 seconds, but the feedback so far from our players has been, pretty positive so we are gonna keep that change and continue to look for more changes that help matchmaking going forward" 29:26
and from this you get "Dustin Browder says he wants to make the HotS rank system more accurate by displaying what is effectively MMR after placement games, and that it would be like how it works in SC2" and "This is NOT how it works in SC2, but it is how it SHOULD work"?
as far as I can tell the system DB is referencing in this interview is pretty much exactly the SC2 ranking system. 1. you do placement matches. 2. you are put where the game thinks you belong based on your MMR calculated from placement matches. 3. adjust from there as you play. this is how SC2 works. DB is saying they want this system in heroes rather than the existing system that works like this: 1. you start at rank 50. 2. adjust from there, you cannot lose rank points while you are rank 50-40.
I just don't see how any of your points is relevant, true the SC2 rank is not your true rank, but neither will the new heroes systems rank be your true rank. the rank is just a placeholder to indicate a rough number, how good you actually are playing vary wildly based on your mood, attitude, whether you are warmed up, whether you are on tilt, whether you are hungry/full, whether you are sick, whether you are drunk, whether you are on drugs (for medicinal purposes or otherwise), etc. there is no real point in telling you your MMR is 3726 if your true mmr varies between 3200 and 3800, its better and more informative to tell you "you are platinum rank which indicates your mmr is between 3400 and 4000".
for example, in SC2 your mmr isn't actually one number, its two numbers. the first number indicates your average skill, the second number indicates the systems confidence in your skill, you can think of the numbers as x and y where the system basically counts your true mmr as: x plusminus y if you win a game you were calculated to lose or lose a game you were calculated to win, then the system increases or decreases your x value as appropriate, but always increases your y value. this means when you surprise the systems expectations your skill is adjusted and the system thinks you are more volatile than it previously thought. if instead you win/lose as expected then your x is still adjusted in the same way, but your y is decreased. so: if you win and lose the way the system expects you to, then the system considers itself to have been correct about your ranking and gets more confident. if you win or lose against the systems prediction then the system thinks it might be wrong and gets less confident.
therefore, simply displaying the mmr is not a great method since your mmr is not so clearcut.
and p.s. parallelluniverse, avoid double, triple or quadrouple posting, its better to just edit your last message to include what you want to say in your other posts than to double-post. reason: double-posting makes it seem like the discussion is more relevant than it seems by bumping the topic to the top if the forum page, this is unwanted behaviour for threads with fairly few active participants.
so 25 placement matches in a 5v5 game, where you only make out 1/5 of the skill of one side. I really wonder why they need to be more precise in Heroes. But yeah I think they will take over the Heroes system. Which removes even more informations. But people seem to like it more. Not sure if it is because its so easy to get to rank 1 though x3. Oh I think because the false progression of the system isn't that obvious !
Also MMR is also just a number, but a rather high one, so only a few will have the same one at the same time. I share my Chess Elo with around 100 people, if you add two more digits into the calculation, the number of people would drop down and it would of course be more accurate. But point differences of about 1000 Points would suddenly not matter at all.
On August 08 2015 22:10 TokO wrote: I never had an issue understanding Sc2 ranks.
No one does. E.g. if you're in gold league in SC2, you're in the top 40% to top 72%. This is 16 times less precision than know you're rank 20 in HotS which is top 40% to top 42%.
You just have to realise that there is significant overlap in leagues. To advance in league, you need to play consistently better than roughly 50% of the next league.
Why is this good? Why isn't the transparent HotS system that Dustin Browder advocates better?
The system requires stability in winrate because that's the best way to ensure you're correctly placed, to precede a promotion. It was just a solution they used to avoid players bouncing between two leagues, because that would screw with the idea of stable divisions in which a player could compare themselves to others.
Divisions are meanginless, so who cares if they are stable. The leagues are wrong. You can have higher MMR than your league and not get promoted, or lower MMR and not get demoted. This is one reason why HotS scraps leagues and replaces with an accurate, fluid ranking system based on a continuum of points, not buckets that are discrete, arbitrary and wrong.
Well, the reason I think it is better is that there is more justification for anyone's rank. Looking at games played that season and placements in division, I have a pretty good idea about another player's skill that season. Obviously, I haven't played the latest iterations with 20-30 games placement in Heroes, but when I played, my team would almost always consist of a shotgun shot of different ranks, and I had no idea of how to translate that into anything comprehensive. You kinda have this image of 50 ranks system being a perfect indicator of skill, when there are so many factors that could distort the indication, and distort it quite easily.
I think a lot of people level a lot of hate towards the SC2 system because they aren't getting a promotion when they believe that they "deserve" it. The opposite is actually the case. I think promotion in SC2 is a much stronger indicator because the rigidity means that the person actually deserved his promotion when he finally achieves it. You only move in SC2 when you're consistent, and when the amount of games played is high, it becomes a consistent indicator. On the other hand, in systems where you fluctuate a lot because of unlucky teams, being carried, etc. the ranking means a lot less.
How do you justify a wrong rank? Even if leagues requires "consistency" (ExcaliburZ says that the requirement of low MMR uncertainty to be promoted has been scrapped, so this might not even be true), there are many reasons why someone is a league. They could have just been put there after 5 games, rather due to "consistency", they could so bad they deserve to be demoted but are locked in, they could deserve a promotion but Blizzard forgot to adjust the league boundary (they have calculate this manually each time, it seems) so they don't get one. Moreover, a league can be 32% of players. So really, when you know the league, you don't know anything.
As you point out, a problem with promotions because people think they deserve one and the promotion criteria is not transparent, and leagues given very liltte indication of skills, with up to 16 times less precision than HotS ranks. That's a reason to scrap leagues as has been done in HotS.
You frequently get matched to people in different leagues in SC2. That's because player's distorted ranked is different from the MMR skill rating. That's the problem that Dustin Browder says he will now fix in HotS by making ranks based off what is the percentile of MMR after placement matches.
That's why you need to look at the amount of games played. The more games, the stronger the indication. Fewer games means the rank is a worse indicator of skill. This is going to be the case regardless of system.
No. Looking at the number of games played is not informative. The player could deserve a demotion after many games, and is locked in, distorting his rank. Or the player could deserve to be demoted after playing small number of games because the initial placement was an overestimate.The player could have only a few games played but have very low uncertainty in the MMR due to lots of games played last season. Or the player could h very low uncertainty in the MMR due to lots of games played. So the number of games is not useful. It's also needlessly complicated to take so many things into account, when you could (and the system does) just look at the MMR. HotS uses 1 number.
I don't really accept your argument of ranks in Heroes being 16 times more precise.
They are 16 times more precise because they use groups of size 2%, which is 16 times more precise than groups of size 32%, by definition. Suppose SC2 used groups of size 50%. Does this not give less information? You could be the worst player or the median player, both are equal.
If the placements lack proper justification, and can more easily due to factors outside of player skill, then the ranks cease to be an accurate indicator. It seems that you've made an assumption that Heroes ranking system is always accurate when it places you, and that SC2 is always inaccurate, because of leagues.
You also argue with the assumption that all gold-leaguers are the same, all platinum-leaguers are the same, etc. But they are not. If we compare the two systems fairly, there is actually 700 ranks different ranks in SC2 that players can tend towards, making it a lot more precise.
Ranks will always be distorted to a certain degree, due to skill decay, rank decay, changes in skill of playerbase composition. That's why you meet people of different leagues and ranks in SC2. It's really naive to believe that Heroes will successfully have each of their 2% brackets perfectly represent the distribution of playerbase's skill in Heroes. As long as some people don't play until they are accurately placed, play deliberately worse, stop playing, only play once in a blue moon, etc. there will be inconsistencies established in Heroes. The same things that you are attacking the SC2 system for.
You are confused between the meaning of precision and accuracy.
"It seems that you've made an assumption that Heroes ranking system is always accurate when it places you, and that SC2 is always inaccurate, because of leagues." I have not made such an assumption.
You're argument that factors X, Y and Z affect ranks, so they are wrong. This is basically the fallacy that if we don't know everything so we know nothing.
I'm some factors distort ranks (e.g. skill decay), some factors, contrary to your claims, don't distort ranks (e.g. changes in playerbase). I'm saying that there is more factors that there are far more factors that distort ranks in SC2 that do exist in HotS: e.g. bonus pool, leagues. And Dustin Browder agrees, that's why his removing them.
When I am looking at the information that the SC2 system gives me, games played definitely gives me a relevant indicator of how accurate the given player's position is. If you think that a systems ability to accurately place a player after 5 games is the same as it is after 500 games, then obviously there is no reason of discussing anything. Obviously previous seasons matter, but other things being equal, many games recently tend to give a better indication of recent skill.
Do you understand that the system gives you more information than that a player is in gold? That there is a point count and a rank within each league? You evaluate the system as if the only feedback the system gave you was in which league you're in. A league can't really be compared directly to Heroes' system. So the 16 times more accurate statement is not true at all.
All I'm trying to argue is that the more rigid system makes the ranking evaluation more informative. You move relatively faster between ranks in Heroes so every rank conveys less information. There are a lot of factors that distorts both systems, maybe there are more systemic distortions in SC2, but as long as those are revealed and well understood, it doesn't make SC2 ranking that much worse all.
In doesn't really make a difference, as long as the system used matches you up with people who are the same MMR. That wasn't my experience in Hero League when I played (When you would move around being awarded between 100 and 200 points out of 300 per match). Obviously, as the ranks become more rigid and advancement slower, ranks in Heroes will become more informative. It just seems like your critique is very severe as a result of a really simplified understanding of the SC2 system. It's not that bad as you would have it.
No. Games played does not given an indication of how accurate your rank is and I have given 4 examples above, where games played is not informative.
Your MMR after 500 games is more accurate than after 5 games. But your rank after 500 games is NOT necessarily more accurate than after 5 games because of all the unique SC2 distortionary factors that creates bias, not just standard error, as I've shown in the examples above. So your understanding of ranking systems is very uninformed.
Your statement that HotS ranks contain less information is flat out false. What's more information: You're MMR percentile plus a bunch of biased distortionary factors is in the top 40% to 72% or your MMR percentile in is the top 40% to 42%? The latter, is far more information and far more accurate, which is why the HotS ranking system is being changed in this way.
On August 09 2015 01:29 Roblin wrote: from your source, timestamps 28:02 to 29:26 before 28:02 they talk about how the matchmaker matches teams (or more accurately, tries not to match teams) of different party sizes together, not about how well your rank represents your skill. after 29:26 they stop talking about matchmaking and start talking about when the next season might be coming up.
28:02 "it really sounds like the ranked system now comes a bit closer to, like, the hidden mmr"
"it'll be, yea it'll be much more realistic, so basically while we are determining your hidden mmr we are also determining your rank and we should be able to get you, then after that if you want to go up you are gonna have to really improve your skills to get there. so that will hopefully be a lot more transparent to players, this is something a little bit more similar to what we had in starcraft and so we are sort of going back a little bit more in that direction so far it feels really good to us we've had good feedback from the people we have talked about it with but you know its, its a work in progress if we get different feedback, we're not shy we are not ashamed, we will change it again if we absolutely need to."
"that's great" "yea"
"when it comes to the core matchmaking, obviously we want to keep doing improvements there as well, we've got engineers dedicated to nothing but that, going forward we've recently introduced a few changes that have made it a little bit better, one of the concerns we had from players was 'hey, y'know I've been playing for 500 games I know I'm not that good but I just got matched with someone who've played 50 games, maybe he's coming from another game in the genre, he's actually almost as good as I am but, he doesn't know the game nearly as well as I do and I get very frustrated playing with him cause I know how the map objectives work, I know how the talent system works, I know how to fight, maybe as a team better than he does, this guy drive me a little crazy, can you, can you get him by himself with other players, you know that are new to the game' so we made that change recently and it's upped wait times by about 20 or 30 seconds, but the feedback so far from our players has been, pretty positive so we are gonna keep that change and continue to look for more changes that help matchmaking going forward" 29:26
and from this you get "Dustin Browder says he wants to make the HotS rank system more accurate by displaying what is effectively MMR after placement games, and that it would be like how it works in SC2" and "This is NOT how it works in SC2, but it is how it SHOULD work"?
as far as I can tell the system DB is referencing in this interview is pretty much exactly the SC2 ranking system. 1. you do placement matches. 2. you are put where the game thinks you belong based on your MMR calculated from placement matches. 3. adjust from there as you play. this is how SC2 works. DB is saying they want this system in heroes rather than the existing system that works like this: 1. you start at rank 50. 2. adjust from there, you cannot lose rank points while you are rank 50-40.
No. Browder is talking about placing people in the rank they deserve after placement using MMR instead of a progression system that trends towards MMR. But in SC2, your rank is not accurate, not based on MMR, it is instead based on a progression system that trends towards MMR as your bonus pool is used up. It is only similar in the sense that you are put in a group after placement based on MMR. But the HotS groups are precise and accurate, with size of 2%, while the SC2 groups are neither precise nor accurate, with size of 32% (16 times less precise) and you're ranks are especially distorted by bonus pool that cause it to intrinsically decline every single hours, and rules about leagues.
I just don't see how any of your points is relevant, true the SC2 rank is not your true rank, but neither will the new heroes systems rank be your true rank. the rank is just a placeholder to indicate a rough number, how good you actually are playing vary wildly based on your mood, attitude, whether you are warmed up, whether you are on tilt, whether you are hungry/full, whether you are sick, whether you are drunk, whether you are on drugs (for medicinal purposes or otherwise), etc. there is no real point in telling you your MMR is 3726 if your true mmr varies between 3200 and 3800, its better and more informative to tell you "you are platinum rank which indicates your mmr is between 3400 and 4000".
for example, in SC2 your mmr isn't actually one number, its two numbers. the first number indicates your average skill, the second number indicates the systems confidence in your skill, you can think of the numbers as x and y where the system basically counts your true mmr as: x plusminus y if you win a game you were calculated to lose or lose a game you were calculated to win, then the system increases or decreases your x value as appropriate, but always increases your y value. this means when you surprise the systems expectations your skill is adjusted and the system thinks you are more volatile than it previously thought. if instead you win/lose as expected then your x is still adjusted in the same way, but your y is decreased. so: if you win and lose the way the system expects you to, then the system considers itself to have been correct about your ranking and gets more confident. if you win or lose against the systems prediction then the system thinks it might be wrong and gets less confident.
therefore, simply displaying the mmr is not a great method since your mmr is not so clearcut.
and p.s. parallelluniverse, avoid double, triple or quadrouple posting, its better to just edit your last message to include what you want to say in your other posts than to double-post. reason: double-posting makes it seem like the discussion is more relevant than it seems by bumping the topic to the top if the forum page, this is unwanted behaviour for threads with fairly few active participants.
Is a rank based on more distortionary factors with less precision more informative than a rank based on much fewer distortionary factors and more precision? You seem to think so.
If you want to be pedantic your MMR is not 2 numbers, it is a probability distribution, that is a real-valued function. So what's the point you were trying to make again? There was no point there relevant to the discussion. Any ranking system will have to summarize the information into 1 number. So the question is, is it better that this 1 number summary be distorted by biased-inducing factors like bonus pool and no demotions than not?
there is no real point in telling you your MMR is 3726 if your true mmr varies between 3200 and 3800, its better and more informative to tell you "you are platinum rank which indicates your mmr is between 3400 and 4000".
there is no real point in telling you that the percentile corresponding to your rank is 37.26 if your true percentile corresponding to your true rank is 24.00 and 27.00, its obviously better and more informative to tell you the lie that "you are platinum rank which indicates your true percentile corresponding to your true rank is between 34.00 and 40.00".
On August 08 2015 22:10 TokO wrote: I never had an issue understanding Sc2 ranks.
No one does. E.g. if you're in gold league in SC2, you're in the top 40% to top 72%. This is 16 times less precision than know you're rank 20 in HotS which is top 40% to top 42%.
You just have to realise that there is significant overlap in leagues. To advance in league, you need to play consistently better than roughly 50% of the next league.
Why is this good? Why isn't the transparent HotS system that Dustin Browder advocates better?
The system requires stability in winrate because that's the best way to ensure you're correctly placed, to precede a promotion. It was just a solution they used to avoid players bouncing between two leagues, because that would screw with the idea of stable divisions in which a player could compare themselves to others.
Divisions are meanginless, so who cares if they are stable. The leagues are wrong. You can have higher MMR than your league and not get promoted, or lower MMR and not get demoted. This is one reason why HotS scraps leagues and replaces with an accurate, fluid ranking system based on a continuum of points, not buckets that are discrete, arbitrary and wrong.
Well, the reason I think it is better is that there is more justification for anyone's rank. Looking at games played that season and placements in division, I have a pretty good idea about another player's skill that season. Obviously, I haven't played the latest iterations with 20-30 games placement in Heroes, but when I played, my team would almost always consist of a shotgun shot of different ranks, and I had no idea of how to translate that into anything comprehensive. You kinda have this image of 50 ranks system being a perfect indicator of skill, when there are so many factors that could distort the indication, and distort it quite easily.
I think a lot of people level a lot of hate towards the SC2 system because they aren't getting a promotion when they believe that they "deserve" it. The opposite is actually the case. I think promotion in SC2 is a much stronger indicator because the rigidity means that the person actually deserved his promotion when he finally achieves it. You only move in SC2 when you're consistent, and when the amount of games played is high, it becomes a consistent indicator. On the other hand, in systems where you fluctuate a lot because of unlucky teams, being carried, etc. the ranking means a lot less.
How do you justify a wrong rank? Even if leagues requires "consistency" (ExcaliburZ says that the requirement of low MMR uncertainty to be promoted has been scrapped, so this might not even be true), there are many reasons why someone is a league. They could have just been put there after 5 games, rather due to "consistency", they could so bad they deserve to be demoted but are locked in, they could deserve a promotion but Blizzard forgot to adjust the league boundary (they have calculate this manually each time, it seems) so they don't get one. Moreover, a league can be 32% of players. So really, when you know the league, you don't know anything.
As you point out, a problem with promotions because people think they deserve one and the promotion criteria is not transparent, and leagues given very liltte indication of skills, with up to 16 times less precision than HotS ranks. That's a reason to scrap leagues as has been done in HotS.
You frequently get matched to people in different leagues in SC2. That's because player's distorted ranked is different from the MMR skill rating. That's the problem that Dustin Browder says he will now fix in HotS by making ranks based off what is the percentile of MMR after placement matches.
That's why you need to look at the amount of games played. The more games, the stronger the indication. Fewer games means the rank is a worse indicator of skill. This is going to be the case regardless of system.
No. Looking at the number of games played is not informative. The player could deserve a demotion after many games, and is locked in, distorting his rank. Or the player could deserve to be demoted after playing small number of games because the initial placement was an overestimate.The player could have only a few games played but have very low uncertainty in the MMR due to lots of games played last season. Or the player could h very low uncertainty in the MMR due to lots of games played. So the number of games is not useful. It's also needlessly complicated to take so many things into account, when you could (and the system does) just look at the MMR. HotS uses 1 number.
I don't really accept your argument of ranks in Heroes being 16 times more precise.
They are 16 times more precise because they use groups of size 2%, which is 16 times more precise than groups of size 32%, by definition. Suppose SC2 used groups of size 50%. Does this not give less information? You could be the worst player or the median player, both are equal.
If the placements lack proper justification, and can more easily due to factors outside of player skill, then the ranks cease to be an accurate indicator. It seems that you've made an assumption that Heroes ranking system is always accurate when it places you, and that SC2 is always inaccurate, because of leagues.
You also argue with the assumption that all gold-leaguers are the same, all platinum-leaguers are the same, etc. But they are not. If we compare the two systems fairly, there is actually 700 ranks different ranks in SC2 that players can tend towards, making it a lot more precise.
Ranks will always be distorted to a certain degree, due to skill decay, rank decay, changes in skill of playerbase composition. That's why you meet people of different leagues and ranks in SC2. It's really naive to believe that Heroes will successfully have each of their 2% brackets perfectly represent the distribution of playerbase's skill in Heroes. As long as some people don't play until they are accurately placed, play deliberately worse, stop playing, only play once in a blue moon, etc. there will be inconsistencies established in Heroes. The same things that you are attacking the SC2 system for.
You are confused between the meaning of precision and accuracy.
"It seems that you've made an assumption that Heroes ranking system is always accurate when it places you, and that SC2 is always inaccurate, because of leagues." I have not made such an assumption.
You're argument that factors X, Y and Z affect ranks, so they are wrong. This is basically the fallacy that if we don't know everything so we know nothing.
I'm some factors distort ranks (e.g. skill decay), some factors, contrary to your claims, don't distort ranks (e.g. changes in playerbase). I'm saying that there is more factors that there are far more factors that distort ranks in SC2 that do exist in HotS: e.g. bonus pool, leagues. And Dustin Browder agrees, that's why his removing them.
When I am looking at the information that the SC2 system gives me, games played definitely gives me a relevant indicator of how accurate the given player's position is. If you think that a systems ability to accurately place a player after 5 games is the same as it is after 500 games, then obviously there is no reason of discussing anything. Obviously previous seasons matter, but other things being equal, many games recently tend to give a better indication of recent skill.
Do you understand that the system gives you more information than that a player is in gold? That there is a point count and a rank within each league? You evaluate the system as if the only feedback the system gave you was in which league you're in. A league can't really be compared directly to Heroes' system. So the 16 times more accurate statement is not true at all.
All I'm trying to argue is that the more rigid system makes the ranking evaluation more informative. You move relatively faster between ranks in Heroes so every rank conveys less information. There are a lot of factors that distorts both systems, maybe there are more systemic distortions in SC2, but as long as those are revealed and well understood, it doesn't make SC2 ranking that much worse all.
In doesn't really make a difference, as long as the system used matches you up with people who are the same MMR. That wasn't my experience in Hero League when I played (When you would move around being awarded between 100 and 200 points out of 300 per match). Obviously, as the ranks become more rigid and advancement slower, ranks in Heroes will become more informative. It just seems like your critique is very severe as a result of a really simplified understanding of the SC2 system. It's not that bad as you would have it.
No. Games played does not given an indication of how accurate your rank is and I have given 4 examples above, where games played is not informative.
Your MMR after 500 games is more accurate than after 5 games. But your rank after 500 games is NOT necessarily more accurate than after 5 games because of all the unique SC2 distortionary factors that creates bias, not just standard error, as I've shown in the examples above. So your understanding of ranking systems is very uninformed.
Your statement that HotS ranks contain less information is flat out false. What's more information: You're MMR percentile plus a bunch of biased distortionary factors is in the top 40% to 72% or your MMR percentile in is the top 40% to 42%? The latter, is far more information and far more accurate, which is why the HotS ranking system is being changed in this way.
you ignore a very simple fact. pretty much everyone in heroes jump up and down in ranks pretty wildly. I myself vary between rank 27 and rank 22 depending on how well im playing that particular day and also of course, varing based on luck (aka unpredictable factors favouring one side over the other). so for me, if I am in rank 25 it is misleading to claim that means I am in percentages 50% to 52% when in reality I vary betweeen percentages 44% to 54%
and besides, it not like SC2 gives no indication of where in your division you are, true its built in leagues which give a wide range, but your placement within the division tells you whether you are closer to 40th percentile rather than 70th percentile.
to any intelligent person that is interested to know one own approximate standing, SC2 is no better or worse than the suggested model for heroes.
edit: also, again, avoid double-posting. I could have written this in a new post, but that would be double-posting, so I didn't. I edited my last message instead.
On August 08 2015 22:10 TokO wrote: I never had an issue understanding Sc2 ranks.
No one does. E.g. if you're in gold league in SC2, you're in the top 40% to top 72%. This is 16 times less precision than know you're rank 20 in HotS which is top 40% to top 42%.
You just have to realise that there is significant overlap in leagues. To advance in league, you need to play consistently better than roughly 50% of the next league.
Why is this good? Why isn't the transparent HotS system that Dustin Browder advocates better?
The system requires stability in winrate because that's the best way to ensure you're correctly placed, to precede a promotion. It was just a solution they used to avoid players bouncing between two leagues, because that would screw with the idea of stable divisions in which a player could compare themselves to others.
Divisions are meanginless, so who cares if they are stable. The leagues are wrong. You can have higher MMR than your league and not get promoted, or lower MMR and not get demoted. This is one reason why HotS scraps leagues and replaces with an accurate, fluid ranking system based on a continuum of points, not buckets that are discrete, arbitrary and wrong.
Well, the reason I think it is better is that there is more justification for anyone's rank. Looking at games played that season and placements in division, I have a pretty good idea about another player's skill that season. Obviously, I haven't played the latest iterations with 20-30 games placement in Heroes, but when I played, my team would almost always consist of a shotgun shot of different ranks, and I had no idea of how to translate that into anything comprehensive. You kinda have this image of 50 ranks system being a perfect indicator of skill, when there are so many factors that could distort the indication, and distort it quite easily.
I think a lot of people level a lot of hate towards the SC2 system because they aren't getting a promotion when they believe that they "deserve" it. The opposite is actually the case. I think promotion in SC2 is a much stronger indicator because the rigidity means that the person actually deserved his promotion when he finally achieves it. You only move in SC2 when you're consistent, and when the amount of games played is high, it becomes a consistent indicator. On the other hand, in systems where you fluctuate a lot because of unlucky teams, being carried, etc. the ranking means a lot less.
How do you justify a wrong rank? Even if leagues requires "consistency" (ExcaliburZ says that the requirement of low MMR uncertainty to be promoted has been scrapped, so this might not even be true), there are many reasons why someone is a league. They could have just been put there after 5 games, rather due to "consistency", they could so bad they deserve to be demoted but are locked in, they could deserve a promotion but Blizzard forgot to adjust the league boundary (they have calculate this manually each time, it seems) so they don't get one. Moreover, a league can be 32% of players. So really, when you know the league, you don't know anything.
As you point out, a problem with promotions because people think they deserve one and the promotion criteria is not transparent, and leagues given very liltte indication of skills, with up to 16 times less precision than HotS ranks. That's a reason to scrap leagues as has been done in HotS.
You frequently get matched to people in different leagues in SC2. That's because player's distorted ranked is different from the MMR skill rating. That's the problem that Dustin Browder says he will now fix in HotS by making ranks based off what is the percentile of MMR after placement matches.
That's why you need to look at the amount of games played. The more games, the stronger the indication. Fewer games means the rank is a worse indicator of skill. This is going to be the case regardless of system.
No. Looking at the number of games played is not informative. The player could deserve a demotion after many games, and is locked in, distorting his rank. Or the player could deserve to be demoted after playing small number of games because the initial placement was an overestimate.The player could have only a few games played but have very low uncertainty in the MMR due to lots of games played last season. Or the player could h very low uncertainty in the MMR due to lots of games played. So the number of games is not useful. It's also needlessly complicated to take so many things into account, when you could (and the system does) just look at the MMR. HotS uses 1 number.
I don't really accept your argument of ranks in Heroes being 16 times more precise.
They are 16 times more precise because they use groups of size 2%, which is 16 times more precise than groups of size 32%, by definition. Suppose SC2 used groups of size 50%. Does this not give less information? You could be the worst player or the median player, both are equal.
If the placements lack proper justification, and can more easily due to factors outside of player skill, then the ranks cease to be an accurate indicator. It seems that you've made an assumption that Heroes ranking system is always accurate when it places you, and that SC2 is always inaccurate, because of leagues.
You also argue with the assumption that all gold-leaguers are the same, all platinum-leaguers are the same, etc. But they are not. If we compare the two systems fairly, there is actually 700 ranks different ranks in SC2 that players can tend towards, making it a lot more precise.
Ranks will always be distorted to a certain degree, due to skill decay, rank decay, changes in skill of playerbase composition. That's why you meet people of different leagues and ranks in SC2. It's really naive to believe that Heroes will successfully have each of their 2% brackets perfectly represent the distribution of playerbase's skill in Heroes. As long as some people don't play until they are accurately placed, play deliberately worse, stop playing, only play once in a blue moon, etc. there will be inconsistencies established in Heroes. The same things that you are attacking the SC2 system for.
You are confused between the meaning of precision and accuracy.
"It seems that you've made an assumption that Heroes ranking system is always accurate when it places you, and that SC2 is always inaccurate, because of leagues." I have not made such an assumption.
You're argument that factors X, Y and Z affect ranks, so they are wrong. This is basically the fallacy that if we don't know everything so we know nothing.
I'm some factors distort ranks (e.g. skill decay), some factors, contrary to your claims, don't distort ranks (e.g. changes in playerbase). I'm saying that there is more factors that there are far more factors that distort ranks in SC2 that do exist in HotS: e.g. bonus pool, leagues. And Dustin Browder agrees, that's why his removing them.
When I am looking at the information that the SC2 system gives me, games played definitely gives me a relevant indicator of how accurate the given player's position is. If you think that a systems ability to accurately place a player after 5 games is the same as it is after 500 games, then obviously there is no reason of discussing anything. Obviously previous seasons matter, but other things being equal, many games recently tend to give a better indication of recent skill.
Do you understand that the system gives you more information than that a player is in gold? That there is a point count and a rank within each league? You evaluate the system as if the only feedback the system gave you was in which league you're in. A league can't really be compared directly to Heroes' system. So the 16 times more accurate statement is not true at all.
All I'm trying to argue is that the more rigid system makes the ranking evaluation more informative. You move relatively faster between ranks in Heroes so every rank conveys less information. There are a lot of factors that distorts both systems, maybe there are more systemic distortions in SC2, but as long as those are revealed and well understood, it doesn't make SC2 ranking that much worse all.
In doesn't really make a difference, as long as the system used matches you up with people who are the same MMR. That wasn't my experience in Hero League when I played (When you would move around being awarded between 100 and 200 points out of 300 per match). Obviously, as the ranks become more rigid and advancement slower, ranks in Heroes will become more informative. It just seems like your critique is very severe as a result of a really simplified understanding of the SC2 system. It's not that bad as you would have it.
No. Games played does not given an indication of how accurate your rank is and I have given 4 examples above, where games played is not informative.
Your MMR after 500 games is more accurate than after 5 games. But your rank after 500 games is NOT necessarily more accurate than after 5 games because of all the unique SC2 distortionary factors that creates bias, not just standard error, as I've shown in the examples above. So your understanding of ranking systems is very uninformed.
Your statement that HotS ranks contain less information is flat out false. What's more information: You're MMR percentile plus a bunch of biased distortionary factors is in the top 40% to 72% or your MMR percentile in is the top 40% to 42%? The latter, is far more information and far more accurate, which is why the HotS ranking system is being changed in this way.
you ignore a very simple fact. pretty much everyone in heroes jump up and down in ranks pretty wildly. I myself vary between rank 27 and rank 22 depending on how well im playing that particular day and also of course, varing based on luck (aka unpredictable factors favouring one side over the other). so for me, if I am in rank 25 it is misleading to claim that means I am in percentages 50% to 52% when in reality I vary betweeen percentages 44% to 54%
and besides, it not like SC2 gives no indication of where in your division you are, true its built in leagues which give a wide range, but your placement within the division tells you whether you are closer to 40th percentile rather than 70th percentile.
to any intelligent person that is interested to know one own approximate standing, SC2 is no better or worse than the suggested model for heroes.
Firstly, 44% to 54% is not 40% to 72%, which is what you'll get in SC2.
Secondly, if someone is 44% to 54%, and another person is 48% to 53%, who should be ranked higher? For any ranking system, even the SC2 ranking system, you need to collapse this information into 1 number, such as a mean, 49% vs 50.5%, so the second should be higher. The problem is that the 1 number in SC2 is wrong, and in HotS it's far less wrong.
Thirdly, why does your rank being between 44% to 54% imply it should be deliberately biased with distortionary factors like bonus pool and no demotions, and made up to 16 times (a number not chosen based on the data but pulled out of Blizzard's ass) less precise than HotS ranks?
Fourthly, division ranks are meaningless and wrong. They intrinsically decline every hour due to the bonus pool, can't be accurately compared when bonus pool is not spent, and doesn't account for the fact that some people don't deserve to be in the division but are stuck there due to rigidities in the league system.
Lastly, if you want less volatility in ranks, the solution is not the SC2 ranking system of wrong ranks, it's to apply a trend filter to the metric that is used to create the percentile ranks (approximate MMR in HotS or points in SC2).
On August 08 2015 22:10 TokO wrote: I never had an issue understanding Sc2 ranks.
No one does. E.g. if you're in gold league in SC2, you're in the top 40% to top 72%. This is 16 times less precision than know you're rank 20 in HotS which is top 40% to top 42%.
You just have to realise that there is significant overlap in leagues. To advance in league, you need to play consistently better than roughly 50% of the next league.
Why is this good? Why isn't the transparent HotS system that Dustin Browder advocates better?
The system requires stability in winrate because that's the best way to ensure you're correctly placed, to precede a promotion. It was just a solution they used to avoid players bouncing between two leagues, because that would screw with the idea of stable divisions in which a player could compare themselves to others.
Divisions are meanginless, so who cares if they are stable. The leagues are wrong. You can have higher MMR than your league and not get promoted, or lower MMR and not get demoted. This is one reason why HotS scraps leagues and replaces with an accurate, fluid ranking system based on a continuum of points, not buckets that are discrete, arbitrary and wrong.
Well, the reason I think it is better is that there is more justification for anyone's rank. Looking at games played that season and placements in division, I have a pretty good idea about another player's skill that season. Obviously, I haven't played the latest iterations with 20-30 games placement in Heroes, but when I played, my team would almost always consist of a shotgun shot of different ranks, and I had no idea of how to translate that into anything comprehensive. You kinda have this image of 50 ranks system being a perfect indicator of skill, when there are so many factors that could distort the indication, and distort it quite easily.
I think a lot of people level a lot of hate towards the SC2 system because they aren't getting a promotion when they believe that they "deserve" it. The opposite is actually the case. I think promotion in SC2 is a much stronger indicator because the rigidity means that the person actually deserved his promotion when he finally achieves it. You only move in SC2 when you're consistent, and when the amount of games played is high, it becomes a consistent indicator. On the other hand, in systems where you fluctuate a lot because of unlucky teams, being carried, etc. the ranking means a lot less.
How do you justify a wrong rank? Even if leagues requires "consistency" (ExcaliburZ says that the requirement of low MMR uncertainty to be promoted has been scrapped, so this might not even be true), there are many reasons why someone is a league. They could have just been put there after 5 games, rather due to "consistency", they could so bad they deserve to be demoted but are locked in, they could deserve a promotion but Blizzard forgot to adjust the league boundary (they have calculate this manually each time, it seems) so they don't get one. Moreover, a league can be 32% of players. So really, when you know the league, you don't know anything.
As you point out, a problem with promotions because people think they deserve one and the promotion criteria is not transparent, and leagues given very liltte indication of skills, with up to 16 times less precision than HotS ranks. That's a reason to scrap leagues as has been done in HotS.
You frequently get matched to people in different leagues in SC2. That's because player's distorted ranked is different from the MMR skill rating. That's the problem that Dustin Browder says he will now fix in HotS by making ranks based off what is the percentile of MMR after placement matches.
That's why you need to look at the amount of games played. The more games, the stronger the indication. Fewer games means the rank is a worse indicator of skill. This is going to be the case regardless of system.
No. Looking at the number of games played is not informative. The player could deserve a demotion after many games, and is locked in, distorting his rank. Or the player could deserve to be demoted after playing small number of games because the initial placement was an overestimate.The player could have only a few games played but have very low uncertainty in the MMR due to lots of games played last season. Or the player could h very low uncertainty in the MMR due to lots of games played. So the number of games is not useful. It's also needlessly complicated to take so many things into account, when you could (and the system does) just look at the MMR. HotS uses 1 number.
I don't really accept your argument of ranks in Heroes being 16 times more precise.
They are 16 times more precise because they use groups of size 2%, which is 16 times more precise than groups of size 32%, by definition. Suppose SC2 used groups of size 50%. Does this not give less information? You could be the worst player or the median player, both are equal.
If the placements lack proper justification, and can more easily due to factors outside of player skill, then the ranks cease to be an accurate indicator. It seems that you've made an assumption that Heroes ranking system is always accurate when it places you, and that SC2 is always inaccurate, because of leagues.
You also argue with the assumption that all gold-leaguers are the same, all platinum-leaguers are the same, etc. But they are not. If we compare the two systems fairly, there is actually 700 ranks different ranks in SC2 that players can tend towards, making it a lot more precise.
Ranks will always be distorted to a certain degree, due to skill decay, rank decay, changes in skill of playerbase composition. That's why you meet people of different leagues and ranks in SC2. It's really naive to believe that Heroes will successfully have each of their 2% brackets perfectly represent the distribution of playerbase's skill in Heroes. As long as some people don't play until they are accurately placed, play deliberately worse, stop playing, only play once in a blue moon, etc. there will be inconsistencies established in Heroes. The same things that you are attacking the SC2 system for.
You are confused between the meaning of precision and accuracy.
"It seems that you've made an assumption that Heroes ranking system is always accurate when it places you, and that SC2 is always inaccurate, because of leagues." I have not made such an assumption.
You're argument that factors X, Y and Z affect ranks, so they are wrong. This is basically the fallacy that if we don't know everything so we know nothing.
I'm some factors distort ranks (e.g. skill decay), some factors, contrary to your claims, don't distort ranks (e.g. changes in playerbase). I'm saying that there is more factors that there are far more factors that distort ranks in SC2 that do exist in HotS: e.g. bonus pool, leagues. And Dustin Browder agrees, that's why his removing them.
When I am looking at the information that the SC2 system gives me, games played definitely gives me a relevant indicator of how accurate the given player's position is. If you think that a systems ability to accurately place a player after 5 games is the same as it is after 500 games, then obviously there is no reason of discussing anything. Obviously previous seasons matter, but other things being equal, many games recently tend to give a better indication of recent skill.
Do you understand that the system gives you more information than that a player is in gold? That there is a point count and a rank within each league? You evaluate the system as if the only feedback the system gave you was in which league you're in. A league can't really be compared directly to Heroes' system. So the 16 times more accurate statement is not true at all.
All I'm trying to argue is that the more rigid system makes the ranking evaluation more informative. You move relatively faster between ranks in Heroes so every rank conveys less information. There are a lot of factors that distorts both systems, maybe there are more systemic distortions in SC2, but as long as those are revealed and well understood, it doesn't make SC2 ranking that much worse all.
In doesn't really make a difference, as long as the system used matches you up with people who are the same MMR. That wasn't my experience in Hero League when I played (When you would move around being awarded between 100 and 200 points out of 300 per match). Obviously, as the ranks become more rigid and advancement slower, ranks in Heroes will become more informative. It just seems like your critique is very severe as a result of a really simplified understanding of the SC2 system. It's not that bad as you would have it.
No. Games played does not given an indication of how accurate your rank is and I have given 4 examples above, where games played is not informative.
Your MMR after 500 games is more accurate than after 5 games. But your rank after 500 games is NOT necessarily more accurate than after 5 games because of all the unique SC2 distortionary factors that creates bias, not just standard error, as I've shown in the examples above. So your understanding of ranking systems is very uninformed.
Your statement that HotS ranks contain less information is flat out false. What's more information: You're MMR percentile plus a bunch of biased distortionary factors is in the top 40% to 72% or your MMR percentile in is the top 40% to 42%? The latter, is far more information and far more accurate, which is why the HotS ranking system is being changed in this way.
you ignore a very simple fact. pretty much everyone in heroes jump up and down in ranks pretty wildly. I myself vary between rank 27 and rank 22 depending on how well im playing that particular day and also of course, varing based on luck (aka unpredictable factors favouring one side over the other). so for me, if I am in rank 25 it is misleading to claim that means I am in percentages 50% to 52% when in reality I vary betweeen percentages 44% to 54%
and besides, it not like SC2 gives no indication of where in your division you are, true its built in leagues which give a wide range, but your placement within the division tells you whether you are closer to 40th percentile rather than 70th percentile.
to any intelligent person that is interested to know one own approximate standing, SC2 is no better or worse than the suggested model for heroes.
Firstly, 44% to 54% is not 40% to 72%, which is what you'll get in SC2.
Secondly, if someone is 44% to 54%, and another person is 48% to 53%, who should be ranked higher? For any ranking system, even the SC2 ranking system, you need to collapse this information into 1 number, such as a mean, 49% vs 50.5%, so the second should be higher. The problem is that the 1 number in SC2 is wrong, and in HotS it's far less wrong.
Thirdly, why does your rank being between 44% to 54% imply it should be biased with distortionary factors like bonus pool and no demotions, and made up to 16 times (a number not chosen based on the data but pulled out of Blizzard's ass) less precise than HotS ranks?
Fourthly, division ranks are meaningless and wrong. They intrinsically decline every hour due to the bonus pool, can't be accurately compared when bonus pool is not spent, and doesn't account for the fact that some people don't deserve to be the division but are stuck there due to rigidities in the league system.
Lastly, if you want less volatility in ranks, the solution is not the SC2 ranking system of wrong ranks, it's to apply a trend filter to the metric that is used to create the percentile ranks (approximate MMR in HotS or points in SC2).
firstly, the size of a league in SC2 is 32% only for gold league, the rest are 20% or less. you are literally quoting the absolute worst case scenario and I think I am completely justfied in accusing you of cherry-picking.
secondly, mmr does not need to be collapsed into 1 number, mmr is collapsed into 2 number, the computer considers a 44-54 person as a 49 +- 5, it considers a 48-53 person as a 50.5 +- 2.5
thirdly, your third point doesn't even make sense. you are right the rank being between 44 and 54 does not imply the things you say, but thats because my rank being between 44 and 54 doesn't imply anything at all about the system at large, this point is moot. the fact that my skill is a range rather than a number does however imply that any attempt of putting a specific number on it, or for that matter putting a smaller-than-actual range on it (such as a 2% range when the real range is 10%), will be deceptive in its granularity.
fourthly, division ranks are meaningful in comparison to active players, inactive players decaying out of the system is to get rid of unwanted statistical noise for the active players.
fifthly, applying a trend estimation would let you see how you have been doing so far and let you see where you will probably be in the future, but it wouldn't be any more accurate than what we currently have without extremely unnecessary large amounts of processing. for example, after each game the current system uses perhaps 10-20 operations to record the result and adjust your mmr, is it really worth it to improve the systems accuracy by a tiny amount if doing so will require 1 000 000 - 2 000 000 operations after each game?
lastly, you do know that heroes doesn't actually use your rank to match you with people right? and it still wont after this change. it has a hidden mmr just the same as in SC2 which is used to match you, have you ever gotten a "skill bonus" in heroes? thats a bonus you get when your hidden mmr has raced far ahead of your displayed rank and the system tries to let your rank catch up. for example, a rank 45 player that has just started climbing in hero league might have a hidden mmr in the 20ies and when that player wins it gets a skill bonus to accelerate the climb up to 20ish, but when it reaches that rank it stops getting the skill bonus, and wont get skill bonus again unless it somehow becomes extremely much better very quickly. the matchmaking is completely unaffected by what rank is being displayed. the rank is literally just there to be an aestethically pleasing indication of your rough position, there is really no need to tell players exactly where in the list of millions of players they are, doing so would just give the deceptive illusion of accuracy.
I always liked sc2 ranking system. It's very rewarding to reach one of the top places in your division and once bonus pool is spent + enough games are played the ladder rank is also a quite accurate indicator for the mmr. Not 100% accurate of course but I don't think that's necessary.
On August 08 2015 23:25 TimeSpiral wrote: I've alway thought matchmaking in SC2 was pretty good--significantly better than many other games, in other genres.
They're improving ranking accuracy in Heres because matchmaking is horribly, horribly bad, atm. But, it's also a wildly different system. Not even apples to apples. More like, apples to hockey, lol.
Improving the SC2 matchmaking should--imho--have only one goal: increase enjoyment for rec-level players. It is not fun to lose a match because you never had a chance. If possible, the vast majority of games should be up against opponents of similar skill. I believe the SC2 matchmaking already does this fairly well, but could probably be improved.
So the SC2 league system appears to be clunky, and perhaps less accurate because there are a small number of divisions. These are just shiny objects to get you to play the game more, lol. I strongly suspect your league badge has very little to do with the matches you play. But *shrugs* that's just me.
The other issue is the number of players and the quality of time playing. Is it better to have longer waits and closer matches, or shorter waits and total stomp-fests? This is a user experience issue that must be addressed, and it can be a downward spiral. Less people playing means the rec-level players will have a harder time having fun with the game.
At the rec-level, it needs to be fun first. If you don't think you're recreational, then none of this matters. You're super tryhard and dedicated, and your MMR should do a decent job at keeping your W/L at 50%. But what the hell do I know?
No one is complaining about the matchmaking. A good matchmaker is only possible with a good estimate of skill (MMR). Therefore, the existence of good matchmaking implies the existence of a good estimate of skill. The problem is that Blizzard is not using this good estimate of skill for ranking. Instead, they deliberately distort ranks, deliberately making them inaccurate, when they could just use that good estimate of skill for ranking as Dustin Browder now wants to do in HotS (and that's the right decision).
The Browder conversation yesterday at GamesCom--with the lovely and talented Soe and super-fun Frodan--was specifically rooted in improving matchmaking. Again, I don't even think the ranking system in Heroes is relatable to SC2.
The leagues are for fun. For a sense of improvement. They're achievement badges. Climb through the ranks of your division, and surpass the names you'll never actually face in a match ever! *grasping heroically into the air* It's motivation. It's easy to understand for the rec players. I think it could be improved, sure, but I think it's one of the best in the business, as far as quality matches are concerned.
Your accusations seem unfounded, hyperbolic, and unfair. Are you getting good matches?
God yes I want MMR to be shown for sc2, get rid of the divisions and just have the 7 leagues and then your mmr. So if you're high masters you check your rank and you see masters 100 or masters 68 and so on. Its so fucking annoying not seeing mmr.
EPT
