|
On October 05 2011 09:13 Excalibur_Z wrote:Show nested quote +On October 05 2011 09:01 c0ldfusion wrote:On October 05 2011 08:51 crms wrote:On October 05 2011 08:03 Excalibur_Z wrote:On October 05 2011 07:37 c0ldfusion wrote:On October 05 2011 07:27 Excalibur_Z wrote:On October 05 2011 07:08 c0ldfusion wrote:On October 05 2011 06:16 Excalibur_Z wrote:On October 05 2011 06:09 Ignorant prodigy wrote:I think it may be a good thing to have shorter seasons.. this is how I envision it working.. Since we sort of agree MRR is a moving target, I think the qty of people actively playing the game effects which ranking certain MMR ranges fall into. This would mean the shorter season weeds out the newer players quicker.. making the variance in skill between the rankings a bit less diluted. For instance if Blizzard is trying to maintain certain percentages of its overall populace into different leagues then shorter seasons reduce the qty of inactive players accounting for those percentages. Meaning your ranking will be more accurate to your skill level amongst active players. I created a quick chart to sort of show what I’m saying. Let’s say right now there’s 50,000 people who’ve played their placement matches in S3. Lets also guesstimate Blizzard wishes to maintain 23% of that 50,000 be dedicated to Bronze (12,750 Players in bronze) If S4 comes along and only 30,000 players have played their placement matches, that 23% shows a lower number of people (7,650 in bronze) Meaning the MMR would slide accordingly. My example shows that in S3 a player with 1780 MMR is Platinum, whereas in S4 that MMR has this player in Diamond. Obviously if S4 grows to 50,000 people and the player in the example below never played a game, then he would drop back down in Platinum. ![[image loading]](http://img855.imageshack.us/img855/7968/mmre.jpg) Of course.. this is all guessing on my part.. I just sort of envision it working this way. Nah the MMR boundaries for the leagues are fixed. It's the population that varies, not the league boundaries. MMR isn't a moving target. It's more like this: ![[image loading]](http://imgur.com/5w92j.jpg) Whoa, wait a minute, how would Blizzard maintain the 20/20/20/20/18/2 ratio if MMR boundaries are fixed? The populations aren't enforced. They fluctuate, and sometimes not insignificantly. However, skill always spreads out to cover gaps. I've used this example before, but say there are three players A B C and D, at 1000 1200 1400 and 1600 MMR, respectively. A is 1000 because he loses to the rest. B is at 1200 because he loses to C and D but he still beats A. C is at 1400 because he beats both A and B but loses to D. D is at 1600 because he beats all of them. If C stopped playing: - B would rise because he isn't losing to C anymore. - A would rise slightly because the gap between A and B will widen. A is still beating people below him. - D would fall slightly because as he loses to people above him, he can't sustain his current level because the gap between B and D is wider than the old gap between C and D. The new distribution might be something closer to 1020 for A, 1350 for B, 1580 for D. If C came back, he would come back at 1400, his old MMR. Gradually, as all players play more games, skill will spread out again and go back to the old distribution. Now if you imagine this on a larger scale and with league boundaries, that's going to come with some league fluctuations. Ok gotcha, so the percentages are already "baked in" the MMR. If that's the case though, then aren't they making an assumption about the distribution? Your graph looks like a normal distribution... is that accurate? Intuitively, I would think something like skill in starcraft 2 would fit better in a distribution with a fat tail. I'm a little sketchy on what it is in reality (obviously since the best resource we have is SC2Ranks which is limited because it includes all accounts) but from what I've heard, there is a bump in Diamond because of the skill difference between "hardcore" and "casual" players. However, I don't know if that was only for Season 1 and if prior distributions were used for later seasons. Can you elaborate on this 'bump'? Are you saying there is something artificially placed in Diamond that you must 'get over' regardless or active MMR to differentiate 'hardcore' and 'casual'. Is getting Diamond->Master the hardest promotion? Aside from GM I suppose. Reading what I typed sounds retarded, ugh. Can you just explain what this diamond 'bump' means? I think he meant between diamond and platinum - this would explain be explained them trying to fit a non-symmetric distribution onto a symmetric one. Though keep in mind that Blizzard can make changes to MMR related calculations (points lost and won from matches, boundaries, etc) whenever they want. They don't need a patch for something like that. Chances are they probably have taken some measures to correct for this phenomenon considering they always stood by the 20/20/20/20/18/2 breakdown. Edit: Correction, I don't mean the boundary between plat and diamond. I mean more than the allocated 20% of active population in diamond. Yes, that's right. Blizzard has modified where the boundaries are in the past. Famously there was a "promotion day" that many players noticed in their 2v2/3v3/4v4 arranged teams and in 1v1 Bronze/Silver in the middle of Season 2. This was a result of the boundaries being changed to suit the current active population and return the league distribution closer to the ideal. For example, say that when Master was first introduced, the MMR boundary was set at like 2600 (another imaginary value). For 1v1 that would mean 2% of the population would be expected to get above 2600. For 4v4, the reality was that there was only 1 arranged team that got into Master league at first, and their record was something amazing like 92-0. That meant that given the skill spread in 4v4 arranged, reaching 2600 was exceedingly difficult and didn't properly represent the desired 2% of the population. So, they changed the boundary, maybe to something like 2100 for 4v4 arranged so that closer to 2% of the population could get in.
Ahh I remember that, makes sense.
So yes -moral of the story- to everyone asking questions about things like bonus pool, league distribution (myself included), we just have to trust that Blizzard is making sure that the ladder population falls roughly into the 20/20/20/20/18/2 breakdown and that more importantly, the only things that tell you anything meaningful about current your skill level are your league, the leagues of your recent opponents (and maybe also the leagues of the other recent opponents of your recent opponents). Those things would probably give you as clear a picture as possible of where you stand relative to the entire population of active players.
|
i think this is a good change. it makes it easier for people that are getting into the game to place well in the ladder, and for the veterans, the mmr really remains unchanged so there isn't much of a difference. Its ultimately not about how much you play, but where your mmr stands that determines what division you're placed in.
|
Hey, just a quick question, and I'm sorry if it has been asked and answered before, is the MMR still active when you play during the lock-down period? I have some leave coming up and was going to log in a few hours of SC2 for my 1v1 and in my 2v2 and 3v3 teams. Will these games be taken into account when season 4 commences after 24/10? Cheers.
|
On October 05 2011 10:00 aZealot wrote:
Hey, just a quick question, and I'm sorry if it has been asked and answered before, is the MMR still active when you play during the lock-down period? I have some leave coming up and was going to log in a few hours of SC2 for my 1v1 and in my 2v2 and 3v3 teams. Will these games be taken into account when season 4 commences after 24/10? Cheers.
Yes, your MMR changes as it would normally during league lock.
|
|
|
On October 05 2011 07:27 Excalibur_Z wrote:Show nested quote +On October 05 2011 07:08 c0ldfusion wrote:On October 05 2011 06:16 Excalibur_Z wrote:On October 05 2011 06:09 Ignorant prodigy wrote:I think it may be a good thing to have shorter seasons.. this is how I envision it working.. Since we sort of agree MRR is a moving target, I think the qty of people actively playing the game effects which ranking certain MMR ranges fall into. This would mean the shorter season weeds out the newer players quicker.. making the variance in skill between the rankings a bit less diluted. For instance if Blizzard is trying to maintain certain percentages of its overall populace into different leagues then shorter seasons reduce the qty of inactive players accounting for those percentages. Meaning your ranking will be more accurate to your skill level amongst active players. I created a quick chart to sort of show what I’m saying. Let’s say right now there’s 50,000 people who’ve played their placement matches in S3. Lets also guesstimate Blizzard wishes to maintain 23% of that 50,000 be dedicated to Bronze (12,750 Players in bronze) If S4 comes along and only 30,000 players have played their placement matches, that 23% shows a lower number of people (7,650 in bronze) Meaning the MMR would slide accordingly. My example shows that in S3 a player with 1780 MMR is Platinum, whereas in S4 that MMR has this player in Diamond. Obviously if S4 grows to 50,000 people and the player in the example below never played a game, then he would drop back down in Platinum. Of course.. this is all guessing on my part.. I just sort of envision it working this way. Nah the MMR boundaries for the leagues are fixed. It's the population that varies, not the league boundaries. MMR isn't a moving target. It's more like this: Whoa, wait a minute, how would Blizzard maintain the 20/20/20/20/18/2 ratio if MMR boundaries are fixed? The populations aren't enforced. They fluctuate, and sometimes not insignificantly. However, skill always spreads out to cover gaps. I've used this example before, but say there are three players A B C and D, at 1000 1200 1400 and 1600 MMR, respectively. A is 1000 because he loses to the rest. B is at 1200 because he loses to C and D but he still beats A. C is at 1400 because he beats both A and B but loses to D. D is at 1600 because he beats all of them. If C stopped playing: - B would rise because he isn't losing to C anymore. - A would rise slightly because the gap between A and B will widen. A is still beating people below him. - D would fall slightly because as he loses to people above him, he can't sustain his current level because the gap between B and D is wider than the old gap between C and D. The new distribution might be something closer to 1020 for A, 1350 for B, 1580 for D. If C came back, he would come back at 1400, his old MMR. Gradually, as all players play more games, skill will spread out again and go back to the old distribution. Now if you imagine this on a larger scale and with league boundaries, that's going to come with some league fluctuations.
I don't think this is true. The matchmaking system does more than determine the players skill order. It measures their actual skill difference, ie the MMR difference between two players is a direct measure of the chances of these two players beating each other. In your example above, B would still lose to D the same way he did before. Imagine that only B and D are left on the ladder. Would their MMR difference widen until they end up in different leagues? I say no. B would still beat D some times (for example 1 out of 5), and when he does he will win a large amount of points. This will keep their MMR difference the same.
|
I don't think this is true. The matchmaking system does more than determine the players skill order. It measures their actual skill difference, ie the MMR difference between two players is a direct measure of the chances of these two players beating each other.
It's not a direct measure, it's a Bayesian ("maximum likelihood") estimate of their chance of one player beating the other based on past game history against ALL other players (not just the one in question.) Their game history determines not just their ordering but how far apart the game thinks they are from each other, but if a player drops out they do redistribute because the people that player would have beat are losing slightly less and the players to which that player would lose are winning less.
What Excalibur_Z is describing is consistent with that kind of system.
|
On October 05 2011 15:22 Lysenko wrote:Show nested quote +I don't think this is true. The matchmaking system does more than determine the players skill order. It measures their actual skill difference, ie the MMR difference between two players is a direct measure of the chances of these two players beating each other. It's not a direct measure, it's a Bayesian ("maximum likelihood") estimate of their chance of one player beating the other based on past game history against ALL other players (not just the one in question.) Their game history determines not just their ordering but how far apart the game thinks they are from each other, but if a player drops out they do redistribute because the people that player would have beat are losing slightly less and the players to which that player would lose are winning less. What Excalibur_Z is describing is consistent with that kind of system.
You know more about this than me, so I can't really argue with you, but if what you are saying is true, the MMR difference between two players does NOT indicate their relative skill difference, ie you cannot calculate the chance of player A beating player B based on their MMR? That does not make sense to me. What about my example with player B and D being the only players on the ladder?
|
ie you cannot calculate the chance of player A beating player B based on their MMR? That does not make sense to me. What about my example with player B and D being the only players on the ladder?
The variation because a player drops out would be smaller than the uncertainty of the estimate of the MMR. The idea is that there's an "actual" MMR and the one in the system is just an approximation of that. This kind of effect would push around people's scores within the range of the uncertainty of the MMR, which a system like this can probably quantitatively estimate.
|
On October 05 2011 17:00 Lysenko wrote:Show nested quote +ie you cannot calculate the chance of player A beating player B based on their MMR? That does not make sense to me. What about my example with player B and D being the only players on the ladder? The variation because a player drops out would be smaller than the uncertainty of the estimate of the MMR. The idea is that there's an "actual" MMR and the one in the system is just an approximation of that. This kind of effect would push around people's scores within the range of the uncertainty of the MMR, which a system like this can probably quantitatively estimate.
Ah. That makes more sense then. If a lot of bronze players drop out, there will be a downward shift, but not enough to completely make up for all of the missing players?
|
Ah. That makes more sense then. If a lot of bronze players drop out, there will be a downward shift, but not enough to completely make up for all of the missing players?
In that case, it's a little different. If there's a wholesale exodus of Bronze players, everyone else is going to move down the ranks.
Over the long term, an active player's MMR should be correlated pretty well to their percentile. That's why the leagues up to Diamond are each "20%" of the population -- they have firm MMR boundaries, and as people exit or enter the game, the population redistributes among available MMRs.
So, if the bottom 20% quit, then the bottom 20% of the higher level players who remain become the new bottom 20%, and probably get demoted.
ELO works the same way, btw.
|
On October 05 2011 18:02 Lysenko wrote:Show nested quote +Ah. That makes more sense then. If a lot of bronze players drop out, there will be a downward shift, but not enough to completely make up for all of the missing players? So, if the bottom 20% quit, then the bottom 20% of the higher level players who remain become the new bottom 20%, and probably get demoted.
Yes, this is the example often quoted, and it's the one I'm having problems with. Obviously, if all of the bottom 20% is quitting, the next 20% will take their place, because there are no true anchors in the system except for the very best and worst players. But this scenario is unrealistic. What happens is that maybe every other of them quit. In a system that measures your true skill every MMR would then be unaffected, because the anchors would still be approximately of the same skill level as before. Due to the probabilistic nature of measured MMR I can understand that there will be a shift, but not enough to completely fill up bronze again. That would imply that the skill distance between the players is completely ignored by the system.
(Edit: spelling)
|
That would imply that the skill distance between the players is completely ignored by the system.
Based on all the information I have, MMR can't be used to derive an absolute probability of win or loss, other than that two players of the same MMR have a 50/50 chance to win or lose. Thus, when a situation occurs like I described, the relationship between different MMRs and the chances of a win or loss would change.
However, I'm not sure that enough information has been released to know for certain that that's the case. Excalibur_Z seems to think so, though, and he does have an inside track.
|
On October 05 2011 18:43 Lysenko wrote:Show nested quote +That would imply that the skill distance between the players is completely ignored by the system. Based on all the information I have, MMR can't be used to derive an absolute probability of win or loss, other than that two players of the same MMR have a 50/50 chance to win or lose. Thus, when a situation occurs like I described, the relationship between different MMRs and the chances of a win or loss would change. However, I'm not sure that enough information has been released to know for certain that that's the case. Excalibur_Z seems to think so, though, and he does have an inside track.
That surprises me a lot. A quick google about Elo for example says this:
http://www.ascotti.org/programming/chess/elo.htm
|
On October 05 2011 18:21 Mendelfist wrote:Show nested quote +On October 05 2011 18:02 Lysenko wrote:Ah. That makes more sense then. If a lot of bronze players drop out, there will be a downward shift, but not enough to completely make up for all of the missing players? So, if the bottom 20% quit, then the bottom 20% of the higher level players who remain become the new bottom 20%, and probably get demoted. Yes, this is the example often quoted, and it's the one I'm having problems with. Obviously, if all of the bottom 20% is quitting, the next 20% will take their place, because there are no true anchors in the system except for the very best and worst players. But this scenario is unrealistic. What happens is that maybe every other of them quit. In a system that measures your true skill every MMR would then be unaffected, because the anchors would still be approximately of the same skill level as before. Due to the probabilistic nature of measured MMR I can understand that there will be a shift, but not enough to completely fill up bronze again. That would imply that the skill distance between the players is completely ignored by the system. (Edit: spelling)
That's exactly what I was trying to say.. but who knows then the 'shifting' occurs. Doesn't sound like it happens often... but with shorter seasons I would think it will have to happen at some point as there's less and less 'new' players with each new season.
|
On October 05 2011 23:44 Ignorant prodigy wrote:Show nested quote +On October 05 2011 18:21 Mendelfist wrote:On October 05 2011 18:02 Lysenko wrote:Ah. That makes more sense then. If a lot of bronze players drop out, there will be a downward shift, but not enough to completely make up for all of the missing players? So, if the bottom 20% quit, then the bottom 20% of the higher level players who remain become the new bottom 20%, and probably get demoted. Yes, this is the example often quoted, and it's the one I'm having problems with. Obviously, if all of the bottom 20% is quitting, the next 20% will take their place, because there are no true anchors in the system except for the very best and worst players. But this scenario is unrealistic. What happens is that maybe every other of them quit. In a system that measures your true skill every MMR would then be unaffected, because the anchors would still be approximately of the same skill level as before. Due to the probabilistic nature of measured MMR I can understand that there will be a shift, but not enough to completely fill up bronze again. That would imply that the skill distance between the players is completely ignored by the system. (Edit: spelling) That's exactly what I was trying to say.. but who knows then the 'shifting' occurs. Doesn't sound like it happens often... but with shorter seasons I would think it will have to happen at some point as there's less and less 'new' players with each new season.
This is pretty much what Excalibur_Z and I were discussing - that Blizzard tweaks calculations/boundaries/etc regarding player MMRs. The main point takeaway though is that these changes can occur at any time. They should be completely independent of the length of the seasons.
The main benefits of shorter seasons, I think, are to give people a chance to catch up within their division so they can feel better psychologically about their progression and for the GM league to cycle out more frequently.
It has the added benefit of making it easier for you to reset your MMR though (since skipping a season intentionally requires a shorter time-frame).
|
United States12238 Posts
On October 05 2011 18:55 Mendelfist wrote:Show nested quote +On October 05 2011 18:43 Lysenko wrote:That would imply that the skill distance between the players is completely ignored by the system. Based on all the information I have, MMR can't be used to derive an absolute probability of win or loss, other than that two players of the same MMR have a 50/50 chance to win or lose. Thus, when a situation occurs like I described, the relationship between different MMRs and the chances of a win or loss would change. However, I'm not sure that enough information has been released to know for certain that that's the case. Excalibur_Z seems to think so, though, and he does have an inside track. That surprises me a lot. A quick google about Elo for example says this: http://www.ascotti.org/programming/chess/elo.htm
No, you're actually exactly right. I decided to ask about this and my example was wrong. The difference in MMR does directly correspond with win chances, just as it does under Elo. This was the response I got, and I'll be correcting any errors in the Ladder Guide:
So, in your example, if C left but A, B, and D already have established ratings, those ratings wouldn't change in either Elo or our system because now D would just beat B 76% of the time, and that difference would be preserved ( in the long run). Their absolute win percentages might change, though that depends a bit on your match-maker. The difference in MMR will always be directly proportional to the estimated win % between those two players. If you and I were the only two players in the system and the system started at, e.g. 1500, and we only played each other, and you beat me 64% of the time, then our ratings would converge to 1450 and 1550 (assuming 100 = 64%). The only real difference as far as PREDICTIONS go in our system is we take the variance into account. All this really does, though, is scale that difference down a little before using it. So, as a loose example, if the difference between us is 100, but we're both pretty new, then that 100 might become 50, since our confidence that you are REALLY 100 better than me is low. And, yes, if we had an established system and then all Bronze left, it would be 0%. We have ways of dealing with that though, if it's necessary 
So Bronze actually wouldn't get repopulated in the hypothetical "what if Bronze quit" scenario, Blizzard would just react to that and adjust MMR boundaries accordingly.
|
On October 06 2011 09:02 Excalibur_Z wrote:Show nested quote +And, yes, if we had an established system and then all Bronze left, it would be 0%. We have ways of dealing with that though, if it's necessary So Bronze actually wouldn't get repopulated in the hypothetical "what if Bronze quit" scenario, Blizzard would just react to that and adjust MMR boundaries accordingly.
Thanks. The only question left for me then is what happens in the unlikely scenario that every single one of the bronzes leave. I understand that if a few of them remain they will force the rest of the population to keep their MMR, but I don't agree that nothing will happen if they all leave.
The question is what Blizzard uses for "anchors". There has to be something, otherwise MMR will tend to drift. In other online systems, Go for example, you use well known players as anchors. "X is 9 dan. Period." In this system, what do they use? My idea was that they simply assign the best player to some MMR, and the worst player to some other MMR. In that case bronze will repopulate if empty, but according to your response that won't happen. So how do they do it?
Actually, I was just going to hit "Post", when the thought occured to me that you can't use two anchors. That would assign a fixed MMR difference between the best and the worst regardless of their skill difference, and that doesn't work (because "MMR will always be directly proportional to the estimated win %" wouldn't be true). Maybe they only use the best players? But that would cause some problems in the long run, because the skill difference between the best and the worst will constantly increase, and bronze would fill up more and more. Whatever anchor you choose, you would see a redistribution over time as the best continue to be better, but the newcomers still are newbies, and we don't see this.
|
Way to soon. Needs to be more then 2 months. Maybe 3 minimum.
|
I'm not really a big fan of the shorter seasons, but hey, doesn't matter that much to me I'll still play the same amount.
|
|
|
|
|
|
EPT
