|
United States12249 Posts
On May 22 2012 05:58 skeldark wrote:Show nested quote +On May 22 2012 05:20 Excalibur_Z wrote:
I don't see any Platinum games in the example you provided, but according to the skill distribution curve it makes sense that most players' uncertainty ranges would span the equivalent of a part of Silver, all of Gold, and a part of Platinum. At the time of our Part 2 analysis post, we believed that the promotion criterion was 3*sigma (meaning if 99% of your curve fits within a league boundary, you'll get promoted), but at Blizzcon the systems designer said that wouldn't be a useful measure for league changes because most players' curves will never shrink that much, so they use an exponential moving average instead. And, now that matchmaking has been loosened a bit for the majority of players, such ranges are undoubtedly even more common today.
The "overlap" of Gold and Silver also isn't very surprising, given that we know there's a confidence buffer that works in conjunction with the exponential moving average requirement. Perhaps as a result of determining offsets we'll also find the confidence threshold. I suppose the moving average equation will be the final piece of the puzzle =) We are not talking of a little overlap. The 73 i take as overlap so far is to small. i think best silver is near EQUAL worst gold! This means leagues are a big scam! Also i loose the faith that offsets are just numbers you have to add. With this assumption you run into logic mistakes. Perhaps i do some mistakes and i have to work with inaccuracy, but i think by now, offsets follow an function that is not as simple as addition!
Wait, you're surprised that the best Silver is near equal to the worst Gold? But that's entirely how the ladder is structured.
To use arbitrary values, if the top Silver division spans MMR ranges 900-1000 and the bottom Gold division spans 1001-1100, then of course the best true Silver would be 1000 and the worst true Gold would be 1001. Then you factor in some flux where the slope of a player's exponential moving average is not close enough to 0 to warrant a league change (for example, a Silver with 1010 who hasn't been promoted yet).
The point reset after a league change doesn't constitute an MMR reset. You still have the same MMR, you just have a new reset point value to use as a baseline.
I don't think the offsets themselves follow any kind of complex function. However, I don't think the numbers "63" and "150" were necessarily arbitrarily chosen either. Let's follow this scenario: it's possible they correspond to points along the skill curve. For example, if the entire MMR scale is 2800, and it was determined that between the 80th and 98th percentiles covered 30% of the x-axis, then 0.30*2800=840. If we assume that division tiers each cover the same skill range (x-axis) then we'd divide that 840 evenly into however many tiers are needed, so let's say 10. We add a confidence buffer of 100, then (840-100)/10 = 74 point offsets per tier.
It could be that Master is the anomaly and that lower leagues may have different buffer sizes. If the tier offsets are indeed derived from a percentage of the MMR scale, then the offsets may not be uniform from league to league, either.
|
On May 22 2012 07:58 SDream wrote:Show nested quote +On May 22 2012 07:41 Not_That wrote:
Is this data point a counter example to what you're saying SDream?
EU, player1, Silver, 40 adjusted points before match, lost 22 points, calculated player2 MMR -382.5 +- 48
player2, Silver, 173 adjusted points before match, won 11, calculated player1 MMR 143.5 +- 14
Maybe come to teamspeak so we could have a more back-and-forth about it? What you're saying is news to us.
edit: even just come to have insta-typed chat capabilities. Forum posts take a while to formulate and we can understand each other quicker live. EU, player1, Silver, 40 adjusted points before match, lost 22 points, calculated player2 MMR -382.5 +- 48 Yes, that's probably wrong data, he couldn't lose more than 11 points! He probably was promoted/demoted somewhere and you made the math wrong because you didn't take this into account, or something! :D I will think about teamspeak later =3
your right. I checked this packet. Its something wrong with the match history of this guy, the webpage dont show a result he just say:
"left" instead of the correct "Loose -12" and the league page did not update so i put the -12 on last game that was only -10 and come up with the wrong -22.
i will fix that.
|
You're right it's bad data point. The guy has been Silver since March 2011 so it's not promotion, strange. We'll look into it.
It's great that you bring this up SDream, we weren't aware of it. Thank you.
|
Excalibur, I'm somewhat surprised it's possible 100 adj points in high Silver is almost identical to ~70 adj points in gold. BTW you mentioned not seeing Plat opponent in the graph, click it to see the whole 14 games.
|
United States12249 Posts
On May 22 2012 08:56 Not_That wrote:
Excalibur, I'm somewhat surprised it's possible 100 adj points in high Silver is almost identical to ~70 adj points in gold. BTW you mentioned not seeing Plat opponent in the graph, click it to see the whole 14 games.
Oh right you are, the thumbnail truncated it.
|
On May 22 2012 08:16 Excalibur_Z wrote:Show nested quote +On May 22 2012 05:58 skeldark wrote:On May 22 2012 05:20 Excalibur_Z wrote:
I don't see any Platinum games in the example you provided, but according to the skill distribution curve it makes sense that most players' uncertainty ranges would span the equivalent of a part of Silver, all of Gold, and a part of Platinum. At the time of our Part 2 analysis post, we believed that the promotion criterion was 3*sigma (meaning if 99% of your curve fits within a league boundary, you'll get promoted), but at Blizzcon the systems designer said that wouldn't be a useful measure for league changes because most players' curves will never shrink that much, so they use an exponential moving average instead. And, now that matchmaking has been loosened a bit for the majority of players, such ranges are undoubtedly even more common today.
The "overlap" of Gold and Silver also isn't very surprising, given that we know there's a confidence buffer that works in conjunction with the exponential moving average requirement. Perhaps as a result of determining offsets we'll also find the confidence threshold. I suppose the moving average equation will be the final piece of the puzzle =) We are not talking of a little overlap. The 73 i take as overlap so far is to small. i think best silver is near EQUAL worst gold! This means leagues are a big scam! Also i loose the faith that offsets are just numbers you have to add. With this assumption you run into logic mistakes. Perhaps i do some mistakes and i have to work with inaccuracy, but i think by now, offsets follow an function that is not as simple as addition! Wait, you're surprised that the best Silver is near equal to the worst Gold? But that's entirely how the ladder is structured. To use arbitrary values, if the top Silver division spans MMR ranges 900-1000 and the bottom Gold division spans 1001-1100, then of course the best true Silver would be 1000 and the worst true Gold would be 1001. Then you factor in some flux where the slope of a player's exponential moving average is not close enough to 0 to warrant a league change (for example, a Silver with 1010 who hasn't been promoted yet). The point reset after a league change doesn't constitute an MMR reset. You still have the same MMR, you just have a new reset point value to use as a baseline. I don't think the offsets themselves follow any kind of complex function. However, I don't think the numbers "63" and "150" were necessarily arbitrarily chosen either. Let's follow this scenario: it's possible they correspond to points along the skill curve. For example, if the entire MMR scale is 2800, and it was determined that between the 80th and 98th percentiles covered 30% of the x-axis, then 0.30*2800=840. If we assume that division tiers each cover the same skill range (x-axis) then we'd divide that 840 evenly into however many tiers are needed, so let's say 10. We add a confidence buffer of 100, then (840-100)/10 = 74 point offsets per tier. It could be that Master is the anomaly and that lower leagues may have different buffer sizes. If the tier offsets are indeed derived from a percentage of the MMR scale, then the offsets may not be uniform from league to league, either.
Agree! Had the same idea, i even coded exactly what you said already.
But the data was against it.
BUT: the data that was against it, was also against sdreams rule!
If i take player out who have under 73 mmr it can be just addition.
I know for sure now that the buffer is lower for lower leagues. Smaller than 50 for most.
|
On May 22 2012 08:56 Not_That wrote:
Excalibur, I'm somewhat surprised it's possible 100 adj points in high Silver is almost identical to ~70 adj points in gold. BTW you mentioned not seeing Plat opponent in the graph, click it to see the whole 14 games.
I wasn't understanding it either!
And I am glad I could help here, I brought bad news, but it's the truth at least. Reliable data > illusions :D
|
United States12249 Posts
On May 22 2012 08:21 Not_That wrote:
You're right it's bad data point. The guy has been Silver since March 2011 so it's not promotion, strange. We'll look into it.
It's great that you bring this up SDream, we weren't aware of it. Thank you.
This was something that was observed in the past but was difficult to explain. I suppose it still is. If you look at a player who tanks his games in order to fall down to Bronze, his match history will look like this:
+ Show Spoiler +
(starting league Diamond)
-12
-12
-11
-11
-10
-10
-9
-9
-9
-8
-8
-8
-11 (league change to Platinum)
-11
-11
-10
-10
-9
-9
-11 (league change to Gold)
-11
-11
-10
-10
-9
-9
-11 (league change to Silver)
-11
-11
-10
-10
-9
-9 (league change to Bronze)
etc.
It's counter-intuitive behavior given that MMR is supposed to travel faster than points, and if you're losing 100% of your games, your MMR should be moving very quickly indeed. The expected result is that your points lost should increase with each consecutive loss because that would move your points more rapidly toward your MMR. However, that's not what happens in reality. Consistently, point changes would start at -11 and gradually fall off until -8 or until they lost all their points, and this would reset after a league change. One possible explanation is that there is a phantom point value that exists which influences your point change in addition to the staple "adjusted points versus opponent MMR" formula, as SDream suggests. If true, then the higher your points drift away from this phantom value, the more you are capable of losing. Or, there could be something else at work.
|
On May 22 2012 09:36 Excalibur_Z wrote:Show nested quote +On May 22 2012 08:21 Not_That wrote:
You're right it's bad data point. The guy has been Silver since March 2011 so it's not promotion, strange. We'll look into it.
It's great that you bring this up SDream, we weren't aware of it. Thank you. This was something that was observed in the past but was difficult to explain. I suppose it still is. If you look at a player who tanks his games in order to fall down to Bronze, his match history will look like this: + Show Spoiler +
(starting league Diamond)
-12
-12
-11
-11
-10
-10
-9
-9
-9
-8
-8
-8
-11 (league change to Platinum)
-11
-11
-10
-10
-9
-9
-11 (league change to Gold)
-11
-11
-10
-10
-9
-9
-11 (league change to Silver)
-11
-11
-10
-10
-9
-9 (league change to Bronze)
etc.
It's counter-intuitive behavior given that MMR is supposed to travel faster than points, and if you're losing 100% of your games, your MMR should be moving very quickly indeed. The expected result is that your points lost should increase with each consecutive loss because that would move your points more rapidly toward your MMR. However, that's not what happens in reality. Consistently, point changes would start at -11 and gradually fall off until -8 or until they lost all their points, and this would reset after a league change. One possible explanation is that there is a phantom point value that exists which influences your point change in addition to the staple "adjusted points versus opponent MMR" formula, as SDream suggests. If true, then the higher your points drift away from this phantom value, the more you are capable of losing. Or, there could be something else at work.
They dont want to make it easy for us ... Oo
I loose track of all "special cases" i have in my program now ...
|
We are quickly making progress at narrowing down the league modifiers. We have the infrastructure down and excel all fired up and ready to go. Now it's mostly a matter of getting more data points from people playing games with the plugin. We already know we were on the wrong direction trying to use the numbers Blizzard posted (still no idea how they came up with them).
We will release an update with much more accurate offsets and league promotion lines soon.
Here's what we do with the data:
For the time being we don't try to estimate MMR change after a win/loss. What we know for certain is that MMR doesn't decrease after a win or increase after a loss. Since we know F, for each win or loss we know exactly a minimum or maximum value for the player's MMR in the next game.
We need two consecutive games to make an observation. There are 6 cases:
('low', and 'high' league refer to the league of opponent in the first match compared with the league of opponent from the second match)
1) Beat opponent from a high league.
2) Lose to opponent from a low leaue.
These two cases are the same in that they both give us a maximum value. We want this value to be as low as possible (which is the case when comparing the highest tier from the low league to the lowest tier of the high league) because then we can narrow down the possible offset between them.
Example:
Gold player beat Platinum player in game one and we calculated Gold player MMR to be 100 +- 14. In next game Gold player played a Gold player (win/loss irrelevant) and we calculated the first player MMR to be 83 +- 14. Since the player had at least 86 MMR in some tier of platinum before his first game and has at most 97 MMR in some tier of Gold before the second game, we know that the biggest difference possible between Platinum and Gold is no more than 11. And this is super conservative, because we don't even assume that the player gained any MMR from beating the Platinum player.
3) Lose to opponent from high league.
4) Beat an opponent from low league.
These 2 cases are the exact opposite than the first two. Instead of getting a maximum point for the minimum difference between the leagues, we get a minimum difference for the maximum difference between the leagues. We want this value to be as big as possible. I won't make this any messier with another example, I'll simply say that using this method we can determine that the difference between low Silver and high Gold is no less than 390, so that's a minimal size for both leagues combined. Together with the first 2 cases this helps create a picture of the offsets.
Finally
5) Beat opponent from same league then see MMR decrease.
6) Lose to opponent from same league then see MMR increase.
Both these cases give you a minimum offset between tiers of the same league. For example the difference between the two Gold tiers on EU are minimum 84. The difference between low Plat and High plat is minimum 170. Generally we're looking for the biggest value here that we can find in our data.
From 2594 game results thus far, here is what we've been able to compile:
(click on pictures for full version)
![[image loading]](http://s8.postimage.org/gpzmc3bvp/data_thus_far.jpg)
For example, on EU we can see that the offset between low Silver and high Gold is no less than 390, and the offset between high Silver and low Gold is no more than 23.
From these minimum / maximum results, here is our current best estimates for the offsets:
![[image loading]](http://s15.postimage.org/jnaej43yj/eu_us.jpg)
Keep in mind these will get much better the more results we have, and some minimum / maximum values are sorely missing / lacking. At the moment Europe offsets are the most accurate, followed by US. Sea has very little data and for KR server we have a grand total of 0 games collected thus far. Unfortunately since it appears each server has it's own offsets, we won't be able to estimate Korean offsets without this plugin being used by Korean players. If anyone here have a hand in a Korean community and can spread the word of this tool it would help a lot.
For the future: Beside getting more and more data and constantly narrowing down the results towards the real numbers, we could start estimating how MMR changes from games. This should help us narrow down each offset by a good 20-40 points and overall make life easier. TODO.
As mentioned ,expect more accurate stats in the plugin soon.
|
That's strange, I have the plugin and I'm playing on the Korean (tw version) server. Im sure many others are also. Why would you have zero datapoints?
Saying that I think I remember seeing some errors and games not recording. I'll see if I can look at the log when I get home.
|
On May 22 2012 13:01 Peleus wrote:
That's strange, I have the plugin and I'm playing on the Korean (tw version) server. Im sure many others are also. Why would you have zero datapoints?
Saying that I think I remember seeing some errors and games not recording. I'll see if I can look at the log when I get home.
Perhaps you're using one of the old versions? I was referring to the data file of 1.7.1+ versions. There were a lot of changes and fixes since the earlier versions and they're hard to work with by now.
|
I feel like my lack of math ability is hindering me in understanding all this. I'm trying to figure out what seems to be the range of MMR in Masters league and where I fall in it. For instance, it appears that I've been hovering around ~400 MMr for the past two days, but I'm really not sure what that means without a solid basis for comparison.
|
Just click 'analyse offsets' and tick the league boundaries and you will see your position on the Bronze->GM spectrum.
|
Maybe there is a bug in the korea sites. Maybe it gets confused with the non-english stuff. Personally, sometimes when I try to go to a korean profile with hangul as the nickname, I get into the account login page instead. I have no idea why this happens, but that might be happening to the plugin. Maybe you should test it more, I'd bet the hangul is bugging somewhere indeed.
+ Show Spoiler +Also, I only played 2 games with the plugin, but you said in the front page that if we allow it, it would send the data to the server, how do I allow the plugin to do it and how do I know if you are getting my data? (Edit: found it, it is marked to do it automatically)
I still prefer the names I and Exc used to use in the past. Rank S (closest to masters) for the highest tier, then A for the second highest, till F (closest to bronze), but in this thread I will do my best to follow your standard.
Question:
If I am at a US Tier1 Platinum division and I aim to diamond (tier0 is fine) next season. Does that mean I'd need 115 adjusted points only? It feels so wrong, specially when it's standard to have at least 73 points.
I have 230 adjusted points right now at plat tier 1, so, if I convert it to Diamond tier1, I'd have 52 points, which is lower than the 73 "cap", so do I deserve a tier1 spot or not?
If you don't know the answer yet, what you'd guess?
|
On May 22 2012 14:00 SDream wrote:Maybe there is a bug in the korea sites. Maybe it gets confused with the non-english stuff. Personally, sometimes when I try to go to a korean profile with hangul as the nickname, I get into the account login page instead. I have no idea why this happens, but that might be happening to the plugin. Maybe you should test it more, I'd bet the hangul is bugging somewhere indeed. + Show Spoiler +Also, I only played 2 games with the plugin, but you said in the front page that if we allow it, it would send the data to the server, how do I allow the plugin to do it and how do I know if you are getting my data? (Edit: found it, it is marked to do it automatically)
I still prefer the names I and Exc used to use in the past. Rank S (closest to masters) for the highest tier, then A for the second highest, till F (closest to bronze), but in this thread I will do my best to follow your standard.
Problem with s and a and so on is that i program this in lists and array and t0 is just first entrance of my array.
way easyer to handle if you dont have to translate all the time.
[
Question:
If I am at a US Tier1 Platinum division and I aim to diamond (tier0 is fine) next season. Does that mean I'd need 115 adjusted points only? It feels so wrong, specially when it's standard to have at least 73 points.
harder to get than you think. you have to do the same work to get the 115 adj in your tier than the other guy have to do to get his 300 in the other tier.
Thats why we do all of this. Look at the graph of my tool and you see how far you are away. if its work correct 
And dont forget this is all shadow and smoke. Diamond , Platinum that are some random offsets.
If you have 1 mmr more you have a dia icon , less a platinum icon. Sometimes you even have more and a platinum.
Its all just icons...
Btw i need min 5 games so the guess of your tier is correct.
I have 230 adjusted points right now at plat tier 1, so, if I convert it to Diamond tier1, I'd have 52 points, which is lower than the 73 "cap", so do I deserve a tier1 spot or not?
depends on your Dmmr not your adjusted...
Play some games and see.
I can not tell my numbers are allways correct but i can tell that you see after 5-10 games how far away you are from dia.
On May 22 2012 13:15 Zennith wrote:
I feel like my lack of math ability is hindering me in understanding all this. I'm trying to figure out what seems to be the range of MMR in Masters league and where I fall in it. For instance, it appears that I've been hovering around ~400 MMr for the past two days, but I'm really not sure what that means without a solid basis for comparison.
This thread is over the theories and the math.
Here is the practical part:
http://www.teamliquid.net/forum/viewmessage.php?topic_id=334561
Korean BUG should be fixed in my unstable version.
Would be nice if you guys can try and confirm that!
I did not notice that sc2/en/profile is a redirect to sc2/ko/profile oO
So i thought /en/ would work ...
|
Very nice plugin, keep it up!
|
I'm pretty satisfied with the speed at which I can integrate new data into the numbers now. My excel fu must be improving. I prefer not say how much time it took the first time...
We're up to 3064 results. I thought since we still missing some numbers for the regions, might as well combine EU and US numbers while we get our bearings in the league offsets. EU and US are almost certainly a bit different, but I think it's very worth it while looking for the rough numbers and we can worry about finding minute differences later. Here's what the data looks like for both regions:
click for full image.
![[image loading]](http://s13.postimage.org/dfe76kyqb/EU_US.jpg)
My observations:
It's still hard to say how big is Bronze league. So far we can only pin it at minimal size of 380 which should be grossly underestimate. Most certainly Bronze is bigger than at least Diamond. ~550-560 is my estimate.
Silver, Gold and Platinum are very small. Put together they can all fit within Diamond league.
Diamond league is about 530 and Master league about 750, with GM from entry level into #1 spot about 460.
One thing to note is that Bronze tiers do not behave similarly to Diamond. Diamond has ~150 offset between high Diamond and Master which appears to be unique to Diamond (I wonder if this has anything to do with what SDream brought up, that Master is the first league that will not try to push a player adjusted points towards 73 should he fall below it in MMR?) High Bronze comes very close to Low Silver, no more than 30 points away, possibly almost identical.
Silver league is no more than 200 points across, comes very close to both high Bronze and low Gold.
Gold is the smallest league in MMR range it covers, a mere 100-110 points or so (no more than 130). High Gold is basically identical to low Plat.
Plat appears somewhat similar to Silver, with minimal size of 170 and maximum distance between high Plat and Low Diamond of 76. I've Got to wonder if this distance will reveal itself to be smaller and High Plat will be very close to Low Diamond, or will there always be a Diamond Tier's worth of difference between the two.
Nothing to add about Diamond than what Excalibur described in his post.
Master and GM are pretty big, at around 750-800 and 460. Between the both of them they're bigger than Silver-Gold-Plat-Diamond combined. Not a bad skill range for leagues that supposed to have 2% of the players in them (closer to 4% of players in reality).
I remember the threads asking how many tiers of players are there in SC2, with a tier being defined as being able to 10-0 the other player or some such. Well, between these league ranges and the F formula, now we can know. Simply let the ladder system calculate it for us:
Let's take an expected score of 90% to represent a tier difference (with player A predicted to win 9 out of 10 games vs player B). Using elo, Expected score 0.9 = 1/(1+10^((MMRb-MMRa)/400)) we get 382 MMR difference. Assuming Bronze size to be 550, we get a total ladder size of 550+200+100+200+535+800+460=2845. Divided by 382 we get 7.4 tiers. Keep in mind this is for EU & US players, and that if Korean pros flocked into EU or US servers these servers will likely see a stretch in skill levels (ignoring MMR cap) adding more tiers.
|
United States12249 Posts
It's strange to me how some of the determined values are pretty close to the Blizzard-reported "promotion numbers" but others are not. Corrected for adjusted points, we get the following values from Blizzard's chart:
For 1v1:
Bronze: 835
Silver: 435
Gold: 435
Plat: 435
Diamond: 535
Master: 770
Bronze, Diamond, and Master are pretty close to the collected data, but Silver, Gold and Plat aren't.
If each division tier is about equal size across all leagues -- and I understand this is an enormous assumption -- and that size corresponds to 63 points, we would expect Silver and Plat to cover 3 tiers' worth of skill because they have 3 tiers compared to Diamond's 7, and it seems that they are (collected value around 200). Gold has 2 tiers and we would expect about a range of 130 or so, which apparently it is.
What's strange is how far those values are from the Blizzard chart of 435 for Silver, Gold, and Plat. Obviously we know that Gold can't cover the same MMR range as Silver and Plat without the population of Gold being intentionally larger than 20% because of the player distribution curve, so the 435 value is already called into question right away. It's still odd that Silver and Plat are off by the degree that they are.
|
One thing to note is that Bronze tiers do not behave similarly to Diamond. Diamond has ~150 offset between high Diamond and Master which appears to be unique to Diamond (I wonder if this has anything to do with what SDream brought up, that Master is the first league that will not try to push a player adjusted points towards 73 should he fall below it in MMR?) High Bronze comes very close to Low Silver, no more than 30 points away, possibly almost identical.
It has nothing to do with the rule.
The rule exists in order for bronze players to not reach a bug that used to exist, where some players are so bad (sorry players) that they could reach into the negatives. There were bronze players with -300 DMMR at bronze in season 1 and 2, that in turn made them forever 0 points, they weren't able to even spend their bonus points... that's why they fixed that in S3 and they extended it till diamond for the same reason they don't show the losses till diamond.
The reason diamond Tier S is 150 points away from Master is because there wasn't a master league in the begging. Tier S Diamond was somewhat supposed to be the "master" league in a way, the highest tier, and that's it.
When they brought master league and made the math to make the cut into 2% of the population, they could have added 2 more tiers into diamond, but they simple decided it wasn't necessary.
The reason that bronze -> silver doesn't have 150 points in between them is pretty obvious to me, let me try to explain. The thing is, if there was a paper league, where only the 2-4% of the worst players would be allocated, then yes, the difference between bronze tier F and this paper league (with no tiers) would be like 150 (or more!).
You need to think about the league as beginning at the gold level. Gold till bronze mirrors the gold till GM (but bronze = diamond + master + GM) at design, so you cannot compare bronze -> silver with diamond -> master, you need to compare bronze <- silver to platinum -> diamond, that is the correct correlation!
Edit: that said, the reason for the rule I found is that there are several bronzes with MMR lower than "0".
|
|
|
|
|
|
EPT
