EPTHow to calculate your MMR from results of 1 match - Page 14
| Forum Index > SC2 General |
|
SDream
Brazil896 Posts
| ||
|
SDream
Brazil896 Posts
He lost to me and he was placed at diamond, probably rank F (t0). Then he lost to another one and he is demoted to plat! 2 losses, 1 placement and 1 demotion, pretty interesting. Also, I won 12 points from him, so it almost feels like I just played THE guy to understand where is the border to plat - diamond, can you updata de data so I can download this one game and have another look at it? o.o It seems that the plat-dia border is 60+/- points lower than what you have in the plugin if that's true and I am sitting at the border right now. | ||
|
Not_That
287 Posts
I'm afraid I think your game isn't there SDream. When you play against someone in his placement match he doesn't win/lose points, thus we can't calculate your DMMR from the match and consider it a bad data point. One thing to note about the other guy: I believe in the first X games you play on bnet your uncertainty value is unusually very high. His MMR may be shifting wildly from match to match (probably few games' worth of change each match). Having said that if you're interested you could stalk him for a bit until he plays another game. Then check the points change of his opponent and calculate his DMMR manually. | ||
|
Tropico
Canada4 Posts
| ||
Excalibur_Z
United States12235 Posts
On May 26 2012 06:23 Not_That wrote: Latest file with 6221 games. This one has the games with from the pre 0.9 versions as well. We're still getting data from that version, so if anyone still uses an old version please update :D I'm afraid I think your game isn't there SDream. When you play against someone in his placement match he doesn't win/lose points, thus we can't calculate your DMMR from the match and consider it a bad data point. One thing to note about the other guy: I believe in the first X games you play on bnet your uncertainty value is unusually very high. His MMR may be shifting wildly from match to match (probably few games' worth of change each match). Having said that if you're interested you could stalk him for a bit until he plays another game. Then check the points change of his opponent and calculate his DMMR manually. That's correct. The initial 5 placement matches use a special "placement MMR" which probably is prone to more dramatic changes after each game, in order to quickly get you to roughly where you should be. One thing is strange though. Allegedly, an exponential moving average of MMR is used over your past X games to gauge volatility, and when the slope falls within a certain threshold, that's when you get promoted. However in SDream's opponent's case, he was demoted after only 1 game which means the slope would certainly have been -1 (assuming his live MMR equals his placement MMR after the 5th game). In what situation would only two data points (the game concluding placement and the game after placement) allow for a league change? Obviously we've seen this happen numerous times, but why? Does the slope requirement only matter after a minimum number of games have been played? EDIT: Oh wait, SDream didn't specify if his opponent was playing the last of his initial 5 placement matches or his inter-season carryover placement match. That's important to know because it's possible the match history persists from season to season, which would have carried over his moving average data as well if he were playing his carryover match. | ||
|
SDream
Brazil896 Posts
I couldn't track his last 1x1 game, the history doesn't have it, but he was placed this season with 1 game, that's for sure. If I won 12 points at the same time he was placed at diamond, doesn't that mean that my current AP = diamond t0 border, slighlty higher? | ||
|
Not_That
287 Posts
Then there could be the slight possibility that he wasn't t0 Diamond. And you need to know your Plat tier. | ||
|
SDream
Brazil896 Posts
On May 26 2012 08:00 Not_That wrote: Your adj pts before the match equal some MMR value in lowest tier Diamond +- 14 seems like a reasonable assumption. If t0 is 63 MMR wide + up to 14 points if we're unlucky, we're looking at Diamond tier 0 lying entirely within {your adj pts - 77, your adj pts + 77} range. Then there could be the slight possibility that he wasn't t0 Diamond. And you need to know your Plat tier. Your plugin says I am t1, plat A, he has enough data now, should it be accurate I guess? My AP right now is 225, so it was 213 before this game. I really doubt he was a higher tier than t0 diamond if he was demoted that fast. 213 within a range of +/- 63 is probably the difference from t1 plat and t0 diamond, unfortunatelly it's still a huge range to work with. 63+63+88 = 214 My wish  | ||
|
Not_That
287 Posts
Back to focusing on known tiers for the moment. I'm looking only at consecutive games vs Master and consecutive vs GMs. This way the tier of the opponent is guaranteed to be the same in both matches, so the DMMR calculated for the player is consistent. Let MMR1 be the DMMR value of the player calculated from the first game. Var1 be the +- uncertainty range of MMR1. Likewise MMR2 and Var2 for the second game. Some stats: - Wins 444, losses 203, 68.6% win ratio. - On average, MMR increased by 16.26689 for wins and decreased by 15.8916 for losses. - On average, player MMR - opponent MMR = 9.98 for wins, and -59.4 for losses. - Avg adj pts change for win = 12.05, and for losses = -10.216 Some observations: - The win ratio and adj pts change for losses being much smaller than adj pts change for wins require explaining. The win ratio is easily explainable by players not using ramscanner. When you lose a game and not use ramscanner, more often then not your DMMR can not be determined and I don't use your game for this calculation. The adjusted points change being close to -10 for losses whereas 12 for wins is more strange. - Seeing how 16 is the average MMR change for wins and losses, I think it's a good value to use. If opponents are about even, 16 MMR change for both players seems like a reasonable assumption. If one player's uncertainty value is very high he will probably experience a higher change, and his opponent will probably experience a smaller one. If we can find the rating system the ladder system implements, this value should come in very useful. Also note it's higher than the average adj pts change of 12, so I think we can now finally say with confidence that MMR is indeed more volatile than adj pts. - I think F may be off by a point in some of the cuts. Otherwise I'm seeing results that don't make much sense. I'm looking for the highest MMR increase/decrease I can be certain of, by looking at MMR2-MMR1-Var1-Var2 when first game is a win, and MMR2-MMR1+Var1+Var2 when first game is a loss. I'm getting some values as high as 42 which I'm sure are F errors / data errors, because they're not even against players much higher / worse and the game count of the player isn't that high, so their uncertainty is probably normal. The highest results I would get behind of right now and that I feel are reasonable considering the MMR difference of the players are +20 and -19.5. That's not to say MMR can't change by a greater amount from a single game, just that perhaps we require additional data to be sure. Keep in mind the stars have to align just right for us to see this value, because we are basically waiting for the value to lie at the top of the uncertainty range in the first game, and at the bottom of the uncertainty range in the second game after a loss (and vice versa after a win). | ||
|
Not_That
287 Posts
![[image loading]](http://s15.postimage.org/hbw63z3nb/glicko.jpg) I entered my MMR at the first game, picked 76 as the RD value that I used for myself and all my opponents, and let it calculate 90 games forward with no intervention. The green bar is my what Glicko calculated my MMR to be. For my opponents' MMR I used the middle of the range. The graph can be improved somewhat if I manually adjust each opponent MMR within it's uncertainty range, but I don't believe getting a hit for all games is possible. Unfortunately the fact that every opponent also has his own uncertainty RD value which affects your MMR change after game may be insurmountable. I think this is a dead end. One thing though: If your RD value increases whenever you stop playing for a certain amount of time, it could help explain some things. | ||
|
Not_That
287 Posts
| ||
|
Not_That
287 Posts
Diamond: 7 Tiers - Tier 1: +150 from Master - Tier 2: +213 from Master - Tier 3: +276 from Master - Tier 4: +339 from Master - Tier 5: +402 from Master - Tier 6: +465 from Master - Tier 7: +528 from Master Well what is 150 from Master? Where does it come from? I'll tell you where it comes from. It comes from 63+88. Diamond is divided evenly into tiers, and Master is located exactly 1 Diamond tier size higher than the highest Diamond tier. Note that these numbers are old, which brings me to... It's time for new offsets... Here they are: ![[image loading]](http://s10.postimage.org/irb90iy45/Offsets_2.jpg) - Master is still 1 Diamond tier up from highest tier Diamond, plus 88 points, at least for EU. For EU 51 Diamond tier size, and 51+82 is distance between highest Diamond and Master (hmmm, this makes a good point actually that Diamond tier should actually be 50 instead of 51, that way it'll be 50+89 which is almost 88). For US it's more off, 53 tier size and 53+77 distance from Master. This is worth looking more into. - Bronze has insufficient data, thus should not be read much into. - Due to 88 acting as a cap (and I dare say now, "zero"), finding the offsets between leagues is more difficult than finding the offsets between tiers of the same league. The problem is that so many inter league games have the opponent from the higher league capped. As a result I am more confident about, say, the difference between Platinum tiers, than I am about the difference between high Plat and low Diamond. Diamond-> Master is an exception because Master has no 88 / "zero" cap. - These are the offsets, not promotion numbers. I would guess the moving average MMR needs to stabilize at 88 points above the offset numbers to be eligible for promotion into leagues below Master. I wouldn't read too much into how #1 US and #1 EU match. Any errors in the numbers are probably more than enough to separate the two. - Master -> GM modifier estimate raised to 452 from 430, however no tight numbers in the data, so who knows. - It may yet be possible that Blizzard's numbers are meaningful. Looking at their old EU numbers, if we subtract 88 from all of them we get 697, 317, 257, 317, 437. My numbers for Europe are 490, 200, 135, 270, 439. Diamond is very close all by itself (same for US). It could still be that for the other leagues they grouped up the league of the player plus part / all of the next league to get their line to make sure they won't get any "I reached the points Blizzard said where is my promotion?" comments. For example Silver+Gold comes out 335 vs their 317 number, and Bronze+Silver comes out 690 vs their 697 (even admittedly my Bronze number is not accurate enough to be used to make such conclusions). My methodology: Seeing as how players on average get 16 MMR for wins and lose 16 MMR for losses, I decided to use that for estimating new offsets. I looked at pairs of consecutive games and calculated the DMMR of the player for both games. Let DMMR1 be the DMMR of the player before the first game, var1 be the uncertainty range for the DMMR1 of the first game, and likewise DMMR2 and var2 for the DMMR calculated from the second game (so it's the DMMR after the first game has been played). By adding / subtracting 16 from DMMR1 depending on win or loss, and then comparing it with DMMR2, I expect to get the difference of offset of the two opponents, plus-minus up to var1, plus-minus up to var2, plus whatever was the difference in actual MMR change from 16. Here's what it looks like for US Diamond league: ![[image loading]](http://s18.postimage.org/8fxfgn9l1/US_Diamond.jpg) The bars are the absolute value of the difference between DMMR1 (+16 for wins / -16 for losses) minus DMMR2. Red lines are averages of tiers. I grouped games into tiers by eye and they are not necessarily accurate. However when averaging many games the errors become smaller. Black lines are my chosen tier offsets. Diamond is a big league with many tiers. How does a smaller league look like? US Platinum: ![[image loading]](http://s16.postimage.org/5dup73dsh/US_Plat.jpg) The rest of the leagues: + Show Spoiler + US Gold: ![[image loading]](http://s13.postimage.org/f88288qgz/US_Gold.jpg) US Silver: ![[image loading]](http://s13.postimage.org/s8szw5rsz/US_Silver.jpg) EU Diamond: ![[image loading]](http://s17.postimage.org/s3qm8b9jf/EU_Diamond.jpg) EU Plat: ![[image loading]](http://s7.postimage.org/idebsua2f/EU_Plat.jpg) EU Gold: ![[image loading]](http://s17.postimage.org/40gpc62i3/EU_Gold.jpg) EU Silver: ![[image loading]](http://s14.postimage.org/m2pao2pz1/EU_Silver.jpg) US Bronze: ![[image loading]](http://s15.postimage.org/cakqpf1on/US_Bronze.jpg) EU Bronze: ![[image loading]](http://s18.postimage.org/4dos0j8zp/EU_Bronze.jpg) As mentioned, Bronze is league with least data considering the number of tiers. There were a few data points for more than a league apart which I also used. | ||
|
Not_That
287 Posts
![[image loading]](http://s17.postimage.org/algk9gc2z/DMMR_vs_below_Master_over_time.jpg) Edit: Problem on our end in version 3.0 of the tool. Nevermind. | ||
|
Excalibur_Z
United States12235 Posts
We already know that 73 is the reset point for adjusted points, but if a separate DMMR exists that's slightly higher than that, it means that points can't reliably be compared between leagues or divisions even if offsets are factored out. The reason is because the DMMR influences points higher than the reset point up to a certain minimum value. That doesn't inherently mean the reset point is 0 nor that the DMMR point is 0, rather that these are just elements that exist which drive up points somewhat. There could be several reasons for this: maybe the DMMR always existed at 73 but was changed to drive up points a little higher so that players would have a reason to play after being promoted, maybe this helps drive points up higher (up toward that DMMR) in the event that you're really sucking and anchored at 0. Remember that as long as the DMMR exists, players you face will always appear to be at that DMMR value for the purposes of awarding points, but this doesn't inherently tell you anything about what their actual MMR is if they are below that DMMR value. | ||
|
SDream
Brazil896 Posts
The 88 cap exist for the reason you already known (low DMMR bronzes mainly), the 150 points from S to master exist just because they need to make the cut in a way that the top 10% of diamond only would get into that league. If they need the top 1% the cut would be higher and the difference between Dia S and Master would be like 800, that is simple as that, they have no connection. That doesn't mean the 88 can't be the "zero" when looking at DMMR, but the zero could as simple be 0, therefore, only players with more than 88 DMMR (unless opponent is master) would be available to our data. To answer this question all we need is some games between master players and diamond tier S. The problem is we have no idea who are in the tier S for sure, all we do have is games between diamond and masters. There is a way to be placed in a rank S for sure, if you are plat and you start winning almost 100% of your games till you start facing master, you will be promoted to a tier S before getting promoted to Master. (the same is true for other leagues) Edit: BTW, does anyone know a way for me to discover the character code (to add as a friend) of a profile? Let's say I want to stalk this player: http://us.battle.net/sc2/en/profile/693604/1/EGIdrA/ Is there a way for me to add him? Does anyone know? If there is and you don't want everyone to know, PM me. I don't want to stalk Idra, but top diamonds that I find using sc2ranks. | ||
|
warblob004
United States198 Posts
| ||
|
Not_That
287 Posts
On May 29 2012 02:46 Excalibur_Z wrote: I like that DMMR + tier offset concept for Master league's offset, since I believe you mentioned Master doesn't carry DMMR with it. Instead, it must be built into the gap between top tier Diamond and Master. We already know that 73 is the reset point for adjusted points, but if a separate DMMR exists that's slightly higher than that, it means that points can't reliably be compared between leagues or divisions even if offsets are factored out. The reason is because the DMMR influences points higher than the reset point up to a certain minimum value. That doesn't inherently mean the reset point is 0 nor that the DMMR point is 0, rather that these are just elements that exist which drive up points somewhat. There could be several reasons for this: maybe the DMMR always existed at 73 but was changed to drive up points a little higher so that players would have a reason to play after being promoted, maybe this helps drive points up higher (up toward that DMMR) in the event that you're really sucking and anchored at 0. Remember that as long as the DMMR exists, players you face will always appear to be at that DMMR value for the purposes of awarding points, but this doesn't inherently tell you anything about what their actual MMR is if they are below that DMMR value. Yeah I didn't mean to imply that all the leagues offsets fit nicely with DMMR values. It was always pretty clear to me that promotion MMR numbers and leagues offset are not necessarily equal. Why 88 and not 73 or any other number I do not know. And yes we can only observe DMMR through the points change. I am hoping (and I believe) that capping it at 88 is the only thing Blizzard is doing with it, so any number above 88 we can trust, and the others may be 88 or lower but we can't tell. SDream, no easy way I'm afraid to know character codes. Several times I wanted to see a player's match history but couldn't due to no way to find him in game. T.T I don't understand what you are looking for in Diamond tier S vs Master games? We can probably identify some such games in our data right now, but what exactly are we looking for from them? Thanks warblob. Hopefully we can walk these last few steps. As always, anyone reading this is more than welcomed to join along and help us figure things out. You can find me on teamspeak quite regularly these days. If not, feel free to PM me, or post here. | ||
|
EnderTX
Israel23 Posts
i just have to say that all of those graphs r giving me a headache :S | ||
|
Not_That
287 Posts
Blizzard themselves released uniform promotion numbers for all servers for the end of season 6. I assumed they just decided to post a single chart so it'll be less confusing, but they may have been accurate after all. This is good for us because it means more data can be analyzed together. First, the new offsets which are my current best estimates based on some 10k games: ![[image loading]](http://s8.postimage.org/kmi1sqv45/offsets_3.jpg) - Bronze still missing a lot of data, so Bronze numbers are less accurate than the others. My methodology was the same as in last time. Assume win awards 16 MMR, a loss decreases MMR by 16. Look at pairs of games against players and note the difference of the calculated DMMR from both matches with my calculated value. To demonstrate the degree of accuracy (and inaccuracy) my method produces, let's look at the Master / Master graph. Since Master is a single tier, all games must include 0 in their uncertainty range. ![[image loading]](http://s16.postimage.org/ca1cmk6n5/master_master.jpg) As you can see some games do not have zero included in their range, particularly at the higher uncertainty ranges. This could be due to the player MMR changing by more than 16 either way, especially considering these matches are matches where the players received very much or very few adjusted points for. Less likely is a slight error in F. Overall I am pleased with the accuracy. Also keep in mind that we expect 75% of the data points to fall within the middle 50% of the range, due to each game having plus-minus half the uncertainty (on average), and we add the two together. Let's start with the easy leagues first, Silver, Gold and Plat: ![[image loading]](http://s18.postimage.org/lk4bs0r4l/Platinum_Platinum.jpg) ![[image loading]](http://s15.postimage.org/4y6xdu6lj/Gold_Gold.jpg) ![[image loading]](http://s18.postimage.org/ldv82o7dh/Silver_Silver.jpg) These leagues are easy because the number of tiers is low. Now let's look at a harder league, Diamond. First let's look at the chart derived from pairs of matches against Master and against Diamond: ![[image loading]](http://s16.postimage.org/q5vst7yqp/Master_Diamond.jpg) Even though Diamond has 7 tiers, we can still get a good match. This is due to all of them being compared on the chart with a single tier of Master. But what would happen if we didn't have Master league to compare to? Let's look at the Diamond / Diamond chart: ![[image loading]](http://s14.postimage.org/4emq6md0t/Diamond_Diamond.jpg) This is much less pretty. You need to make sure you're reading this chart correctly. This is NOT the seven tiers laid in order. These are the individual offset gaps between 2 consecutive games against Diamond opponents. So at the bottom we see all the pairs of games that were played against 2 guys of the same tier. Followed by all the games that were played against 2 guys one tier apart, then two tiers apart, etc. Finally at the top we see games that were played against highest tier player followed by lowest tier player (or vice versa). Obviously this is much more rare. Therefor it is not surprising that the bottom portion of the chart is most populated. If this was the only chart we had and did not have Master / Diamond chart, we would have harder time. Last of the inner league offsets, Bronze league, the hardest of them all: ![[image loading]](http://s11.postimage.org/jk79sf48v/Bronze_Bronze.jpg) Not only does it not have an adjacent league with a single tier, but it also has a small amount of games. I got, at most, a single pair of games against a highest Bronze tier player followed by lowest Bronze tier game, and I believe I got 0 such pairs. This is how I got all the tiers within the leagues. To get the offset between highest tier in one league and lowest tier in next league, I did the same. Unfortunately this has much less data than the same league cases, due to me having to throw out every game against opponent below Master where the calculated DMMR has 88 in it's possible DMMR range, due to the DMMR cap of 88. Once again, Diamond-Master offsets are easiest and most accurate by far. Keep in mind these charts are more hectic due to them having all the tiers from both leagues combined in them. Easiest is to simply looking at the minimal offset, which is what we care about. Platinum Diamond: ![[image loading]](http://s11.postimage.org/4r29iis73/Platinum_Diamond.jpg) Platinum Gold: ![[image loading]](http://s16.postimage.org/ikag89yqp/Platinum_Gold.jpg) Gold Silver: ![[image loading]](http://s11.postimage.org/frxenjkfz/Gold_Silver.jpg) Bronze Silver: ![[image loading]](http://s18.postimage.org/njhprtavp/Bronze_Silver.jpg) Master Grandmaster: ![[image loading]](http://s14.postimage.org/bbvlr4bcd/Master_GM.jpg) Now what? - I'm curious whether we can now figure out how Blizzard reached the numbers that they published. I'm still thinking along the lines of league_size + next_league_size = their numbers, or possibly next_league_size minus 1 tier. - As always with more data, these numbers can all be adjusted slightly, but I think they're pretty good. These are based on some 10k games. - GM -> Master offset is now 454 - Even though the offsets are the same, keep in mind MMR across multiple servers can not be directly compared due to different player pool. Heres the file I was working with in case anyone want to have a look | ||
|
Excalibur_Z
United States12235 Posts
Bizarrely, Gold appears from this chart to be much wider per tier. That seems like an anomaly to me, but it could be true after all. I don't know that the offsets are the same across all the servers. That may be the case for EU and US and maybe even KR, but I have a feeling SEA will forever be an outlier. - I'm curious whether we can now figure out how Blizzard reached the numbers that they published. I'm still thinking along the lines of league_size + next_league_size = their numbers, or possibly next_league_size minus 1 tier. The numbers provided in the chart are by Blizzard's own admission estimates and guidelines. In the past I thought we could reverse-engineer those values by factoring out the bonus pool for the season and dividing the tiers evenly, but that's probably not the right move. We need to keep in mind that the numbers selected are, first and foremost, clean round easily processed numbers and that the inner workings of the ladder itself are Not_That  I still believe there is some meaning behind those numbers but we can't read too deeply into them either. For example, if the given Bronze -> Silver number is 1100, then why did they not choose 1000 or 1200? As for how they were derived, bottom tier to next league's top tier sounds plausible to me, maybe with an extra tier's worth as a buffer? If you're anywhere in Bronze and you jump ten plus one tiers in points, pretty safe bet you won't be Bronze anymore next season. | ||
| ||
