|
On May 17 2012 09:16 celeryman wrote:Show nested quote +On May 17 2012 09:12 skeldark wrote:On May 17 2012 08:47 celeryman wrote:Thank you for your responses. One clarification: Then because each player has a different MMR, I added a different value to each player's MMR (same value to all of that player's data points) to see if they would fit on a line. I understand that you took a set offset for each player, and then used that across all their games. I don't think your assumption that MMR doesn't fluctuate between the games you sampled is bad; there's very little bias I could imagine being introduced from that fact (MMRs should fluctuate randomly with the exception that some bias might be upward, but that's a slow, and consistent bias). But how did you determine which player got which offset, and how were the offsets spaced? Bonus pool thoughtDoes the sc2gears plugin back-calculate the bonus pool adjustment? Here's why I ask. If someone's bonus pool is zero, then it's easy, just subtract the total pool from their total points (with the exception of if they ever lost with their points at 0, but even then, small detail). But imagine a player who's on day one (bp at 0) gets 500 points. Almost all of those points will be non-bonus. Then, they don't play a few days (bp at 100). By our standard formula, at this later date they should have 400 adjusted points. But, they actually have 500. This problem will show up for any player who plays with a bonus pool at 0, and then later has an unspent bonus pool. The degree to which it skews the calculations will be a function of the time played while bonus was at 0 (think most high level players), and how large the unspent is. Maybe in practice it's not big and we can ignore it, but when taking the raw data to try and establish the "F" function, the data needs to be only players with BPs at or near 0. I'd like to know if you think, from the data you've seen, that's a common phenomena. To find out the adjustment points i do: His points - his unspend unspend points 20 sek after the game is finished - total bonuspool of this sec of the server he play on. So the guy with 500 points have: 500 points +100 unspend -100 total = 500 adjusted To find out first two i can check the webpage. For the total bonuspool i could check a inactive player but thats a very dirty method. Instead i calculate it with the bliz bonuspool algo from the time back the session starts. The session start time i backcaluated depending on the bonuschange +- 5 sec. The bonus pool is calculated with your computer clock and for the case this one is not accurate i backcheck a internet atomic clock. The problem is when he then plays the next 100 points off (say wins 100 actual + 100 bonus). Now your formula will have him off by 100. I didn't include that second step because I thought it was implicit.
500 +100 unspend -100 total= 500 adjusted
now he spend the 100 with 100 real = 700 points
700 + 0 unspend -100 total = 600 adjusted.
Thats no problem at all. when you have to find out how many of the changepoinsts e.g. +10 ,is bonus and how many are real it gets complicated.
|
You're right about the second post, I was wrong in that. I got sidetracked.
But my original point was not that it's hard to calculate adjusted points, but that if you're going to walk backwards through a line of player's games, their bonus pools at the time of that game can fluctuate from what you observe later, even only hours later. If you knew the precise time of that game you could possibly calculate the total bonus pool at that time, and then walk backwards through their games, adjusting for how many points they gained or lost, and arrive at what their points where at the time of the earlier game, but I don't think anyone's doing that right now. And even if they were, you'd have to use the time-stamp info out of sc2gears, not off the in-game interface.
|
I found out F!
The ladder system uses an ELO system with k value of 24 to calculate points change after a match. My initial linear F with a slope of 32 was a good approximate, but it missed by a lot at the extreme ranges (0-2, 22-24). Here are the new numbers (next to the old linear F):
![[image loading]](http://s13.postimage.org/6di76zk83/ELO_F.jpg)
Some math:
+ Show Spoiler +playerwon?: equals 1 if true, 0 if false.
adj_change: points change of player. if player lost the game add 24 to this.
new_MMR_a = old_MMR_a + 24 * ( playerwon? - Expected_score_a)
Expected_score_a = Qa / (Qa + Qb)
Qa = 10^(old_MMRa/400)
Qb = 10^(old_MMRb/400)
adj_change = 10^(old_MMRa/400) / (10^(old_MMRa/400) + 10^(old_MMRb/400))
When calculating player A's MMR, use player B's adjusted points for player B's "MMR" and player B's adjusted points change. As always, F(player A MMR, player B adjusted points) -> {0..24}
For fractions I rounded adj_change to the nearest integer which I believe is logical. Even if Blizzard keeps MMR as floating numbers, rounding is probably best we can do with what we have.
Here is a graph showing old linear F vs new ELO F for my last 95 games:
![[image loading]](http://s15.postimage.org/yya3pn99j/ELO_vs_linear_F.jpg)
The bars are my MMR calculated with old linear F. Green bars are wins, Red bars are losses, error margin at the top of the bars are 32 across representing possible MMR values. The black area on top of the blue area represents possible MMR range using new ELO F calculation. Black line is adjusted points. Missing games are games vs Diamond opponents or opponents I have no data for how many adjusted points they won.
The old and new F are fairly close usually. This is mostly due to me usually getting opponents against whom I get 4-20 points for winning. The new F represents a much bigger change for games at the extreme ranges, typical at new seasons, for highly rated players, for players whose adjusted points haven't caught up with their MMR yet, etc etc.
This discovery should help explain some of the irregularities we've seen in the data, as well as help us on our journey to to figuring out how MMR changes after each match, which in turn will go a long way in helping us finding all the division tiers and leagues offsets.
|
I will add this to next version of the tool and correct old data automatic,
After the f function is done we can concentrate on the offsets.
However with the higher deviation at high point change we can not use the edge data to do so.
But we can now work more accurate with the "middle change" data.
|
|
|
This is amazing thanks dude
|
I'm impressed how much of the hidden system has been figured out by now... I bet those guys at Blizzard feel owned by now  (Same with sc2gears for the win/loss%, although that is much much simpler).
|
On May 21 2012 17:15 Morphs wrote:I'm impressed how much of the hidden system has been figured out by now... I bet those guys at Blizzard feel owned by now (Same with sc2gears for the win/loss%, although that is much much simpler).
Why would they feel owned? They aren't deliberately trying to hide it in the sense of it's horrible if people know, it's simply the didn't make it visible because the masses would QQ over how volatile it is if you were looking at it after every game.
Either way, nice work for figuring it out.
|
I debated with myself whether or not I should make a new thread for data analysis. Eventually I decided not to do so, mostly because I don't think it'll take much longer.
Look at this graph:
![[image loading]](http://s8.postimage.org/guyboqcw5/graph1.jpg)
Graph's legend:
+ Show Spoiler +These are 14 consecutive games of an EU player (let's call him Smith) in Silver league. Smith won all 14 games. I know these games are consecutive and there aren't any losses between them that are missing from the data because his adjusted points are consistent, and win numbers consecutive. His first win here is his 114th win this season. First game is on the left.
The bars represent Smith's MMR before the match using ELO F function, with error bars as usual. Note that the bar height is always in relation to opponent's league and division offsets.
The bars are colored by the league of Smith's opponent, and it's also written on X axis.
The dots represent the opponent MMR in relation to Smith's Silver tier offset, which is constant. Error bars as usual.
Written on the X axis are Smith's adjusted points change, followed by opponent's adjusted points change.
What does this graph tell you?
Take a moment and think about it.
To me, it tells this:
![[image loading]](http://s15.postimage.org/d8s53n9fv/graph2.jpg)
Tier 1 being the lowest skilled tier that gets the most bonus points, and tiers being more skilled and given fewer points the higher the tier number.
You can easily get an estimate of tier sizes from this graph. I won't give any numbers just yet, but you can eye gauge it yourself based on the graph.
Immediately noticeable is the (somewhat surprising) observation that there's a big overlap between highest skill tier in Silver and the lower tier in Gold. Even more astounding is that the gap between high Silver and low Platinum is very small, almost swallowing Gold league whole.
Really the only thing I'm struggling with at the moment is how do I reconcile this fact with these numbers (derived from here), which I am now confident we are interpreting wrong, at least for the leagues below Diamond (Diamond is a special case since it is the first league where the league above it only has 1 tier. This could come into effect when the system promotes you, because as soon as your moving average breaks the Diamond-Master barrier you are eligible for a promotion, whereas if you break, say, the Bronze-Silver barrier, the system still has to wait for your moving average to stabilize within a Silver tier before promoting you. This could affect the numbers Blizzard chose to share with us regarding the 'points promotion lines'. I certainly believe that the numbers we calculated from their numbers for Silver-Platinum are a big over estimation of Silver-Platinum leagues sizes).
|
United States12224 Posts
I don't see any Platinum games in the example you provided, but according to the skill distribution curve it makes sense that most players' uncertainty ranges would span the equivalent of a part of Silver, all of Gold, and a part of Platinum. At the time of our Part 2 analysis post, we believed that the promotion criterion was 3*sigma (meaning if 99% of your curve fits within a league boundary, you'll get promoted), but at Blizzcon the systems designer said that wouldn't be a useful measure for league changes because most players' curves will never shrink that much, so they use an exponential moving average instead. And, now that matchmaking has been loosened a bit for the majority of players, such ranges are undoubtedly even more common today.
The "overlap" of Gold and Silver also isn't very surprising, given that we know there's a confidence buffer that works in conjunction with the exponential moving average requirement. Perhaps as a result of determining offsets we'll also find the confidence threshold. I suppose the moving average equation will be the final piece of the puzzle =)
|
Are you taking into account the fact that bronze-diamond leagues will aim for you to have 70-90 adjusted points regardless?
A diamond player with 150 AP won't lose more than 15 points to anyone, event o a true bronze. A platinum player with 75 points won't lose more than 12 points, even against another true bronze. A gold player with -70 AP won't win less than 16 points after a victory, even if against a bronze player.
That said, there is a cap on the numbers (bronze-diamond) that make the data almost useless for us.
And before Excalibur says that he doesn't know if that's true, I rerererere confirmed it, it exists.
|
On May 22 2012 05:20 Excalibur_Z wrote:
I don't see any Platinum games in the example you provided, but according to the skill distribution curve it makes sense that most players' uncertainty ranges would span the equivalent of a part of Silver, all of Gold, and a part of Platinum. At the time of our Part 2 analysis post, we believed that the promotion criterion was 3*sigma (meaning if 99% of your curve fits within a league boundary, you'll get promoted), but at Blizzcon the systems designer said that wouldn't be a useful measure for league changes because most players' curves will never shrink that much, so they use an exponential moving average instead. And, now that matchmaking has been loosened a bit for the majority of players, such ranges are undoubtedly even more common today.
The "overlap" of Gold and Silver also isn't very surprising, given that we know there's a confidence buffer that works in conjunction with the exponential moving average requirement. Perhaps as a result of determining offsets we'll also find the confidence threshold. I suppose the moving average equation will be the final piece of the puzzle =)
We are not talking of a little overlap. The 73 i take as overlap so far is to small.
i think best silver is near EQUAL worst gold! This means leagues are a big scam!
Also i loose the faith that offsets are just numbers you have to add. With this assumption you run into logic mistakes.
Perhaps i do some mistakes and i have to work with inaccuracy,
but i think by now, offsets follow an function that is not as simple as addition!
On May 22 2012 05:50 SDream wrote:
Are you taking into account the fact that bronze-diamond leagues will aim for you to have 70-90 adjusted points regardless?
A diamond player with 150 AP won't lose more than 15 points to anyone, event o a true bronze. A platinum player with 75 points won't lose more than 12 points, even against another true bronze. A gold player with -70 AP won't win less than 16 points after a victory, even if against a bronze player.
That said, there is a cap on the numbers (bronze-diamond) that make the data almost useless for us.
And before Excalibur says that he doesn't know if that's true, I rerererere confirmed it, it exists.
server id league division name adjusted changepoint
US 465568 diamond Overlord Delta 226 -17
i have several thousand games that say otherwise!
Btw guys we are on TL teamspeak very often. I think we are the only 4 guys who really care so much about mmr.
Join us and discuss if you have some time...
|
226 adjuted points that lost 17 points, is that what you are saying?
Because that's exactly my point.
226 - 138* = 88 AP
226 - 172* = 54 AP
And what am I saying?
This player couldn't lose more than 17 points, this was his cap, he would lose 17 points even to the lowest bronze player the system had, because there is a cap here at 70-90 AP for dia-bronze leagues.
* = These numbers come from Elo F Graph.
|
On May 22 2012 06:51 SDream wrote:
226 adjuted points that lost 17 points, is that what you are saying?
Because that's exactly my point.
226 - 138* = 88 AP
226 - 172* = 54 AP
And what am I saying?
This player couldn't lose more than 17 points, this was his cap, he would lose 17 points even to the lowest bronze player the system had, because there is a cap here at 70-90 AP for dia-bronze leagues.
* = These numbers come from Elo F Graph.
i dont get your point
AP = mmr or adjusted points
because we dont call it adjusted after you put the f into it. From there on we call it mmr.
Also thats not his that the mmr of the opponent than.
can you join teamliquid teamspeak?
take this game
League , Adjusted Point , Change Points , MMR , League , Adjusted Points , Change Points , MMR
silver, 226 , 18 , 282 , gold , 270 , -12 , 400
or more in general take my newest datafile:
V 1.7.1 and above
|
On May 22 2012 07:16 skeldark wrote:Show nested quote +On May 22 2012 06:51 SDream wrote:
226 adjuted points that lost 17 points, is that what you are saying?
Because that's exactly my point.
226 - 138* = 88 AP
226 - 172* = 54 AP
And what am I saying?
This player couldn't lose more than 17 points, this was his cap, he would lose 17 points even to the lowest bronze player the system had, because there is a cap here at 70-90 AP for dia-bronze leagues.
* = These numbers come from Elo F Graph. i dont get your point AP = mmr or adjusted points because we dont call it adjusted after you put the f into it. From there on we call it mmr. Also thats not his that the mmr of the opponent than. can you join teamliquid teamspeak? take this game League , Adjusted Point , Change Points , MMR , League , Adjusted Points , Change Points , MMR silver, 226 , 18 , 282 , gold , 270 , -12 , 400
It's easier for me to write/read than speak/hear, but we can PM if you don't want this discussion to continue in this topic.
I won't call it MMR, maybe DMMR (division "mmr"), but definitely not MMR, MMR will take into account the division tiers and represent the whole league in itself, probably going from -1500 to 1500 (or 0 to 3000).
Oh well, I feel that my bad English is making it excessivelly confusing, but I still am 100% sure that what I am trying to say ("meaning") is true.
The game you posted above still agrees with my theory, you won't find a game that doesn't, I need to explain what I mean better somehow...
|
On May 22 2012 07:33 SDream wrote:Show nested quote +On May 22 2012 07:16 skeldark wrote:On May 22 2012 06:51 SDream wrote:
226 adjuted points that lost 17 points, is that what you are saying?
Because that's exactly my point.
226 - 138* = 88 AP
226 - 172* = 54 AP
And what am I saying?
This player couldn't lose more than 17 points, this was his cap, he would lose 17 points even to the lowest bronze player the system had, because there is a cap here at 70-90 AP for dia-bronze leagues.
* = These numbers come from Elo F Graph. i dont get your point AP = mmr or adjusted points because we dont call it adjusted after you put the f into it. From there on we call it mmr. Also thats not his that the mmr of the opponent than. can you join teamliquid teamspeak? take this game League , Adjusted Point , Change Points , MMR , League , Adjusted Points , Change Points , MMR silver, 226 , 18 , 282 , gold , 270 , -12 , 400 It's easier for me to write/read than speak/hear, but we can PM if you don't want this discussion to continue in this topic. I won't call it MMR, maybe DMMR (division "mmr"), but definitely not MMR, MMR will take into account the division tiers and represent the whole league in itself, probably going from -1500 to 1500 (or 0 to 3000). Oh well, I feel that my bad English is making it excessivelly confusing, but I still am 100% sure that what I am trying to say ("meaning") is true. The game you posted above still agrees with my theory, you won't find a game that doesn't, I need to explain what I mean better somehow...
1)This topic is for this kind of discussion!
2) Thats clear that its without tier offset. your right we should give it a diffrent name.Dmmr sounds good.
Joint ts. notthat is here too if you dont have mic we can chat.
|
ok nevermind this message it's been edited beyond recognition. I'll make a new one.
|
I just saw your edit and file.
Your data agrees with me, you don't have there a single person whose "DMMR" is lower than 70 at leagues bronze-diamond.
That's because there is a cap that don't let players from bronze-dia go bellow 70 (probably 85 I'd guess but...).
Your data supports me, now I just have to make you undestand what I am meaning XD
Edit: what that means is that every data for bronze-dia from players with DMMR inferior to 90 can't be taken seriously, it's capped...
|
On May 22 2012 07:41 Not_That wrote:
Is this data point a counter example to what you're saying SDream?
EU, player1, Silver, 40 adjusted points before match, lost 22 points, calculated player2 MMR -382.5 +- 48
player2, Silver, 173 adjusted points before match, won 11, calculated player1 MMR 143.5 +- 14
Maybe come to teamspeak so we could have a more back-and-forth about it? What you're saying is news to us.
edit: even just come to have insta-typed chat capabilities. Forum posts take a while to formulate and we can understand each other quicker live.
EU, player1, Silver, 40 adjusted points before match, lost 22 points, calculated player2 MMR -382.5 +- 48
Yes, that's probably wrong data, he couldn't lose more than 11 points! He probably was promoted/demoted somewhere and you made the math wrong because you didn't take this into account, or something! :D
I will think about teamspeak later =3
|
This is way too confusing
|
|
|
|
|
|
EPT
