EPTSC2 Ladder Analysis: Part 2 - Page 16
| Forum Index > SC2 General |
|
Matrim
United Kingdom16 Posts
| ||
|
BenKen
United States860 Posts
On October 15 2010 23:34 ashaman771 wrote: Great OP. But at some point, which you learn after university, you have to be able to digest the information to present it to people in a clear way. Essentially, can you take the information you have and explain it to someone in high school. Again, great OP, but I suspect it's from someone green in university. You mean like this? http://www.teamliquid.net/forum/viewmessage.php?topic_id=150367 Also, no offense, but there are better ways to make suggestions than back-handed "oh you must be young, you'll learn" insults. | ||
Excalibur_Z
United States12224 Posts
On October 15 2010 23:34 ashaman771 wrote: Great OP. But at some point, which you learn after university, you have to be able to digest the information to present it to people in a clear way. Essentially, can you take the information you have and explain it to someone in high school. Again, great OP, but I suspect it's from someone green in university. It isn't. Vanick is a college graduate with a CS degree, and we've gone through several phases of proofreading and editing before posting this. I don't have a formal college education on statistics since I went to a trade school, so over those passes we made it more accessible and easier to understand (it used to be a lot more verbose than this). Not to sound condescending, but we've had a lot of comments praising this post because of how thorough and straightforward the explanations are. To quote a poster from SomethingAwful where this thread was linked, "That poster has a fucking spectacular ability to communicate, holy shit." | ||
|
Excalibur_Z
United States12224 Posts
On October 16 2010 00:13 Mendelfist wrote: When I post this total available bonus points are exactly 1120 (for EU, I don't know if other regions are in sync with this). Those interested can keep track of it either by looking at how your own bonus pool increases and add it to this number, or by just counting hours from this moment (1 point per 2 hours). Utilised bonus pool is then of course total available bonus pool minus remaining bonus pool. As far as I know this works even after promotions/demotions. Correct me if I'm wrong. Yeah, that's what I've noticed too. However, there was one notable exception (so it's probably a bug). When I got promoted from Platinum to Diamond, my last game earned me +20 bonus points, but -40 was deducted from my bonus pool. I saw this in someone else's match history when being promoted to Diamond too, and it was the same phenomenon: earning 20 but deducting 40. Every other match history parse I've seen has kept the bonus pool total consistent. My guess is it's just a Diamond-promotion double-deduction bug. | ||
|
iDreamStar
United States2 Posts
I am a first time poster however I have been reading carefully all the threads relating to Analysis of the SC2 Ladder. I am a bronze player and though this might been insignificant to most of you I am still confused as to why I still remain in Bronze league. I started off 1-4 due to some disconnections and continued to a dismal record of 6-19 until I started to step my game up watch some Day9 and read some TL. Now in my past 13 or so games I have been playing Top Bronze with Top Silver have won all but 2 games and a couple hours ago I defeated a rank 44 platinum that was "even" with me. Now I know that being "even" is just the displayed ratings being compared however if anyone could answer my question as to why I remain in bronze league it would be appreciative. Link to Profile: http://sc2ranks.com/us/889656/iDreamStar Thanks  | ||
|
pGa.Ghu)Z(dan
Germany2 Posts
| ||
|
mDuo13
United States307 Posts
On October 16 2010 11:47 iDreamStar wrote: Hello, I am a first time poster however I have been reading carefully all the threads relating to Analysis of the SC2 Ladder. I am a bronze player and though this might been insignificant to most of you I am still confused as to why I still remain in Bronze league. I started off 1-4 due to some disconnections and continued to a dismal record of 6-19 until I started to step my game up watch some Day9 and read some TL. Now in my past 13 or so games I have been playing Top Bronze with Top Silver have won all but 2 games and a couple hours ago I defeated a rank 44 platinum that was "even" with me. Now I know that being "even" is just the displayed ratings being compared however if anyone could answer my question as to why I remain in bronze league it would be appreciative. Link to Profile: http://sc2ranks.com/us/889656/iDreamStar Thanks Simple: You have now confused the system significantly, and you'll get promoted when it confidently establishes which league you belong in. In more technical terms, your sigma is very high from causing so many upsets and your MMR is likely fluctuating significantly as you play players with such varied MMRs. You will get promoted when your MMR +/- sigma falls cleanly in a particular league range. | ||
|
Impaler
United States11 Posts
Sigma should be split into two separate Sigmas a high and a low. Currently the system MUST create a symmetrical bell curve around ones MMR which represents a total probability of expected performance. If you win against a higher ranked player the MMR goes up but so dose the single Sigma which causes the lower end 'tail' of your performance expectation to stagnate. Thus a players Performance curve just keeps getting 'smeared' upward and the system won't promote because of the high confidence threshold needed. So for example Bob a Platinum with 2000 MMR plays against Jon a Diamond player with 2300 MMR. Jon is favored but Bob wins an upset victory. The systems single sigma means the system is forced to basically say "Oh I seem to have Bobs skill level wrong, he might actually be Diamond material or he could equally likely be Gold caliber." Now obviously any inference that Bob win means he is more likely to Gold caliber should not be drawn from a win against a higher ranked opponent. While the upper end of the performance curve should indeed go up because your win means you might really be that good, it should indicate to the system that your not going to perform poorly. An upset win would incresse only the Upper-Sigma, while an upset loss would increase only the Lower-Sigma, on the flip-side an expected win decreases Lower-Sigma and and an Expected loss decreases Upper-Sigma, all of course scaled by the degree to which one is favored. In each case the system ONLY takes the logical inference of the result of a game and not an illogical 'mirror' inference on the other side of the curve. I find it rather silly that such an obvious and easily fixed flaw in the system exists. | ||
|
Excalibur_Z
United States12224 Posts
On October 20 2010 07:27 Impaler wrote: While reading the article on Microsoft True Skill a thought occurs to me. Sigma should be split into two separate Sigmas a high and a low. Currently the system MUST create a symmetrical bell curve around ones MMR which represents a total probability of expected performance. If you win against a higher ranked player the MMR goes up but so dose the single Sigma which causes the lower end 'tail' of your performance expectation to stagnate. Thus a players Performance curve just keeps getting 'smeared' upward and the system won't promote because of the high confidence threshold needed. So for example Bob a Platinum with 2000 MMR plays against Jon a Diamond player with 2300 MMR. Jon is favored but Bob wins an upset victory. The systems single sigma means the system is forced to basically say "Oh I seem to have Bobs skill level wrong, he might actually be Diamond material or he could equally likely be Gold caliber." Now obviously any inference that Bob win means he is more likely to Gold caliber should not be drawn from a win against a higher ranked opponent. While the upper end of the performance curve should indeed go up because your win means you might really be that good, it should indicate to the system that your not going to perform poorly. An upset win would incresse only the Upper-Sigma, while an upset loss would increase only the Lower-Sigma, on the flip-side an expected win decreases Lower-Sigma and and an Expected loss decreases Upper-Sigma, all of course scaled by the degree to which one is favored. In each case the system ONLY takes the logical inference of the result of a game and not an illogical 'mirror' inference on the other side of the curve. I find it rather silly that such an obvious and easily fixed flaw in the system exists. I don't think you have it right. Say Bob's curve range is 1500-2500 and he beats Jon who's at 2300 and it's an upset and sigma increases. Bob's new range doesn't become 1450-2550. It would move more toward the right because his MMR increased, so the new range would be more like 1525-2625. | ||
|
Impaler
United States11 Posts
Lets take your your example and construct a scenario to demonstrate my point, Bob's initial MMR is 2000 with sigma spread of 500 on each side. After the upset victory the MMR is increased to 2075 with a sigma spread of 550 on each side for the new 1525-2625 spread. The low end tail increased only 25 points. This seems reasonable for a single upset game but if the player continues being placed against opponents 300 MMR above them (cause the system is trying to bring your win percentage down like it should). Each win will add 75 points to Bobs MMR but add 50 to the sigma spread on each side resulting in the tail end of their expected range increasing by only 25 points and the upper end by 125 points. After three such matches Bobs MMR would be 2225 but the spread would be a whopping 1625-2875, Now Bob has just beaten a 2300, 2375 and 2450 player in that order so the upper bound is not unreasonable, Bob might really be that good and until he starts to lose matches against high ranked players that upper bound should be very fuzzy. But the low end estimate is really quite absurd after that strong a performance, Bob has demonstrated play consistently above that level, if Bob was to choke on a few games and lose to a 2000 or lower player then that low end estimate is reasonable but not after repeated strong performance. Imagine instead two separate sigma spreads, which are initially each 500 around the 2000 mean. As Bob wins upset victories the MMR moves in the same manor but only the upper sigma increases by 50 rather then both, Now the lower bound is moving up in sync with the MMR and the upper is moving up just as it was before. Now after the same matches the MMR is 2225 the lower sigma spread is still unchanged at 500 but the upper is elongated to 650 for a total spread of 1725-2875 which is a more reasonable spread given the matches that have taken place. Another interesting side effect of this dual-sigma approach is an ability to actually capture a players likely-hood of choking. Normal True-Skill makes the in my opinion false assumption that an individuals has an equal probability of deviating above and below their average performance. In reality is is FAR more likely for performance in any skill to fall dramatically aka 'choke' then it is to proportionally over-perform. Take a runner for example who normally runs a mile in 5 minutes, it's far more likely that one day he has a bruised hamstring and dose it in 6 then it is for them to out of the blue run a 4 minute mile. And the higher the level of ones average performance the more easy it is to choke (the smallest error will do it) and the harder it is to over-perform (so many factors would have to be elevated and you may be near the maximum humanly possible performance). With a dual-sigma approach you would expect to see a larger low end sigma on nearly all high end players because of occasional chokes, players who consistently never choke will have this reflected in a narrower low-end sigma and this could significantly improve the accuracy of predictions made under this system. | ||
|
Excalibur_Z
United States12224 Posts
I mean, it's entirely possible that a two sigma system exists, and you make good points for how it could address the shortcomings of the TrueSkill system. However, a bell curve is more computationally efficient and is also accurate enough for the purposes of estimating skill. It doesn't seem as likely that player curves would have two peaks. | ||
|
Impaler
United States11 Posts
Also the distribution wouldn't be two peaked, it would be a skewed distribution. http://en.wikipedia.org/wiki/Skewness http://en.wikipedia.org/wiki/Skew_normal_distribution The math is way over my head so the best lay description I can give is that of two sigmas one above the average and one below. In reality it seems a skewed normal distribution uses a third factor to express the skew with 0 being the plain symmetrical Gaussian. In that case upset wins increases skew and upset losses decrease skew, while an expected win or loss dose not change skew (or at least changes it less). It would indeed make the math even more complex (perhaps unnecessarily) but I'm sure people though the same thing about expanding on ELO, I'll do some more searching to see if this idea has ever been explored (it would be hard to imagine that it hasn't been explored by Vegas bookies) | ||
|
Mendelfist
Sweden356 Posts
On October 16 2010 02:08 Excalibur_Z wrote: Yeah, that's what I've noticed too. However, there was one notable exception (so it's probably a bug). When I got promoted from Platinum to Diamond, my last game earned me +20 bonus points, but -40 was deducted from my bonus pool. I saw this in someone else's match history when being promoted to Diamond too, and it was the same phenomenon: earning 20 but deducting 40. Every other match history parse I've seen has kept the bonus pool total consistent. My guess is it's just a Diamond-promotion double-deduction bug. Found another bug, and this time it was not after a promotion: My points before match: 959. Spent bonus before match: 886 Points after match (loss): 978. Spent bonus after match: 902 The score screen and my match history says I lost 13 points, but my score increased 19 points. This makes no sense at first, but the difference is 32 points, and 16 points were deducted from the bonus pool. I suspect that the system thinks that I both won and lost the match at the same time. -13 for losing and 16 + 16 for winning. Curious. And annoying if you are trying to keep track of everything and make sense of it. My match history is here: http://www.lysator.liu.se/~john/history.html | ||
|
Ownos
United States2147 Posts
 Nice they confirm that this is all pretty close. Good job! They also mentioned something very important in that your ranking/points(?) is relative to the skill of your division. Not sure on the exact wording there.That point alone makes things 10000% more complicated? If it's comparing your performance vs the 99 others in your division. | ||
|
Ryalnos
United States1946 Posts
On October 15 2010 23:34 ashaman771 wrote: Great OP. But at some point, which you learn after university, you have to be able to digest the information to present it to people in a clear way. Essentially, can you take the information you have and explain it to someone in high school. Again, great OP, but I suspect it's from someone green in university. I'm flabbergasted by this post. Has this guy 'been to university'? Clear presentation is not something you magically learn there. And what's the point of speculating on someone's level of higher education? If I were to speculate myself, I might think that this is a high school kid who can't understand the really not too sophisticated math in the OP... Anyway, well done OP, and congrats on that MVP status on the blizzard forums. It is a role for the brave and patient, I imagine. | ||
|
conqueso
United States22 Posts
My elo might be inflated because of a large bonus pool, I recently switched races. After the bonus pool i often come out ahead in visible rating, even though my record is quickly approaching 50/50. | ||
|
Excalibur_Z
United States12224 Posts
The other concept that he introduced was a moving average which has a similar function as sigma. Basically, if you were to track player skill game by game, it would have a ton of sharp peaks and deep valleys. Blizzard chooses to use a moving average to slowly gauge where you belong. Once your moving average crosses a certain threshold (some kind of confidence buffer), that's when you get promoted. This means that if you bomb your initial placement matches and go down into Bronze, then rapidly improve to Diamond level, it will take a long time for your moving average to cross into Diamond level and cement that level of confidence for a promotion. We believe that the moving average only covers your last X games (maybe 100 for example) otherwise players with 4000+ games played would never get out of their league. I'll be making more corrections to the original post later on today or tomorrow. | ||
|
lololol
5198 Posts
Copied from another thread on TL: by Benzenn 18 Mar 2010, 19:43 Sorry I misunderstood what the OP was referring to. I didn't mean to imply that one division is ranked better than the other, but simply explaining the basics of divisions. As far as comparison across divisions it's certainly something we've considered but there are issues, such that the rankings in one division don't directly translate to the other divisions. So you couldn't compare division 10 to division 48 and compare one player's points to another. | ||
|
Happy Frog
Australia490 Posts
On October 25 2010 03:12 Excalibur_Z wrote: So after the Multiplayer panel yesterday, we asked the Doc some more specific questions. There are some things that we're not sure about now. We now know that divisions are not equal which adds a great deal of confusion because sites like SC2Ranks are specifically designed to ignore division weighting. It seems like they've gone out of their way to put emphasis on your own division rather than your league ranking, which sort of has a side effect of screwing up global point rankings like SC2Ranks. The other concept that he introduced was a moving average which has a similar function as sigma. Basically, if you were to track player skill game by game, it would have a ton of sharp peaks and deep valleys. Blizzard chooses to use a moving average to slowly gauge where you belong. Once your moving average crosses a certain threshold (some kind of confidence buffer), that's when you get promoted. This means that if you bomb your initial placement matches and go down into Bronze, then rapidly improve to Diamond level, it will take a long time for your moving average to cross into Diamond level and cement that level of confidence for a promotion. We believe that the moving average only covers your last X games (maybe 100 for example) otherwise players with 4000+ games played would never get out of their league. I'll be making more corrections to the original post later on today or tomorrow. Thanks for all your work, and nice sleuth work at Blizzcon. I had a question for you; I'm not sure if you saw the Day 1 SC panel but Greg Canessa stated that Bronze / Silver / Gold / Plat / Diamond were designed to be evenly distributed at 20% of the servers player base each, obviously this doesn't match up with data from Sc2ranks which looks more like top 7% in Diamond and bottom 50% in Bronze. Any theories on this? A symptom of inactive players perhaps? | ||
|
Excalibur_Z
United States12224 Posts
On October 25 2010 03:41 lololol wrote: That the divisons are not equal has been known since early beta. Copied from another thread on TL: We figured that something changed because it didn't make sense (and still doesn't make sense) to set it up that way. Now that we have official confirmation of that it throws a wrench in sites like SC2Ranks. We sort of had a sinking feeling that was how it behaved but ugh... silly that it does. He did mention that when he pulls the Top 200 lists for the week that they're purely by points, only without the division weighting. | ||
| ||
