|
Just thought I'd add in my experience. Having just started my second account and knowing that I'm a Diamond player, it took me 34 games to be promoted to Diamond.
My placement matches were TERRIBLE. I played 3 high level Diamonds and 2 people also doing their placements (wtf?). Needless to say, I went 3-2 and was placed in Silver-- I felt disgusted at my luck. But I had faith in the system to put me in the right place. Almost immediately, I was playing and beating 700+ Diamonds (as a Silver, must suck for them). Anyway, I was promoted to Gold around ~14 games, then promoted again to Platinum at 24 games, and finally to Diamond at 34; I was 20-14.
My last 5 matches before promotion went like this:
Me vs 600 Diamond - Loss
Me vs 500 Platinum - Win
Me vs 500 Gold (wtf?) - LOSS!?
Me vs 700 Diamond - Win
Me vs 700 Diamond - Win
Promoted to Diamond.
The points are approximate, but accurate enough. I hope I've helped at least a little-- perhaps I can go back later and get a more detailed match history.
It's been said a couple times in this thread-- honestly, just have faith in the system. Yeah, you might be one of those unfortunate ones that get stuck in Platinum (lol Cauthon), but whatever you do, don't lose on purpose. Yeah, you might need some losses for the system to finally feel comfortable about placing you, but the losses will come eventually, don't cheat yourself and throw games.
My first account hit Diamond almost immediately... but that was near launch. I imagine it must have been significantly easier to achieve that back then?
Regardless, good luck fellas!
edit: Forgot to add that I think I got pretty damn lucky on my points! I still have some bonus pool left! I was #1 Platinum before promoted (it was a brand new division) at around 480pts. When I was placed in Diamond, I had 465pts. I'll take that.
|
United States12240 Posts
That Gold player probably had something like a 17-3 record though. I'm sure that guy will end up in Diamond soon. Thanks for the report, it's too bad there aren't more details though =)
|
I'll fix that when I get back from work today  .
|
This is more detailed then most of the papers I did in college!
|
I have just a couple of small comments to make on the OP to clarify the TruSkill system.
I am a research associate at the University of Saskatchewan. I have corroborated in the past with one of the designers of TruSkill for my own implementation of the TruSkill algorithm for Go ratings purposes.
First, the outcome of a game always decreases both players associated sigmas - both expected outcomes and unlikely outcomes. This is contrary to what is stated in the OP. I see in other posts that you suggest this is the case, but you might want to update the OP to reflect this.
The way uncertainty is added to the system to account for skill drift (e.g. learning) is through another factor (TruSkill calls tau). The process is to simply add (in quadtrature) a constant amount (or an amount based on the amount of time between games) of uncertainty between every game.
|
That may be accurate for TruSkill; however, I have reason to believe that unexpected outcomes in SC2 do NOT decrease the certainty values.
|
Excellent work!
I got promoted to platinum couple of days ago upon losing a match xD
Too bad that same leagues have so much skill discrepancy. I beat Diamond players no problem and they say I should be in Diamond then comes a Platinum or even Gold player and runs me to the ground.
|
On September 02 2010 03:18 LiQuidTalon wrote:
That may be accurate for TruSkill; however, I have reason to believe that unexpected outcomes in SC2 do NOT decrease the certainty values.
If Blizzard increases uncertainty after a unexpected outcome, than they do so contrary to Bayesian Inference and Information theory.
Having sigma always decrease after an outcome isn't a flaw in the way TruSkill works. Consider that each game outcome is a piece of information and the sum of all player sigmas is the degree to which you are uncertain about the system as a whole. How can a new piece of information make you less certain about the system as a whole. This is irrational. New information can only make you *more* certain about the system as a whole, otherwise it wouldn't be information.
The correct way to account for skill changes is the way TruSkill works: to add in quadrature a best estimate of skill volatility between every game. The point of contention becomes how much volatility to add. Add to much, and the system remains uncertain even about stable players. Add to little and the system reacts slowly to quickly increasing/decreasing players. It is a trade-off.
I imagine Blizzard like the TruSkill implementation I am familiar with, use a global variable as it would be quite computationally intensive to try and come up with a time dependent player dependent variable. Perceived flaws in TruSkill and systems like this probably come about because players skill increases too fast for the system to account for; but the system designers have to think of the whole and not of a small percentage of players.
I guess my suggestion to the OP would be to describe the TruSkill algorithm as accurately as possible as it is probably the best starting point to understanding Blizzard's hidden algorithm, then explain how he thinks Blizzard's system might deviate from this based on collected evidence.
I also think Blizzard should publish their algorithms (at least a paper on the mathematics not necessarily the codebase) and make the underlying match making variables known --- not just the semi-useless points they have now.
|
United States12240 Posts
I think there are definite differences between Blizzard's SC2 and WoW Arena systems and TrueSkill. We borrow some concepts from TrueSkill because the information is published, but we're not saying that Blizzard uses an implementation of TrueSkill. In WoW Arena, where the MMR is visible, the change of MMR increases during a streak. Vanick says that means that either sigma increases as we've argued here, or the skill drift factor is per-user, is updatable, and is functionally very similar. Either way, the end result in rating update is the same (in the theoretical Blizzard system) and there's no way for us to make a distinction because the information isn't readily available. Sigma increasing is not impossible in TrueSkill, and to my knowledge is not contrary to Bayesian inference.
|
I have read your ladder analysis and I must say it explained a lot to me. I also played WoW Arena couple years ago, 2150 rating!, so it felt all familiar. MMR is the main ingredient in your summary. But I have a problem with your assumption between MMR and the public ladder.
Let me start with these 2 statements.
(1) When you promote you lose points.
(2) I started in Bronze league, 100 games later I find myself in Platinum league. Also I have been playing against Platinum players while being in Gold.
My problem:
In the 100 games I played I almost always played against slightly more favored people. Except when I was topping my division and promotion was near. According to you this indicates that my public rating is trying to get higher to get closer to my MMR
Now with promoting from Gold to Platinum I lost a bit more than 200 points. But this doesn’t make sense if there is such a big correlation with your MMR. I know that everybody loses points when they promote. It is not about my QQing over my points. But I like to understand why.
When I -as gold player- played against a platinum player than this player was always (slightly) favored. This means that my public rating is lower as the average platinum player his hidden MMR. In your analysis you assume that MRR and Public rating try to reach the same amount. You also assume that your MMR needs to exceed a certain number to qualify for promotion. One of these 2 assumptions is wrong and it seems to be the first. Because it doesn’t make sense that I and others lose points when gaining promotion if obviously your Public rating is lower than the required MMR for platinum.
Unless there is something I am overlooking. Your paragraph about displayed rating is incorrect. You are probably close to the truth but not on it.
PS: Does any1 know if you gain points when you demote? If it is a fixed number of points you lose when you promote or a percentage?
|
lol i wasnt able to comprehend this post but i realise it was alot of effort and judging from the great feedback others have given you seem to have hit the nail on the head, so well done sir.
|
United States12240 Posts
On September 02 2010 22:48 Koshi wrote:
I have read your ladder analysis and I must say it explained a lot to me. I also played WoW Arena couple years ago, 2150 rating!, so it felt all familiar. MMR is the main ingredient in your summary. But I have a problem with your assumption between MMR and the public ladder.
Let me start with these 2 statements.
(1) When you promote you lose points.
(2) I started in Bronze league, 100 games later I find myself in Platinum league. Also I have been playing against Platinum players while being in Gold.
My problem:
In the 100 games I played I almost always played against slightly more favored people. Except when I was topping my division and promotion was near. According to you this indicates that my public rating is trying to get higher to get closer to my MMR
Now with promoting from Gold to Platinum I lost a bit more than 200 points. But this doesn’t make sense if there is such a big correlation with your MMR. I know that everybody loses points when they promote. It is not about my QQing over my points. But I like to understand why.
When I -as gold player- played against a platinum player than this player was always (slightly) favored. This means that my public rating is lower as the average platinum player his hidden MMR. In your analysis you assume that MRR and Public rating try to reach the same amount. You also assume that your MMR needs to exceed a certain number to qualify for promotion. One of these 2 assumptions is wrong and it seems to be the first. Because it doesn’t make sense that I and others lose points when gaining promotion if obviously your Public rating is lower than the required MMR for platinum.
Unless there is something I am overlooking. Your paragraph about displayed rating is incorrect. You are probably close to the truth but not on it.
PS: Does any1 know if you gain points when you demote? If it is a fixed number of points you lose when you promote or a percentage?
There needs to be some correlation between MMR and displayed rating or else the displayed rating is meaningless and things start to break down. The only disconnect we see between displayed rating and MMR that still keeps things consistent is the bonus pool, which inflates displayed rating for everyone at the same rate. For this reason, we believe any points earned through the bonus pool are ignored when determining favored status. I believe the maximum bonus pool at the time of this post is somewhere around 560.
As far as losing points when you're promoted, we're aware of that but just don't have enough data to explain it. Earlier in this thread, in one of the later pages, I posted one possible explanation offered by someone on the Bnet forums, where point gains and losses are worth more or less based on the league of your opponent. For example, points earned against Diamond players would be worth twice as much as points earned from Platinum players, but points lost to Diamond players would cost you only half as much as points lost to Platinum players. It may be possible that this is how it works when translating your Platinum points to Diamond points at the time of promotion, but until we see more full match histories like the one I made on page 8, we don't have enough concrete data to draw any firm conclusions.
|
I think that point loss is there to prevent abuse. Since you can't get below 0 in your rating (correct me if I am wrong), you could lose a lot to pull your MMR down, and then climb up the ladder by having "free wins" until you reach your level. Now if the theory of point loss mentioned by Excalibur_Z is correct, the point you gain at bronze are worth 1/2^4 "diamond' points. so the free wins you get don't really get you much diamond points.
|
On September 02 2010 04:48 CaeZaR wrote:
The correct way to account for skill changes is the way TruSkill works: to add in quadrature a best estimate of skill volatility between every game. The point of contention becomes how much volatility to add. Add to much, and the system remains uncertain even about stable players. Add to little and the system reacts slowly to quickly increasing/decreasing players. It is a trade-off.
Yes, but with this system, the sigma can increase even with true skill. Consider this: you add a uncertainty to the sigma, and you win according to what is expected (like when you have a winning streak). Then your sigma will not change much, but since there was uncertainty added, it will increase. Cf:
this example
This makes sense from an information theory point of view: the quantity you are trying to measure (skill) is not fixed, so the quantity of information you have about it may not increase (for an extreme example: consider you switch one player for the other, then you have 0 information on the new player).
|
United States12240 Posts
On September 03 2010 05:00 gondolin wrote:
I think that point loss is there to prevent abuse. Since you can't get below 0 in your rating (correct me if I am wrong), you could lose a lot to pull your MMR down, and then climb up the ladder by having "free wins" until you reach your level. Now if the theory of point loss mentioned by Excalibur_Z is correct, the point you gain at bronze are worth 1/2^4 "diamond' points. so the free wins you get don't really get you much diamond points.
That's possible, but I'm really hesitant to add that to the original post as fact until we get some more data. I asked Vanick to bomb all his 1v1 placement games and tank his MMR, so we'll see when he finally gets promoted whether Angstrom's point translation theory is correct, as well as how quickly his MMR climbs.
I encourage everyone who hasn't played in a particular bracket to start tracking their matches from the beginning, too, and to post full histories in this thread so we can work with more data and figure this thing out.
|
Excalibur. Fantastic work.
Albeit my match history is a little botched here it is.
http://www.sendspace.com/file/d0d39y
Currently I'm Gold Rank 1 at 901 points. I only recently started adding the points of my most recent opponents. And I regret not throwing in where my promotions were (as I started out in Bronze).
I'll make another post when I manage to reach diamond glhf
|
United States12240 Posts
On September 03 2010 15:37 pezyuan wrote:Excalibur. Fantastic work. Albeit my match history is a little botched here it is. http://www.sendspace.com/file/d0d39yCurrently I'm Gold Rank 1 at 901 points. I only recently started adding the points of my most recent opponents. And I regret not throwing in where my promotions were (as I started out in Bronze). I'll make another post when I manage to reach diamond glhf
No offense, but could you paste the contents of the spreadsheet into this thread, or convert it to txt or something? I don't have Office on my home computer and I'm wary of opening strange xls's on my work computer (particularly since I have macros enabled there).
|
No worries. Will this work okay?
Map Result Race League / Rank Rank Rating Points
Blistering Sands W T Platinum 6
Metalopolis L P Forfeit Forfeit
Blistering Sands L P Forfeit Forfeit
Stepps of War L P Forfeit Forfeit
Metalopolis L P Forfeit Forfeit
Kulas Ravine W Z Bronze 33
Desert Oasis W P Silver 7
Kulas Ravine W Z Gold 24
Kulas Ravine W Z Gold 10
Lost Temple W Z Platinum 47
Desert Oasis W P Platinum 24
Blistering Sands W P Platinum 6
Desert Oasis W P Diamond 14
Stepps of War W T Diamond 92
Xel'Naga Caverns L P Platinum 17
Scrap Station W Z Diamond 49
Delta Quadrant W P Platinum 52
Kulas Ravine W P Diamond 75
Scrap Station W Z Diamond 59
Blistering Sands W P Diamond 58
Kulas Ravine W T Diamond 46
Metalopolis W Z Diamond 30
Lost Temple W Z Platinum 79
Kulas Ravine W Z Platinum 12
Stepps of War W P Diamond 67
Metalopolis L Z Diamond 11
Lost Temple W T Diamond 50
Delta Quadrant W T Platinum 23
Blistering Sands L T Diamond 27
Scrap Station W T Match not Found (Suspect Dia)
Lost Temple L P Diamond 9
Delta Quadrant L Z Diamond 12
Scrap Station W P Diamond 11
Metalopolis W Z Diamond 26
Blistering Sands W P Diamond 16
Delta Quadrant L P Diamond 9
Stepps of War L T Diamond 1 976
Metalopolis L T Diamond 12 852
Delta Quadrant W Z Diamond 21 808
Lost Temple W P Diamond 44 349
Xel'Naga Caverns L P Diamond 2 901
Metalopolis W P Diamond 1 908
Metalopolis W Z Diamond 40 895
Stepps of War L T Diamond 4 976
Metalopolis W Z Diamond 41 538
|
Q: Would it take longer to get promoted if you've played lots of games? Assuming someone played a large amount of games (say a 100 with a 50% win/loss ratio). If he were to start winning 70% of his games, would it be harder for him to get promoted than someone with similar percentages but fewer games played?
A: It would take longer, yes. If you've played 100 games and gone 50-50, your sigma is probably fairly small because the system feels confident that it's put you where you belong. If someone else has played 16 games and gone 8-8, that person's sigma is going to be larger. The exact scale is something that we don't know, but we do know that your MMR never truly gets "locked" in place (it's always changing to some degree after each win or loss). Depending on where you are in the ladder, you may need to play quite a few more games to increase your sigma before you can decrease it again within the threshold of a higher league, which would make you eligible for promotion.
So that makes demotion kind of an everlasting curse, then? By the time you've built your skill up to where you belong back in the division you were demoted from, you have a lot more games under your belt, and the ladder's going to be a real hardass about promoting you. What effect would having most of your wins at the end of your career have on this situation?
I placed in silver and got demoted because I kept getting fucking dropped from BNet and it counts as a loss whenever that happens. So I was never really bad enough to be bronze, yet here I am, sitting here in the top 5 of my division forever with like a 15 game win streak. Are all my promotions going to be like this because of a fluke demotion fucking with my uncertainty factor or whatever?
|
United States12240 Posts
Unfortunately, without the opponent rating information, opponent win-loss record, promotion markers, and the amount of points gained and lost per match, there's not too much we can do with this information. The ranks are determined by rating, so it's the rating that we really would need to see. The amount of points gained/lost helps determine how close you are to your MMR. The opponent win-loss record helps identify whether they may be close to their MMR (and therefore validates their rating). For example:
L Diamond 200 20-6 (-4)
That alone doesn't tell us very much, but we can get some info from it. With a 20-6 record, the chances that this guy actually belongs at 200 rating is very low. He was probably recently promoted. The -4 from the loss tells us that he was Favored, which reinforces that his MMR is high (and at the very least, higher than our current rating). Now if we added another game:
W Diamond 600 70-56 (+15)
This, combined with the previous game, tells us that the guy's 600 rating is a little more accurate than the other guy's 200 rating because he has more games played. It also tells us that his MMR is a little closer to our displayed rating because he was only slightly favored. That likely means that our MMR is closer to that area, but we don't know for sure until we see more games.
Now, if we take a look at your last 9 games, we can still get a fair amount of info from them. We don't know how concrete these players' ratings are because we can't see their records, but we can at least assume that if they're in the 900 range that they must be pretty good (inflation is about 570 or so at this point in time). The outliers here are the guy with 349 and the guy with 538, so if you're being matched against them it probably means their MMRs are around that of the 900-level players but they just haven't played enough games to get up that high.
I'd say that a promotion to Diamond is inevitable, but it may depend on the variance in opponents shrinking first. If you're holding your own against 900 Diamond players, you'll just have to even out a bit more to decrease your uncertainty value which will allow promotion.
|
|
|
|
|
|
EPT
