SC2 Ladder Analysis: Part 2 - Page 2
Forum Index > SC2 General |
catamorphist
United States297 Posts
| ||
paralleluniverse
4065 Posts
On August 08 2010 11:56 virgozero wrote: yes but thats also very icky because we have no idea how accurate that is and how that differentiates from person to person. I am assuming it is a constant which would assume all players learn @ the same rate which they don't. Sure you can get a general consensus that in 1 week time a player should be X better and therefor we would adjust our system in accordance with X by mutliplying certain varaibles by Y or w/e but it still won't be accurate or anything near accurate. Again I dont quite understand how this can be accurate though, this minimum value? Can you explain a lil more. The way it determines your skill level (MMR) is to move your MMR up and down until it finds a point where you win and lose 50% of your games. This post explains it in great detail for WoW: http://forums.worldofwarcraft.com/thread.html?topicId=14910422788&sid=1&pageNo=3#49 Of course, we would get such an explanation for SC2. | ||
Necrosjef
United Kingdom530 Posts
| ||
Synwave
United States2803 Posts
On August 08 2010 12:08 catamorphist wrote: How did you infer all this? Do you have a lot of mined data sitting around about wins and losses? Good question. I think obviously is the answer but Im curious too. | ||
paralleluniverse
4065 Posts
On August 08 2010 12:13 Synwave wrote: Good question. I think obviously is the answer but Im curious too. From the way the WoW arena system works. | ||
vanick
United States53 Posts
On August 08 2010 12:13 Synwave wrote: Good question. I think obviously is the answer but Im curious too. No mined data. We don't have that for SC2. It is based on what we know about WoW Arena, what we know about SC2 ladder, and how Bayesian inference ranking systems work. This isn't necessarily how it works, though obviously I believe this is close to the truth. I'm sure there are incorrect parts not only of this post, but of the original ladder analysis. This is a theory that tries to describe the behavior we're observing, since the actual ladder mechanics occur within a black box. virgo: paralleluniverse is right. Your MMR is an approximation of your true skill, and it moves according to your performance by winning/losing. How much it moves depends on many factors, as detailed in the original post. | ||
SnakeChomp
Canada125 Posts
On August 08 2010 08:59 Excalibur_Z wrote: Q: So how do bonus points affect all this? Wouldn’t that mess up the rating gains over time? A: Yes, they would. I am not sure how they correct for this. It’s possible that they inflate MMR over time by an equal amount so things still work. It’s possible they correct for the bonus pool instead. I don’t know. I don't think the displayed rating value has any bearing on match making at all. If it did then the bonus pool would make no sense. I'm a freshly placed platinum player and I was winning 42 points on a win (due to my bonus pool) and losing 2 or 4 for losses. Assuming I was going 50/50 for my first few games (I wasn't but that isn't the point), it is an impossibility for my MMR to constantly be sky rocketing after each win and not changing after each loss; otherwise the system would not be an accurate reflection of my performance. No, I believe that the displayed points are purely fluff which is inflated by the bonus pool to make everyone feel as if they are constantly progressing when playing multiplayer. It is used to reflect rank on the ladder, again purely for the sense of progression, but does not and cannot reflect skill. Further consider that the displayed rating cannot be compared across divisions as stated by Blizzard; therefore it makes no sense to use displayed rating to influence the "opponent is favored" display because your opponents can come from divisions other than your own. | ||
virgozero
Canada412 Posts
1.) You play a game 2.) The system assigns you a random MMR with a 50% uncertainty factor 3.) You play more games 4.) The system moves your uncertainty factor up or down depending on how correct the system was in guessing your MMR (if you win vs the ones you should win, the ufactor goes down and if you loose vs the one you should win, the ufactor goes up) If that is rigth so far, I am asking on the fact that between 2-4, you are required to play more games. "4.)" is based upon the games from "3.)" and the analyzation from "2.)". If you have improved since "2.)" then how would that be accurate? Vanick tells me there is this Even if you're playing games back to back this factor will have a minimum value that will still increase sigma. This allows the system to adapt to a player whose skill increases over time This factor will have a minimum value that wil still increase sigma? I dont quite understand that? What is this minimum value and how do they come about it? | ||
Rumpfriction
7 Posts
thanks, Rump | ||
vanick
United States53 Posts
On August 08 2010 12:41 SnakeChomp wrote: I don't think the displayed rating value has any bearing on match making at all. If it did then the bonus pool would make no sense. I'm a freshly placed platinum player and I was winning 42 points on a win (due to my bonus pool) and losing 2 or 4 for losses. Assuming I was going 50/50 for my first few games (I wasn't but that isn't the point), it is an impossibility for my MMR to constantly be sky rocketing after each win and not changing after each loss; otherwise the system would not be an accurate reflection of my performance. No, I believe that the displayed points are purely fluff which is inflated by the bonus pool to make everyone feel as if they are constantly progressing when playing multiplayer. It is used to reflect rank on the ladder, again purely for the sense of progression, but does not and cannot reflect skill. Further consider that the displayed rating cannot be compared across divisions as stated by Blizzard; therefore it makes no sense to use displayed rating to influence the "opponent is favored" display because your opponents can come from divisions other than your own. Excal thought this Q/A was worded confusingly and I said "nooo that's impossible" but it is true I agree with you that displayed rating value has no bearing. The question here arises from my theory on how display rating change is calculated, since it seems that the change in displayed rating comes from the comparison of that rating to the opponent's MMR. I was merely illustrating an unanswered question regarding that. Change in MMR is always purely a result of the players' MMRs and sigmas. I've asked Excal to help clarify this segment. virgo: Please read through the TrueSkill primer and other pages on the site if you're having trouble understanding the post. A lot of the concepts are better explained on the Microsoft Research site. | ||
paralleluniverse
4065 Posts
On August 08 2010 12:46 virgozero wrote: Okay so heres what Im stuck on 1.) You play a game 2.) The system assigns you a random MMR with a 50% uncertainty factor 3.) You play more games 4.) The system moves your uncertainty factor up or down depending on how correct the system was in guessing your MMR (if you win vs the ones you should win, the ufactor goes down and if you loose vs the one you should win, the ufactor goes up) If that is rigth so far, I am asking on the fact that between 2-4, you are required to play more games. "4.)" is based upon the games from "3.)" and the analyzation from "2.)". If you have improved since "2.)" then how would that be accurate? Vanick tells me there is this This factor will have a minimum value that wil still increase sigma? I dont quite understand that? What is this minimum value and how do they come about it? You don't start with a random MMR, you start with an average MMR, i.e. a new player is assume to be better than 50% of players, and worse than 50% of players. This MMR and sigma moves up and down depending on your performance in games. You seem to think that the system guesses a random MMR and tests whether its guess is correct. That's not how it works. The link above to a thread on the WoW forums explains this better. | ||
Reason.SC2
Canada1047 Posts
On a more serious note, thanks for your efforts here, this is quality work. | ||
potatoedoughnut
United States334 Posts
It would be really nice if we had a way to see our MMR & σ, rather than just a league and "rating" which don't mean much. I'm interested in how the bonus pool plays into this. Do you think eventually the available pool points will do down as a player's rating value approaches their MMR? Or will it stay entirely separate from the MMR rating? I don't know how this worked in WoW Arena, or if they will use the system the same way. | ||
virgozero
Canada412 Posts
On August 08 2010 12:57 paralleluniverse wrote: You don't start with a random MMR, you start with an average MMR, i.e. a new player is assume to be better than 50% of players, and worse than 50% of players. This MMR and sigma moves up and down depending on your performance in games. You seem to think that the system guesses a random MMR and tests whether its guess is correct. That's not how it works. The link above to a thread on the WoW forums explains this better. Alright thanks for that clarification. But the problem I am proposing remains there, the system will keep on adjusting your mmr and sigma but it will never reach a point that is accurate or anywhere near accurate of you. Is anyone understanding what I am saying or am I stuck in my own world lol. | ||
paralleluniverse
4065 Posts
On August 08 2010 13:09 potatoedoughnut wrote: Thanks for the summary of this type of matchmaking. It would be really nice if we had a way to see our MMR & σ, rather than just a league and "rating" which don't mean much. I'm interested in how the bonus pool plays into this. Do you think eventually the available pool points will do down as a player's rating value approaches their MMR? Or will it stay entirely separate from the MMR rating? I don't know how this worked in WoW Arena, or if they will use the system the same way. I personally think the bonus pool will inflate everyone's points equally and indefinitely, at least until the ladder resets. On August 08 2010 13:12 virgozero wrote: Alright thanks for that clarification. But the problem I am proposing remains there, the system will keep on adjusting your mmr and sigma but it will never reach a point that is accurate or anywhere near accurate of you. Is anyone understanding what I am saying or am I stuck in my own world lol. The MMR will get more accurate as sigma, which is the uncertainty, decreases. As sigma decreases your MMR is adjusted less after each game, because the system is more certain of your skill. The WoW forum link above has an explicit numerical example of this. | ||
Synwave
United States2803 Posts
Thats a quality OP right there. Fix that stuff for the cherubs OP! or not...I wouldn't know lol | ||
Excalibur_Z
United States12224 Posts
On August 08 2010 12:41 SnakeChomp wrote: I don't think the displayed rating value has any bearing on match making at all. If it did then the bonus pool would make no sense. I'm a freshly placed platinum player and I was winning 42 points on a win (due to my bonus pool) and losing 2 or 4 for losses. Assuming I was going 50/50 for my first few games (I wasn't but that isn't the point), it is an impossibility for my MMR to constantly be sky rocketing after each win and not changing after each loss; otherwise the system would not be an accurate reflection of my performance. No, I believe that the displayed points are purely fluff which is inflated by the bonus pool to make everyone feel as if they are constantly progressing when playing multiplayer. It is used to reflect rank on the ladder, again purely for the sense of progression, but does not and cannot reflect skill. Further consider that the displayed rating cannot be compared across divisions as stated by Blizzard; therefore it makes no sense to use displayed rating to influence the "opponent is favored" display because your opponents can come from divisions other than your own. I edited the second question to make it easier to understand: Q: So how do bonus points affect the display rating changes? If the displayed rating change is based upon the comparison of the opponent's MMR with the player's displayed rating + the player's league cutoff, then wouldn't bonus points inflate the displayed rating and cause problems? A: I'm not sure how they account for this. One possibility is they keep track of bonus points that make up your displayed rating, and ignore them when performing the calculation in the back-end. Excal: It seems more likely that the bonus pool is only used to increase the displayed rating for division ranking purposes and ignored in back-end calculation because the bonus pool increases at the same rate for all players. This introduces a constant that is easily discarded when assessing actual skill within the system. Furthermore, if bonus points were considered in the process of point calculation, it would present an unfair advantage for players who have not yet used up their bonus pool (because their rating is therefore inflated giving them more to lose). Better? | ||
Synwave
United States2803 Posts
You win sir | ||
Sabu113
United States11035 Posts
| ||
Jonoman92
United States9101 Posts
http://research.microsoft.com/en-us/projects/trueskill/details.aspx that is supposed to be laymen? Very surprised xbox uses such a complex system... | ||
| ||