|
A short note/possible clarification:
On August 08 2010 08:59 Excalibur_Z wrote:
Note that this figure is taken from a TrueSkill presentation, and is copyright Microsoft. TrueSkill incorporates the possibility of a draw. More intuitively, it can be thought of as the “matchmaking sweet spot”, and something similar is likely used by SC2’s ladder to provide the system some wiggle room in matchmaking. When the system says “Expanding Search…” it is probably expanding this green stripe.
The bolded part doesn't really make sense, with reference to the graph - the green area is a function of the two MMR/sigma values, and cannot be "expanded" by the system without modifying those values (which it is not, obviously, doing).
However, in functional terms the description is correct. What the system is trying to do, initially, is to find a player whose MMR/sigma meshes with yours to make the likelihood of each player winning rather equal. In terms of the graph shown above what I quoted, the system is trying to pick a player with whom you will form a graph that is roughly (for instance) 45% red, 10% green, and 45% blue (although given how draws are impossible, more like 50/50 red/blue).
Obviously, this is impossible if certain players are not in the matchmaking queue. So, presumably, the system has some kind of threshold in which a game is "acceptable" as long as the balance is (again, an arbitrary pick) 40/60 (or 60/40).
In the interests of speed, however, the system will expand this threshold so that people can at least find a game in a reasonable timeframe - so, as the system expands a search, the likelihood of a wildly unbalanced game increases.
That wasn't as short as I expected it to be :x
|
Totally awesome post, excal. Really well explained. I do have one question though; do you know precisely how the MMR change is calculated after each match? In case I'm not being clear enough, what numbers does the system use to determine the increase or decrease of a player's MMR after each game? I scanned that WOW forum thread and couldn't see an obvious answer and I'm too rusty at maths(not to mention lazy) to figure it out for myself. Perhaps knowing exactly how this number changes over time might shed some light on the eternal platinum bug.
|
On August 08 2010 14:12 brad drac wrote:
Totally awesome post, excal. Really well explained. I do have one question though; do you know precisely how the MMR change is calculated after each match? In case I'm not being clear enough, what numbers does the system use to determine the increase or decrease of a player's MMR after each game? I scanned that WOW forum thread and couldn't see an obvious answer and I'm too rusty at maths(not to mention lazy) to figure it out for myself. Perhaps knowing exactly how this number changes over time might shed some light on the eternal platinum bug.
That would be impossible to know because MMR is hidden so we can't use regression to find a formula, nor will Blizzard tell us the formula.
It's probably like the formulas used for xbox true skill with modification for league and bonus pool.
|
On August 08 2010 14:12 brad drac wrote:
Totally awesome post, excal. Really well explained. I do have one question though; do you know precisely how the MMR change is calculated after each match? In case I'm not being clear enough, what numbers does the system use to determine the increase or decrease of a player's MMR after each game? I scanned that WOW forum thread and couldn't see an obvious answer and I'm too rusty at maths(not to mention lazy) to figure it out for myself. Perhaps knowing exactly how this number changes over time might shed some light on the eternal platinum bug.
I'm fairly certain the formula for MMR change has never been determined or released. The old arena system used a zero-sum system which made it quite easy to determine the pre-match ratings from the rating change post-match, but for some reason that now escapes me Blizzard decided to keep the formula for the new system hidden.
One possible reason for the eternal platinum thing is that TrueSkill-esque systems (at least in some implementations) will severely overweight early performances. For instance, if you played your first 50 games against players who had lower MMRs than you, (at least by my understanding) your MMR would never actually change - only your sigma value would change. This sigma value change would make it much harder for the system to later increase your MMR rapidly in response to wins against players rated higher than you.
I know in Global Agenda they used a TrueSkill based system to determine "combat ratings" for players, and for the most part people had a tremendously difficult time getting the highest rating on their first character because they'd "ruined" their hidden rating when they were learning the game.
|
kzn - you're right, that is inaccurate. It can be useful to think of how the system is expanding the search space by visualizing it with the green stripe starting at x=y (plus the draw probability) and slowly widening outwards which would allow for worse and worse matches. I must have got some wires crossed while writing.
edit: TrueSkill has some drawbacks for sure. Coping with rapid skill change in a player is a major one. I do think Blizzard has incorporated refinements to their system.
|
Also, since I'm something of a WoW Arena expert, I'll try to add to this section.
Displayed Rating
Ok, how does all of this tie into displayed rating and the whole “favored” deal? If you remember back to WoW, ratings changed based on a direct comparison of your displayed rating to the other team’s MMR. So if your current rating was 500 and you were playing people with MMRs of 2000, your rating would jump significantly after every win because of the wide disparity. Now, we’ve identified that on the loading screen quite often players are seeing the other person as favored and the opponent (who is nominally “favored”) also sees his opponent as favored! How can this be? The theory put forth here is the system is again comparing your displayed rating to your opponent’s hidden MMR.
This is not, I believe, strictly correct as a description of how WoW's system worked. As it was explained when MMR was first introduced, there was actually no interaction between your team's rating and the opposing team's MMR.
How it works was like this: Say you've got a MMR of 2500, and you start a new team. It starts at 0 rating, but the matchmaking system will match you with other players of MMR 2500. If you lose a game, your team rating would not change at all. If you won, it would increase by 47 (a hard cap that was in place at least when I played). This was not explained as arising due to an interaction between the team rating and the opponent's MMR, however - it was explained as the system trying to get your team's rating as close as possible to your team's MMR rapidly. This is perhaps accomplished by a system such as that in the quote, but I'm not sure thats actually whats going on.
However, in SC2 we do not see the same sort of drastic jumps every time we beat a “favored” opponent. If we’re playing diamond-level players then their MMR must be way higher than our displayed rating, right? Or more to the point, if you were to lose and compare your small displayed rating to your opponent’s MMR, you would lose almost no points and clearly that is not how SC2 behaves.
In my experience, limited as it is, that is actually how SC2 has behaved. WoW's system was characterized by hard caps on the change that could be made to your displayed rating from any one match (-47 to +47), and this has matched my experience in SC2 (although its closer to the high 20s, I think). The matchmaking system also makes it extremely unlikely that you will be placed in a match that could possibly result in a significant rating loss (the only way you could manage it in WoW was to swap out a player with a high MMR for one with an extremely low one, thereby bringing your team's average MMR much lower than the displayed rating. In such a case, you would see minor gains for wins and extreme losses for defeats).
[edit] The eternal platinum phenomenon actually reminds me of how the arena rating system behaved when you were at the very top end of the displayed rating scales. I played with some of the first players to ever legitimately hit a displayed rating of 3000, and their runup to that rating was dominated by hugely punishing losses, such that they had to maintain a win ratio in excess of 80% to actually make it to 3000 (which was a hard cap at the time).
|
kzn: Bliz probably just have the formula patented(retardedly).
I see your point with the bug. The more games you play, the lower your sigma gets, the more likely you are to be matched with someone with an extremely similar MMR and thus your MMR presumably changes by a much smaller amount each match. Some numbers on that MMR change calculation sure would be nice. Aren't there any math buffs here who would be able to roughly extrapolate from the numbers in this thread?
|
This is why I love math. All this analytical stuff can be applied to even video games! =D
|
On August 08 2010 14:32 brad drac wrote:kzn: Bliz probably just have the formula patented( retardedly). I see your point with the bug. The more games you play, the lower your sigma gets, the more likely you are to be matched with someone with an extremely similar MMR and thus your MMR presumably changes by a much smaller amount each match. Some numbers on that MMR change calculation sure would be nice. Aren't there any math buffs here who would be able to roughly extrapolate from the numbers in this thread?
There are no numbers to extrapolate from. MMR is hidden.
|
The link he gives has MMR numbers for WoW (and indeed personal MMR is not hidden in WoW). I don't think the extrapolation is possible, however, as the link gives us no numbers for sigma and I believe I am correct in thinking that there is an infinite number of sigma/function combinations that could satisfy the given MMR numbers.
[edit] Not to mention there's no guarantee that the MMR operates on the same scale in SC2 as it does in WoW. The displayed ratings certainly don't seem to.
|
United States12151 Posts
On August 08 2010 14:30 kzn wrote:Also, since I'm something of a WoW Arena expert, I'll try to add to this section. Show nested quote +Displayed Rating
Ok, how does all of this tie into displayed rating and the whole “favored” deal? If you remember back to WoW, ratings changed based on a direct comparison of your displayed rating to the other team’s MMR. So if your current rating was 500 and you were playing people with MMRs of 2000, your rating would jump significantly after every win because of the wide disparity. Now, we’ve identified that on the loading screen quite often players are seeing the other person as favored and the opponent (who is nominally “favored”) also sees his opponent as favored! How can this be? The theory put forth here is the system is again comparing your displayed rating to your opponent’s hidden MMR. This is not, I believe, strictly correct as a description of how WoW's system worked. As it was explained when MMR was first introduced, there was actually no interaction between your team's rating and the opposing team's MMR. How it works was like this: Say you've got a MMR of 2500, and you start a new team. It starts at 0 rating, but the matchmaking system will match you with other players of MMR 2500. If you lose a game, your team rating would not change at all. If you won, it would increase by 47 (a hard cap that was in place at least when I played). This was not explained as arising due to an interaction between the team rating and the opponent's MMR, however - it was explained as the system trying to get your team's rating as close as possible to your team's MMR rapidly. This is perhaps accomplished by a system such as that in the quote, but I'm not sure thats actually whats going on. Show nested quote +However, in SC2 we do not see the same sort of drastic jumps every time we beat a “favored” opponent. If we’re playing diamond-level players then their MMR must be way higher than our displayed rating, right? Or more to the point, if you were to lose and compare your small displayed rating to your opponent’s MMR, you would lose almost no points and clearly that is not how SC2 behaves. In my experience, limited as it is, that is actually how SC2 has behaved. WoW's system was characterized by hard caps on the change that could be made to your displayed rating from any one match (-47 to +47), and this has matched my experience in SC2 (although its closer to the high 20s, I think). The matchmaking system also makes it extremely unlikely that you will be placed in a match that could possibly result in a significant rating loss (the only way you could manage it in WoW was to swap out a player with a high MMR for one with an extremely low one, thereby bringing your team's average MMR much lower than the displayed rating. In such a case, you would see minor gains for wins and extreme losses for defeats). [edit] The eternal platinum phenomenon actually reminds me of how the arena rating system behaved when you were at the very top end of the displayed rating scales. I played with some of the first players to ever legitimately hit a displayed rating of 3000, and their runup to that rating was dominated by hugely punishing losses, such that they had to maintain a win ratio in excess of 80% to actually make it to 3000 (which was a hard cap at the time).
kzn you're exactly right. I think the passage you quoted is something that I missed in my initial proofread, because the way the system operates does mirror the WoW system. I'll make the appropriate edits.
|
kzn, regarding the interaction of the team's rating and opponent's MMR I was under the impression that was how it worked. In any case, what I mean by the drastic rating changes in SC2 is that it is indeed only +22ish points not counting bonus pool on beating a "favored" team. However, when you were at very low WoW arena ratings playing people 2k+ you would lose 0 points on a loss. In SC2 you will lose display rating on a loss even if the other team was favored. Does that make sense?
|
On August 08 2010 14:47 vanick wrote:
kzn, regarding the interaction of the team's rating and opponent's MMR I was under the impression that was how it worked. In any case, what I mean by the drastic rating changes in SC2 is that it is indeed only +22ish points not counting bonus pool on beating a "favored" team. However, when you were at very low WoW arena ratings playing people 2k+ you would lose 0 points on a loss. In SC2 you will lose display rating on a loss even if the other team was favored. Does that make sense?
I'm not professing to understand how SC2 is determining when to display "favored" notices - if matches are indeed starting with both players being told they're favored, I can't think of anything from the WoW system that would explain that (not least because there was no such indicator in WoW). If it really is doing it, I think the best explanation is that its just fucked, and we shouldn't be reading anything into it.
I haven't played many games past placement in SC2 yet, so my experience is quite severely limited, but I certainly see indications that it is the same as WoW's system in how it determines displayed rating changes.
One of the major differences between SC2 and WoW's arena system is that we're only shortly out of release, and WoW's MMR system actually never had a "release" time on the live realms (Tournament realms, however, are a different story. More on this later). Because of this, every single player was basically assigned a default MMR.
Now, to digress to Tournament Realms. These essentially create a "release" situation in WoW's arena system, because the realm is entirely populated by new players with no history whatsoever. Thus, all players start at a default MMR (which I believe was 0, although this is getting into 2 years ago, so I'm not entirely sure). Thus, obviously, in the very first game that was played on a Tournament Realm, you had two players (teams, really, but its irrelevant) with an MMR of 0 and a displayed rating of 0. The team which won gained identical amounts on both scales, and the team which lost lost nothing (because it was at 0). This resulted in a player's MMR being identical to his team's rating for as long as that player played 100% of the games.
In these early stages, losses and wins were pretty much equal in value. The system acted approximately like the old zero-sum system, except that, since it started at 0, some inflation was guaranteed from the outset. However, as teams started to diverge and the matchmaking started to get less equal, the system started to operate more as was familiar from live realms.
Coming back to SC2, I suspect a lot of the "odd" behavior we're seeing is odd only because we are comparing a release-environment behavior with the behavior of a system that has been running for 2+ years now (I think).
[edit] Also, the league system in SC2 has no analog in WoW. The threshold values being added to displayed rating makes some sense, but strictly that would suggest that negative displayed ratings would be possible in all leagues above Bronze.
One thing I do wonder about with regards to the league system is whether or not players experience jumps or falls in the quality of their opposition after a promotion or demotion, respectively. If the quality of opposition is continuous, we could conclude that MMR is operating completely separately from the league system and displayed ratings - but if it experienced discontinuities, that would suggest that MMR itself was interacting with the league system (which I doubt, but might explain some of the weirdness).
|
No, I think you are. Call me simple but your the only one doing this.
Nice massive paragraph info to basically say "Im right until Im not and if Im not I dont have enough info"
Awesome use of semantics.
|
No, I'm really not.
There's basically no way to explain how the favored notices are given out, unless we guess that it has to do with displayed rating and we're correct - which is doubtful, just based on what that kind of notice is supposed to do.
If you have a point, you're welcome to make it, but if you're just going to bitch do it in a thread thats less worth reading.
|
Being favored is almost certainly based on comparing your displayed rating to your opponents MMR.
Yes, this is meaningless and deceptive, and doesn't actually tell us who is really favored to win.
Yes, it *should* compare your MMR to your opponents MMR.
And no, there is no discontinuity in skill level, as you move up leagues. Matchmaking is great and is purely based on MMR, and is separate from the disastrous ranking system.
|
On August 08 2010 15:12 paralleluniverse wrote:
Being favored is almost certainly based on comparing your displayed rating to your opponents MMR.
Yes, this is meaningless and deceptive, and doesn't actually tell us who is really favored to win.
Yes, it *should* compare your MMR to your opponents MMR.
And no, there is no discontinuity in skill level, as you move up leagues. Matchmaking is great and is purely based on MMR, and is separate from the disastrous ranking system.
Do we have evidence for this? Isn't it possible that a player is displayed favoured if his MMR is greater than your own minus a factor based on sigma? Or something along those lines. I don't really see how you could directly compare displayed rating to MMR considering displayed ratings are division independent, not to mention league independent, while MMR is an absolute measure of a player's success in the system.
|
United States12151 Posts
kzn, about starting MMR, I'm almost completely positive that everyone starts at 1500. If you remember, when the system was first introduced, everyone's rating (and also their MMR) started at 1500. After the first few seasons, they changed it so that new teams and players start at 0 rating but they still start at 1500 MMR. The reason for the change was that under the old system, it was common for people to create new teams whenever the old one fell below 1500, thereby cluttering up the 1500 range which was supposed to be the average value on the scale (from 0-3000). Under the new system, teams still rapidly gained rating even if they had never played a single game in their history because their MMR was so far from their new rating of 0. The average was still 1500 under the new system, but the key difference was that you had to work -- and more importantly, win -- to get there. The decision to start everyone at 0 rather than a default value was also done in the early stages of the SC2 beta, where everyone went from starting at 1000 to starting at 0, presumably for similar reasons (though complicated by the fact that you can't reset your stats).
|
United States12151 Posts
On August 08 2010 15:32 brad drac wrote:Show nested quote +On August 08 2010 15:12 paralleluniverse wrote:
Being favored is almost certainly based on comparing your displayed rating to your opponents MMR.
Yes, this is meaningless and deceptive, and doesn't actually tell us who is really favored to win.
Yes, it *should* compare your MMR to your opponents MMR.
And no, there is no discontinuity in skill level, as you move up leagues. Matchmaking is great and is purely based on MMR, and is separate from the disastrous ranking system. Do we have evidence for this? Isn't it possible that a player is displayed favoured if his MMR is greater than your own minus a factor based on sigma? Or something along those lines. I don't really see how you could directly compare displayed rating to MMR considering displayed ratings are division independent, not to mention league independent, while MMR is an absolute measure of a player's success in the system.
We don't have direct evidence for that, but it would make sense if each league had a threshold (and we could think of this as essentially a "rating boost" for the purposes of point calculation). The original post attempts to cover this in more detail, but just to throw out some arbitrary numbers as an example, if we said it was something like:
Bronze -- 0
Silver -- 1000
Gold -- 1500
Platinum -- 2000
Diamond -- 2500
The theory we're tinkering with now is that if you're 300 in Diamond, your "global rating" is 2800. To go even further out on a limb, these points don't include anything gained from the bonus pool. This is our attempt to prove some kind of bridge or relationship between league point values, which are so far impossible to quantify.
For the record, though, displayed ratings are not division independent -- they're comparable across all divisions in a league because everyone in that league is playing against (mostly) the same player pool. I'm going to be careful about taking this too far because there's little evidence to support it, but it's an idea to throw out there that makes rating values translatable across leagues.
|
Entirely possible. Its been a long time since I played WoW, and even longer since the TR I played. That TR might have been before the 1500->0 change happened. I just remember that the rating system started acting very much like the old zero-sum system when everyone's MMR's matched the displayed rating.
|
|
|
|
|
|
EPT
