|
Updated list of noteworthy posts in this thread:
+ Show Spoiler +
Following from the success of my Battle.net feedback thread last week, I’ve compiled my thoughts on the ladder system. About half of the arguments should be familiar if you’ve read my previous posts. I outline the flaws of the bonus pool, points and division systems. I suggest how they can be fixed and integrated with a global ladder. This thread can be found on the Battle.net forums here.
The square brackets in the summary indicate the section where additional information can be found.
1. Summary
The points system is flawed because of the bonus pool. The bonus pool distorts points, making it inaccurate for ranking players [2.2]. The goals of the system appear to be: (1) to prevent stagnation, (2) to encourage playing, (3) a positive psychology version of a decay system to allow activity to be factored into points and ranks.
But it fails at these goals. On the psychology of (1) and (2) the bonus pool causes the inflation of other player’s points, which in turn perpetually causes your rank to fall, a punishment symmetric to the reward of increasing points. So overall, it's not a psychological reward, it creates a “treadmill effect” [2.1, 2.3]. On (3), For the purpose of accurate ranking, there is no reason to account for activity, other than as a proxy to uncertainty about MMR. But there is no need to use activity as that proxy because uncertainty about MMR is an already-known quantity which can be included into points [2.4].
To address (2), a wider range of shinier rewards and increased grind should be added to the leveling system [3.1]. Similar to TrueSkill, points should be redesign to be MMR-1.96*sigma, where sigma is the uncertainty about MMR that is increased at the end of the week if the player isn’t sufficiently active. This addresses (1) by explicitly accounting for uncertainty, so there is no need to use activity measured through bonus pool as a proxy [3.2].
The only remaining legitimate role of bonus pool is (3). The following changes are required to properly address stagnation while avoiding the flaws of the current bonus pool system: Points should be changed to converge to MMR-1.96*sigma, instead of MMR. To reduce distortions, the amount of bonus pool can then be significantly reduced, and the more bonus pool a player has the faster it should be consumed. These changes are only sensible in light of the suggestions to address (1) which accounts for psychology, and (2) which accounts for accuracy. To further reduce distortions and the treadmill effect, bonus pool should only be rewarded in bulk at the start of the week and the ladder should only be updated then, instead of in real time [3.3].
The most fundamental purpose of a ladder system is to correctly and accurately ranked players. Since division ranks are meaningless [4.1], a global ladder needs to be implemented using the changes to the points and bonus pool system described above. For each player, the global ladder should give a percentile out of all active players [4.2]. GM league is flawed and should be scrapped. It would also be obsolete if there’s a global ladder [4.4].
As Blizzard says, unranked play should largely address ladder anxiety. However, the following changes should help to further reduce ladder anxiety. Ranks should be made less prominent and points more prominent in order to reduce the treadmill effect [5.1]. A tutorial about ranked games should be produced to explain that the design of the ladder system, the fact that MMR is self-correcting, implies that it is impossible to stuff up your current season stats no matter how long the losing streak [5.2]. Past season history should be hidable as it is the only remaining way to permanently stuff up your account [5.3]. Team games should be encouraged before solo games [5.4].
2. The Flaws of the Points and Bonus Pool System
In this section, I will explain how the bonus pool system fails as a positive psychological gimmick and distorts points and ranks.
2.1 Bonus Pool is not a catch up mechanism
Blizzard claims that the bonus pool is to help casuals keep up on the ladder.
Q. What is the Bonus Pool and how are bonus points acquired? A. The Bonus Pool is an accumulation of points that every player receives whether they're online and playing or not. They're essentially used as a means to help give a player a catch-up boost if they haven't played in a while. The pool does have a cap, but it increases slowly until the end of a season. Source: http://us.battle.net/sc2/en/blog/110519
This is completely wrong. The bonus pool perpetually inflates the points of more active players, which causes your rank to fall and continually requires you to play on a daily basis, even to maintain the same position. This creates a "treadmill effect”. If Alice and Bob are equally skilled and Alice becomes less active, then her points will diverge from Bob's. The bonus pool doesn't help Alice close this divergence, it’s the cause of this divergence in the first place. It’s the problem, not the solution.
If there were no bonus pool, after approximately 30 games, players will hit their MMR and fluctuate around this point unless there’s a legitimate change in their skill. This is what happens when bonus pool is used up anyway. In this case, players would stagnate at their true skill level, meaning that there would be no need to catch up. Thus, having no bonus pool system would be far more helpful to players catching up, since they won’t have to chase a moving target. So Blizzard is wrong about bonus pool and their justification for it is completely fallacious.
2.2 Bonus Pool distorts ranks and increases the time taken for points to self-correct
The bonus pool system causes points to inflate until the season lock, so that the ladder never really stabilizes. Suppose that Alice and Bob have used up their bonus pool and are ranked 10 and 15 respectively. Alice is more skilled than Bob. The next day, Alice doesn’t play, so she falls to rank 14 because other players have used their bonus pool. Bob uses his bonus pool moving him to rank 12. Bob is now erroneously ranked higher than Alice, until Alice and the other players use their bonus pool to increase their points and bump Bob down. Until the season lock, this situation is perpetual.
The bonus pool also obstructs and hinders MMR from self-correcting, because a player’s points cannot decrease until bonus pool is used, even when skill legitimately does. For example, suppose Alice plays actively, she has true MMR 1000, and bonus pool is given at the rate of 50 per week. In week 5, Alice will correctly have 1250 points. Now suppose that she takes a week off and her true MMR has dropped to 800 (e.g. she had a real life injury, or forgot how to play, etc.). Then in week 6, her correct points would be 1100 (1300 minus 200 MMR for loss skill). But, she’ll still be stuck on 1250 points, which wouldn’t decrease until the bonus pool is used up. Even without the HotS change where lost points are absorbed by the bonus pool, Alice's points will on average change very little, until the bonus pool is spent. Hence, Alice's points have been distorted to be erroneously higher than is correct, with adjustment only happening after the bonus pool is used up. In contrast, adjustment would be immediate had there not been a bonus pool system.
These distortions mess up ladder ranks.
2.3 Bonus pool is not a “feel good” decay system, nor a psychological reward
Some apologists of the bonus pool system claim that it’s all about positive psychology. Bonus pool prevents stagnation by letting points increase even if skill and MMR plateaus. A decay system is defined as one that deducts points at the end of each week where the player has not played enough games. Instead of a decay system where players are punished for not playing, the bonus pool system "rewards" players for playing.
At least that's what it tries to be. Bonus pool was seemingly designed with the same philosophy as WoW’s rested XP system. Back in WoW's beta, instead of punishing players by reducing 50% of XP gain when they've played too much, Blizzard doubled all XP and made rested XP a psychological reward by having it always give a 100% bonus.
But such logic cannot be applied to a ranking system, where one player's gain is another player’s loss. Every day you log in, you see your rank fall because of the treadmill effect. Accounting for the fact that other player’s bonus pool causes your rank to continually fall, obviously the reward of increased points is symmetric to the punishment of falling ranks, it’s self-defeating, it cancels itself out.
Thus, these positive psychology arguments are also completely wrong. However, the bonus pool system has replaced the traditional decay system. In this section, I’ve shown why bonus pool is a flawed decay system for the purposes of positive psychology. In 2.4, I show why it fails as a decay system for the purposes of accurate ranking.
2.4 Bonus pool rewards activity in a needless and suboptimal way
Another common argument is that the bonus pool allows for the ladder to reward activity without rewarding mass gaming. While it’s a good idea to encourage activity for the sake of getting people to play the game, this should be done with a levelling system, not a bonus pool system, because the latter distorts ranks as explained in 2.2.
The bonus pool tends to increase the points of active players. But for the purpose of accurate ranking, why should activity even matter?
If we were psychic and simply knew the skill of each player at a given moment, without needing any games to be played, then we would only use this knowledge for ranking, i.e. in an ideal world ranking will be 100% skill based. However, we don't completely know someone's skill at a given moment, unless they play. This is the only reason to consider factoring activity into points and ranks, as higher activity is usually a good proxy to a higher probability that the player's MMR is correct. To the extent that we have good knowledge of a player's current skill, activity should not matter for the purpose of ranking.
This means that ideally, we want to minimize the weight given to activity as a factor, subject to the constraint that the player is active enough to give a reasonably good estimate of his current skill. For example, to have accurate ranks, decay systems that penalize players after a week of inactivity are superior to the current bonus pool system, because they reduce the weight given to activity as it doesn't matter as long as you play a little each week. The bonus pool system, however, requires that you be active always, every single day, so does not satisfy the above-mentioned criteria. Therefore, decay systems result in significantly less distortions than bonus pool systems, particularly for active players.
Suppose it takes about 30 games for the ladder system to calculate a player’s MMR to within an acceptably small uncertainty of 50, any more games would just cause uncertainty to fluctuate a little around 50. Alice joins the ladder on Jan 6. By Feb 20 she has played 90 games, just enough to consume her bonus pool, giving her 1500 MMR with an uncertainty of 50. Bob joins the ladder on Feb 20 and plays 30 games that day, ending with 1500 MMR and the uncertainty about his MMR would also be 50. Bob will have fewer points than Alice because he is 60 games short of consuming his bonus pool. But for the purposes of accurate ranking, there is absolutely no reason why Alice should have more points, since they both have equal MMR and equal uncertainty about MMR. Note that a decay system does not face this problem. Therefore, activity as measured by consumed bonus pool can be a bad proxy to uncertainty about MMR.
In fact, it’s completely unnecessary to use activity as this proxy, because the system already measures it directly and it’s called sigma. Therefore, bonus pool is flawed because it factors activity into points and ranks, when there is no reason for activity to matter since what we ultimately want from it is uncertainty about the player’s MMR, which is a number the system already knows.
3. Fixing the Points and Bonus Pool System
Above we have identified 4 goals of bonus pool.
(1) As a catch up mechanism.
(2) To prevent points from stagnating.
(3) To encourage playing more games.
(4) To allow activity to be factored into points and ranks.
It is logically impossible to achieve (1) for any serious ranking system, as explained in 2.1, so this goal will be ignored. So far I have shown that the bonus pool fails at all of these goals, except (2). But worse than failing, I have shown that bonus pool distorts points and ranks thereby screwing up the ladder. In this section, I suggest how to design a ladder system that achieves all of these goals, while only distorting points and ranks to the smallest possible extent.
3.1 Only encourage playing and reward activity through the leveling system.
Cosmetic rewards should be used to address (3) because any encouragement through giving bonus points is self-defeating. These cosmetic rewards should be attached to the leveling system. They should include portraits, decals, unit models or B.net backgrounds, and not be as completely lame as they currently are in the HotS beta. There would need to be an option to turn off special decals and unit models as explained in section 40 in the Battle.net feedback thread. In addition, the leveling system should be made more grindy. For example, playing games could reward tokens that are used to purchase the shiniest cosmetic rewards. The goal here is purely to encourage playing.
3.2 Make a more accurate ranking system by ignoring activity and explicitly including sigma
Note that goals (2) and (3) already address psychology. Therefore, in addressing (4), we are purely concerned with accurate ranking. To explicitly account for uncertainty about MMR, instead of indirectly using consumed bonus pool as an imperfect proxy, points should simply be set to MMR-1.96*sigma (possibly scaled so that the numbers fall into a reasonable range), which is the lower bound of the 95% confidence interval for MMR. Then the current bonus pool system becomes unnecessary. This is essentially what is done in TrueSkill (it uses 3 instead of 1.96).
Here sigma is the uncertainty about MMR, which is initially chosen so that points are equal to 0 for new accounts. It should increase at the end of each week if the player has not played enough games, reflecting the fact that we are less certain about a player’s current skill if he has not recently played.
Currently, the ladder system ranks by points that converge to MMR as long as the player is active enough to spend bonus pool. The proposed system converges to MMR as long as the system gets increasingly sure about the player’s MMR.
A technical issue:
+ Show Spoiler +There is a potential complication with setting points to MMR-1.96*sigma. In SC2's ladder system, if you win against a higher skilled player, your MMR will increase, but so will sigma, because the result is a surprise. So despite winning against the odds, MMR-1.96*sigma can change little or even decrease when you beat a higher skilled player. This is caused by a combination of 2 issues. Firstly, in SC2's ladder system sigma doesn't necessarily decrease as it does in TrueSkill (so this problem is nonexistent in TrueSkill). Secondly, in both systems skill is modelled by a symmetric normal distribution. This means that if you a win against a higher skilled player, the SC2 ladder system says that your skill level is higher than previously believed, but the surprise of your improbable win makes the spread of your likely skill level wider, both above and below what was previously believed. However, this is clearly wrong, the spread of your likely skill level should be heavily weighted above what was previously believed and lightly weighted below what was previously believed, instead of being widely spread in both directions. For example, recall that at SC2's launch there were a few players with records around 50-0 ( Ownage and CauthonLuck) who failed to be promoted. They clearly passed the MMR threshold, but failed the sigma threshold. But after winning 50-0, the system should clearly have a lower bound on their skill level, and hence they should have been promoted. This problem demonstrates an ineluctable mathematical truth about normal distributions, sigma basically measures the spread above and below the mean and both must be equal. And we know from the Bayesian inference formula shown at Blizzcon that Blizzard uses normal distributions. To fix this, skill needs to be modeled with a skewed, not symmetric distribution, like a gamma or skew-normal distribution. Due to the lack of symmetry, MMR-1.96*sigma would need to be replaced with the more general concept of the 2.5th quantile of the posterior distribution for skill (in the case of a normal distribution, both concepts are completely identical). This would solve both problems outlined here. All of my suggestions are still applicable in this case. The change to a skewed distribution should be adopted even if points aren’t changed to MMR-1.96*sigma, because it’s more correct and fixes the promotion problem outlined above.
3.3 Do the bonus pool correctly: make points converge to MMR-1.96*sigma significantly reduce bonus pool, make bonus pool consume at a faster rate the more bonus pool a player has, give out bonus pool weekly not hourly, update the ladder weekly not in real time, deemphasize ranks and emphasize points
So far we have addressed (3) and (4) without needing the distortionary bonus pool system. The only way to address stagnation without some sort of bonus pool is to increase every player’s points every hour, regardless of their activity. This is not a completely terrible idea. However, this section explains how bonus pool can be redesigned to address (2), while minimizing the distortionary and treadmill effects that are caused by the current system.
The only remaining legitimate reason for bonus pool is to prevent stagnation. Firstly, the suggestion in 3.2 should be slightly amended so that points converge to MMR-1.96*sigma instead of being precisely that as having both a decay system and bonus pool system doesn’t make sense. Next, bonus pool can be significantly reduce, from about 110 per week in HotS, to 20 per week. Additionally, the more bonus pool a player has the faster it should be consume. For example, if you have 100 bonus pool, getting 12 points for winning should use, say, 24 bonus pool, if you have 200 bonus pool, it should use 84 bonus pool. Note that these changes only make sense when implemented together with the suggestions in 3.1 and 3.2 that have already addressed the need to encourage activity and account for uncertainty about MMR. Hence, these changes to trivialize bonus pool have only the purpose of preventing stagnation and nothing more.
These are positive changes because significantly reducing the bonus pool would significantly reduce the distortionary and treadmill effects it creates. Allowing bonus pool to be consumed faster when players have large bonus pools partly addresses the problems in the second example in 2.2 and the “Jan 3 vs Feb 20” example in 2.4. It also partly addresses (1), but no changes in any serious ranking system can (nor should) entirely fix (1).
In addition, bonus pool should be given in bulk, once weekly, instead of in small amounts each hour, and the ladder should only be updated at this time, instead of in real time. Updating the ladder once a week will significantly reduce the treadmill effects since players will no longer see their rank perpetually fall due to other player’s bonus pools. But more importantly, these changes will mostly eliminate the distortionary effects that bonus pool has on ranks as explained in the first example of 2.2. In that example, Alice is more skilled than Bob. She doesn't play for a day and falls below Bob's rank as a result of Bob's bonus pool. So the ladder ranks have become wrong. Now if the ladder were to update only once weekly and bonus pool were changed as I've suggested, then Alice would be able to get back ahead of Bob, before the next ladder snapshot. If she didn't, it would be because she was inactive for the week, so it could be justified that her rank should fall as a small penalty for the chance that her skill has decreased due to prolonged inactivity. However, such an argument cannot be applied to the current bonus pool system because Alice would not lose any skill due to not having played for one hour or one day. The skill lost for 2 weeks of inactivity is far more than 14 times the skill lost in 1 day of inactivity.
Lastly, to further reduce the treadmill effect and to maximize the benefits of having prevented points from stagnating, ranks need to be removed from the matchmaking page and the score screen. Instead, points should be emphasized, as they no longer stagnate. Ranks should be kept in the ladder page in the profile. They are critically important for competition in a competitive game.
4. The Meaningless Division System vs a Global Ladder
4.1 Division ranks have no useful interpretation
The division system is pointless. Your division rank says nothing about how you compare with other players, even within the same league.
Blizzard claims that division ranks have a meaning.
But this is simply untrue. If players in the same league were randomly placed into divisions, and if for each player in the league, their skill doesn’t change throughout the season, then it would only be true that being rank 2 will on average mean that the player is in the top 2%. Even in this ideal case, there is no guarantee that some randomly created divisions aren’t by chance more skilled than others. But more importantly, this isn’t how divisions are created. They’re created on a first come, first serve basis, and hence divisions created at the start of the season will tend to have more active, and hence more skilled, players than divisions created later.
Moreover, with the change that allows players to leave leagues, players can shop for easier divisions by checking the points of players in it using SC2ranks. This actually gives empirical evidence that Blizzard’s statement is factually wrong, so it is untrue that removal of division tiers somehow validates division ranks. There is no point in ranking against 100 arbitrary and faceless players. Despite Rob Pardo’s misguided attempt to convince us otherwise, there is still no reason to care.
Leagues do not solve the problem of division ranks being meaningless and there being no way to get a reasonable measure of your skill relative to all active players. The 5 leagues other than Masters and GM cover an approximately 20% skill range, in the sense that Platinum league contains players in the top 20%-40%. This is a very large skill gap.
4.2 Creating a global ladder
The solution is to create a global ladder, where all players in the season and on the server are ranked by points. This does not necessarily mean that the division ladders should be removed, just that division ranks are meaningless. Both global and division ladders can coexist.
The points system suggested in 3.2 and 3.3 should be used to ranked players on a global ladder. As previously explained, if the bonus pool is not scrapped, then the global ladder should be updated once weekly instead of in real time. A percentile should be used. For example, 84.2 instead of 3756 out of 20000. This number is very easy to understand, 84.2 simply means that you’re better than 84.2% of all players that are ranked (this is exactly how university admission ranks work in Australia).
Attrition throughout the current season can reduce the usefulness of the global ladder. For example, if you're in the top 30% only because 50% of players have 0-1 records and are no longer active, then you're not really in the top 30%. It isn't helpful to be compared with people who no longer play. Therefore, the default view of the global ladder and the percentile should exclude inactive players, but there should also be a view that includes all players in the current season. Sometimes you want to include marathon runners who failed to reach the finish line, but other times, only competition with marathon runners who cross the finish line matters. Alternatively, there could be a minimum number of games per season, say 10, before a player is added to the global ladder, any less means that the player isn’t serious enough to be ranked.
4.3 Tired arguments against a global ladder
Objections to a global ladder can generally be classified into 2 types of arguments. Firstly, the global ladder causes ladder anxiety and hurts casuals. Secondly, players cannot be accurately ranked on a global ladder so that Blizzard shouldn’t bother.
On the first argument, section 5 outlines changes to reduce ladder anxiety. In fact, the ladder system I’ve proposed would be far more accommodative to casuals, as it fixes the treadmill effect of the current system, removes the obsessive, harmful and meaningless fixation on division ranks, prevents points from stagnating, and encourages playing, while at the same time making the ladder far more accurate. But ultimately, this is a competitive game. Competition and epeen matters. The fundamental purpose of a ladder system is to correctly and accurately rank players, not to tend to the hurt feelings of sensitive players or casuals.
When Blizzard introduced win loss stats in HotS, they said that:
Players who play competitively on the ladder can now better track their progress regardless of which league they’re in. And those players that would like to enjoy the benefits of matchmaking, but are not interested in the pressure of being ranked can now use the unranked play mode. Source: http://us.battle.net/sc2/en/blog/7634957/
This argument can be applied to a global ladder. After all, a global rank is just another stat.
Ironically, the “bad for casuals” argument is seemingly most often made by players who strongly defend Blizzard’s current ladder system as being good for casuals despite either not realizing or ignoring the fact that bonus pool hurts casuals due to the treadmill effect, that the removal of division tiers makes it harder for casuals to rank up on the division ladder (this was the most significant change Blizzard has made to improve accuracy and comparability in the ladder system), and that Blizzard brands a mediocrity badge called “Bronze League” on 42.1% of players forehead. They defended Blizzard for hiding losses, and still defend Blizzard despite losses returning in HotS.
On the second argument, it is sometimes claim that a global rank is too noisy, meaning there are so many players on the ladder that getting 12 points for a win would move you, say, 847 places up the ladder. Using a percentile and updating the ladder only once a week would eliminate noise.
But even if there is noise, so what? Game results provide useful information for accurately updating points and ranks. Should we avoid ever updating points because it adds “noise”? If noise is a problem, then what about all the noise in the current division ladders?
There are also claims that skill simply cannot be measured accurately with MMR and that there’s no way to account for uncertainty about MMR, so that global ladders are meaningless. But this is just completely wrong. The uncertainty about MMR, is already measured by the system, and I’ve suggested that it should be explicitly included into points by using MMR-1.96*sigma. And even if sigma isn't directly used to calculate points it still tends to reduce as games are played. Skill can be measured with good accuracy, as shown by the near 50-50 matchmaking SC2 achieves using MMR, and the empirical evidence from similar skill rating system such as TrueSkill which have remarkable success.
4.4 Clearly articulate league promotion and demotion criteria and remove the GM league
Currently, the specific criteria for league promotion are not known. Blizzard occasionally releases a table with approximate criteria. This one was only valid until December 21 2011. This table would be unknown to those who don’t visit the website, and it’s not even updated for the current season. Blizzard needs to be more transparent on league promotions.
Using the global ladder I’ve suggested, the promotion criteria for Masters, Diamond, Platinum, Gold, Silver and Bronze should simply be having a percentile greater than 98, 80, 60, 40, 20, and 0, respectively. Since the current requirement, MMR > a and sigma < b, is equivalent to MMR-1.96*sigma > a-1.96*b, where a is chosen so that 2%, 18%, 20%, 20%, 20%, and 20% of active players are in the corresponding leagues, it follows that the proposed promotion criteria is approximately the same as a current promotion criteria. It accounts for uncertainty about MMR. And it’s very clear. Similarly, sticky boundaries for demotions can be set, e.g. having a percentile below 97.6, 76, 56, 36, 16, 0 will prompt a demotion. Note that one benefit of excluding inactive players from the percentile is that it allows for the proposed promotion and demotion criteria to be fixed.
The GM league should be scrapped because it is fundamentally flawed and would become irrelevant with the introduction of a global ladder. Currently, it takes the top 200 players at the start of the season. These players stay in the GM league as long as their bonus pool is less than some threshold. Consequently, the GM league does not rank the current top 200 players, instead it mostly ranks the top 200 players at the start of the season who have remained active, even if their skill level falls out of the top 200. Even worse, if a player during the middle of the season becomes more skill than the players in the top 200, than he will be unable to enter the GM league unless someone currently in the GM league becomes insufficiently active. Blizzard’s attempt to prevent players from flip-flopping in and out of GM league has created this flaw. I have no problem if Blizzard wants to reward the top 200 players on the global ladder with a golden hexagon and a feat of strength, but they should be put in a Master’s league division.
5. Ladder Anxiety
By Blizzard’s own admission, the addition of unranked played should alleviate ladder anxiety, so there’s no reason to not introduce a global ladder. In this section, I outline some additional steps to deal with ladder anxiety.
5.1 Deemphasize ranks and emphasize points
Blizzard should remove the harmful and pointless obsession with division ranks, which together with the current bonus pool system, perpetuates the treadmill effect. The first pain-free step would be to display points more prominently and ranks less prominently since points tend to increase. As suggested in 3.3, ranks should be replaced in the matchmaking page and the score screen by points. The division rank and global ladder percentile should be displayed in the ladder page in the profile. Also, only updating the ladder once a week will significantly reduce the treadmill effect.
5.2 Produce a ranked games tutorial to explain how the design of the matchmaking system implies current season profiles cannot get screwed up so that there is no need to have ladder anxiety
The reason why I don’t get ladder anxiety is simply because I know how the matchmaking system works. Therefore, I understand that MMR is self-correcting so it’s not possible to stuff up your current season ladder stats. If I’m on a massive losing streak, I know that I will be matched to easier opponents, thereby returning me to my previous MMR. So this must be a temporary losing streak which is followed by a recovery, unless my skill has legitimately fallen. In that case, it is deserved. If I eventually improve, my points will be returned to the correct value. In the long run, the MMR is always right.
Moreover, overall win loss ratios are meaningless because the matchmaker creates games with equally skilled opponents, so it nearly always converges to 50%. The only exception is for very high (or very low) skilled players, because in these cases there may not be equally skilled players, which causes the matchmaker to expand search to find less (or more) skilled players. Hence, focusing on your overall win ratio or aiming for a greater than 50% overall win ratio is both pointless and in a sense, out of one’s control.
Blizzard should add a tutorial for ranked games where the adjutant outlines how the ladder and matchmaking system works. It should explain that the system has been designed in a way that implies current season profiles cannot get screwed up, as I’ve done here. Then there would be no rational reason to have ladder anxiety.
On a side note, Blizzard’s official explanations of the ladder system are an infantile joke, they're grossly oversimplified to the point of uselessness, lack basic details, scattered across the internet, and occasionally littered with outright untruths (a few of which have been pointed out in this post). This would be a good opportunity to come clean.
5.3 Allow an option for previous season history to be hidden
As explained in 5.2, it's not possible to screw up current ladder stats. However, the previous season history is unchangeable, and it's the only way players can permanently screw up their account. So allowing an option to hide previous season history should help alleviate ladder anxiety. While this change would be a loss of information, at least it's not current information (which would be unacceptable regardless of ladder anxiety), it’s old information.
5.4 Emphasize team games
Encourage new players and casual players to play team games before venturing into solo ranked games. This piece of advice could be included in the tutorial suggested in 5.2. It’s less stressful. We also know from official WC3 stats that most people prefer team games. They makes up for almost 75% of all games played.
6. Previous Suggestions
In my Battle.net feedback thread sections 5, 8, 9, 11 relate to the ladder system.
|
Well-written post as you do most of the time.
Blizzard should remove the harmful and pointless obsession with division ranks, which together with the current bonus pool system, perpetuates the treadmill effect. The first pain-free step would be to display points more prominently and ranks less prominently since points tend to increase. As suggested in 3.3, ranks should be replaced in the matchmaking page and the score screen by points. The division rank and global ladder percentile should be displayed in the ladder page in the profile. Also, only updating the ladder once a week will significantly reduce the treadmill effect.
What's your suggestion for keeping players motivated to play then? I think ranks have a two-sided effect. On one side players really wanna get to top 8 or w/e, and on the other hand they are too scared to lose as it might mean they drop out of top 16 or get demoted to a worse league.
I think an optimal ladder looks at both issues and rewards activity regardless of whether you lose or win. The current HOTS xp system seems to be an improvement, but I am not convinced casuals will really care.
|
On December 22 2012 04:53 Hider wrote:Well-written post as you do most of the time. Show nested quote +Blizzard should remove the harmful and pointless obsession with division ranks, which together with the current bonus pool system, perpetuates the treadmill effect. The first pain-free step would be to display points more prominently and ranks less prominently since points tend to increase. As suggested in 3.3, ranks should be replaced in the matchmaking page and the score screen by points. The division rank and global ladder percentile should be displayed in the ladder page in the profile. Also, only updating the ladder once a week will significantly reduce the treadmill effect. What's your suggestion for keeping players motivated to play then?
3.1 Only encourage playing and reward activity through the leveling system.
Cosmetic rewards should be used to address (3) because any encouragement through giving bonus points is self-defeating. These cosmetic rewards should be attached to the leveling system. They should include portraits, decals, unit models or B.net backgrounds, and not be as completely lame as they currently are in the HotS beta. There would need to be an option to turn off special decals and unit models as explained in section 40 in the Battle.net feedback thread. In addition, the leveling system should be made more grindy. For example, playing games could reward tokens that are used to purchase the shiniest cosmetic rewards. The goal here is purely to encourage playing.
I think ranks have a two-sided effect. On one side players really wanna get to top 8 or w/e, and on the other hand they are too scared to lose as it might mean they drop out of top 16 or get demoted to a worse league.
I think an optimal ladder looks at both issues and rewards activity regardless of whether you lose or win. The current HOTS xp system seems to be an improvement, but I am not convinced casuals will really care.
As I've explained in the post there is no reason to reward activity for the sake of rewarding activity. Why reward activity? The only reason to consider activity is as a proxy to uncertainty about MMR, in which case it would be more optimal to explicitly account for uncertainty about MMR by using MMR-1.96*sigma.
Another possible reason to reward activity is to encourage playing. In this case, the only acceptable reward for losing or winning are cosmetic rewards through the leveling system, otherwise you distort ladder ranks as explained in 2.2.
Are there any other possible reasons why the ladder system should reward activity?
|
On December 22 2012 05:00 paralleluniverse wrote:Show nested quote +On December 22 2012 04:53 Hider wrote:Well-written post as you do most of the time. Blizzard should remove the harmful and pointless obsession with division ranks, which together with the current bonus pool system, perpetuates the treadmill effect. The first pain-free step would be to display points more prominently and ranks less prominently since points tend to increase. As suggested in 3.3, ranks should be replaced in the matchmaking page and the score screen by points. The division rank and global ladder percentile should be displayed in the ladder page in the profile. Also, only updating the ladder once a week will significantly reduce the treadmill effect. What's your suggestion for keeping players motivated to play then? 3.1 Only encourage playing and reward activity through the leveling system.Cosmetic rewards should be used to address (3) because any encouragement through giving bonus points is self-defeating. These cosmetic rewards should be attached to the leveling system. They should include portraits, decals, unit models or B.net backgrounds, and not be as completely lame as they currently are in the HotS beta. There would need to be an option to turn off special decals and unit models as explained in section 40 in the Battle.net feedback thread. In addition, the leveling system should be made more grindy. For example, playing games could reward tokens that are used to purchase the shiniest cosmetic rewards. The goal here is purely to encourage playing. Show nested quote +I think ranks have a two-sided effect. On one side players really wanna get to top 8 or w/e, and on the other hand they are too scared to lose as it might mean they drop out of top 16 or get demoted to a worse league.
I think an optimal ladder looks at both issues and rewards activity regardless of whether you lose or win. The current HOTS xp system seems to be an improvement, but I am not convinced casuals will really care. As I've explained in the post there is no reason to reward activity for the sake of rewarding activity. Why reward activity? The only reason to consider activity is as a proxy to uncertainty about MMR, in which case it would be more optimal to explicitly account for uncertainty about MMR by using MMR-1.96*sigma. Another possible reason to reward activity is to encourage playing. In this case, the only acceptable reward for losing or winning are cosmetic rewards through the leveling system, otherwise you distort ladder ranks as explained in 2.2. Are there any other possible reasons why the ladder system should reward activity?
I think a reason why COD is motivating to play is that you feel like you accomplished something whether you lost or won (because you levelled op). I am not sure whether a level system purely based on winning games can motivate people at the same degree without causing ladder anxiety. Or are you suggesting that peolple should be able to level up even they are losing games?
EDIT: Read it again, I mostly agree with you. I think that you are correct that your suggestion could reduce ladder anxiety while still keeping motivation up through the levelling system. Misunderstood your in my previous post, sorry about that.
|
Not only should we be allowed to hide our season history but also our current stats and rank. A part of ladder anxiety is the feeling of others judging our lack of skill. The bad feeling of being in bronze or silver etc is exacerbated by the fact that you wear it like a badge on top of your hat for everyone to see. It's like having a banner saying "I suck".
Not only that, but you play not knowing when, if you will be demoted. Each loss can mean demotion. Actually with this kind of system it would be surprising if there wasn't ladder anxiety.
|
On using sigma:
Using MMR-1.96*sigma as a convergence points has a couple problems with it. The main one you've solved - which is that by using a nonsymmetric distribution for your confidence intervals, you can avoid the problem of having uncertainty create the obvious error of penalizing you for winning. However, the distribution in general still behaves in some odd ways because you've placed the convergence point on a bound rather than in the heart of the distribution. Consider this: if your nonsymmetric distribution has the effect of creating more uncertainty on the upper bound when you win games you shouldn't, it should create more uncertainty on the lower bound when you lose games you shouldn't also, correct? Effectively, doesn't this mean you'd be gaining only a little in your convergence point for winning games, while losing a lot in your convergence point for losses? Of course, this wouldn't effect matchmaking or anything (as points are an entirely separate game IIRC), but doesn't this have the effect of allowing a small number of sloppy losses to introduce a disproportionately large impact on your convergence point? How can you be sure of your 95% confidence interval if in the last 30 matches you happened to have 2 random d/cs against 2 really poor players? See what I'm driving at? MMR is used as a convergence point because it's more stable.
You could potentially fix this by letting points converge to MMR+1.96*sigma (so that you're giving players the benefit of the doubt, kinda, instead of being conservative). But really, the point is that MMR is already relatively unknown. By using a bound of a 95% confidence interval, you're making something that looks more and more like noise and less and less like something useful.
On Ladder Anxiety:
I'm sorry to say that you've missed the point a bit on ladder anxiety in 5.1-5.3 (though I like the inclusion of team games, though, that would do a ton). People aren't afraid to queue because they see and understand the potential volatility in their ladder ranking, or their points, or whatever. People are afraid to queue for a much more basic reason - they're afraid they'll do something which makes them look silly.
It's like this - you can explain to someone that by asking a girl to a dance, they can't actually possibly lose anything except the indifference of a girl who's probabilistically unlikely to date you anyway, unless you count the fact that you could get married to said girl and she could rule your life with an iron fist, leaving you a crushed and hopeless shell of a man who has to ask permission to use the bathr... oh look, it appears I've wandered - but anyway, the point of going with said girl in the first place is to initiate such a relationship, so I wouldn't think one should count that. And you can explain that if for some reason said girl does decide to go to the dance with you, it could be a great boon to the rest of your life.
But explaining the logic of the matter doesn't deal with the anxiety. If it promotes an action one anyone's part, psychologically speaking it isn't generally because you've dealt with the anxiety, but rather because you've created a social situation in which people feel silly for being afraid of feeling silly - you can see the possible issues this might present people. But really, any "tutorial" about how someone is supposed to feel is most likely to be ignored or misunderstood anyway.
The only ways I know of to deal with anxiety is through practice and discipline, through medication, or through evasion (hence my like of team games, where everything wrong can always be someone else's fault).
|
On December 22 2012 06:04 Hider wrote:Show nested quote +On December 22 2012 05:00 paralleluniverse wrote:On December 22 2012 04:53 Hider wrote:Well-written post as you do most of the time. Blizzard should remove the harmful and pointless obsession with division ranks, which together with the current bonus pool system, perpetuates the treadmill effect. The first pain-free step would be to display points more prominently and ranks less prominently since points tend to increase. As suggested in 3.3, ranks should be replaced in the matchmaking page and the score screen by points. The division rank and global ladder percentile should be displayed in the ladder page in the profile. Also, only updating the ladder once a week will significantly reduce the treadmill effect. What's your suggestion for keeping players motivated to play then? 3.1 Only encourage playing and reward activity through the leveling system.Cosmetic rewards should be used to address (3) because any encouragement through giving bonus points is self-defeating. These cosmetic rewards should be attached to the leveling system. They should include portraits, decals, unit models or B.net backgrounds, and not be as completely lame as they currently are in the HotS beta. There would need to be an option to turn off special decals and unit models as explained in section 40 in the Battle.net feedback thread. In addition, the leveling system should be made more grindy. For example, playing games could reward tokens that are used to purchase the shiniest cosmetic rewards. The goal here is purely to encourage playing. I think ranks have a two-sided effect. On one side players really wanna get to top 8 or w/e, and on the other hand they are too scared to lose as it might mean they drop out of top 16 or get demoted to a worse league.
I think an optimal ladder looks at both issues and rewards activity regardless of whether you lose or win. The current HOTS xp system seems to be an improvement, but I am not convinced casuals will really care. As I've explained in the post there is no reason to reward activity for the sake of rewarding activity. Why reward activity? The only reason to consider activity is as a proxy to uncertainty about MMR, in which case it would be more optimal to explicitly account for uncertainty about MMR by using MMR-1.96*sigma. Another possible reason to reward activity is to encourage playing. In this case, the only acceptable reward for losing or winning are cosmetic rewards through the leveling system, otherwise you distort ladder ranks as explained in 2.2. Are there any other possible reasons why the ladder system should reward activity? I think a reason why COD is motivating to play is that you feel like you accomplished something whether you lost or won (because you levelled op). I am not sure whether a level system purely based on winning games can motivate people at the same degree without causing ladder anxiety. Or are you suggesting that peolple should be able to level up even they are losing games? EDIT: Read it again, I mostly agree with you. I think that you are correct that your suggestion could reduce ladder anxiety while still keeping motivation up through the levelling system. Misunderstood your in my previous post, sorry about that.
I'm talking about improving the HotS leveling system, where you gain XP and work towards cosmetic rewards even if you lose.
|
On December 22 2012 06:48 Apolo wrote:
Not only should we be allowed to hide our season history but also our current stats and rank. A part of ladder anxiety is the feeling of others judging our lack of skill. The bad feeling of being in bronze or silver etc is exacerbated by the fact that you wear it like a badge on top of your hat for everyone to see. It's like having a banner saying "I suck".
Not only that, but you play not knowing when, if you will be demoted. Each loss can mean demotion. Actually with this kind of system it would be surprising if there wasn't ladder anxiety.
I completely disagree with hiding any current season stats.
I agree that previous season history should be hidable because it's the only way to permanently stuff up your account, and then it can never be changed. However, current season stats cannot be screwed up because MMR is self-correcting, so why hide it? Previous season history is old information, so it's not so important.
The most fundamental purpose of a ladder system is to rank correctly, not to hide stats because of the hurt feelings of those players with ladder anxiety. These suggested changes and unranked play should already solve a lot of ladder anxiety.
You're absolutely right about Blizzard's league system stigmatizing bad players. I find it puzzling when people defend the current system as good for casuals even though Blizzard brands a mediocrity badge called “Bronze League” on 42.1% of players forehead.
One of the advantages of a global ladder is that you could scrap the league system or make promotion and demotion criteria clearly stated in terms of percentiles. While in a global ladder, a person with a 0-20 percentile would be equivalent to Bronze, removing the league system and confining the percentile to the ladder summary page prevents these players from being force to wear an "I suck" on their heads, and it removes the anxiety of possibly getting demoted. No leagues system means no demotion. It's also deprives them of a derogatory label. tI might not make much of a difference, but it definitely wouldn't make things worst than they already are.
|
I do feel that ranks are quite useless to most people as it only matters to those top 10-15 active players in the division.
Maybe Blizzard can just simply show true MMR value? "I am a MMR 3000 Diamond" is more of a bragging right than say "I am top 12 Diamond last season."
Blizzard can add in "Tag" similar to Call of Duty where players can unlock more logos, icons added next to their name. Some are rewarded by grinding games out, some are rewarded base on ranks and MMR.
|
On December 22 2012 06:50 Treehead wrote:
On using sigma:
Using MMR-1.96*sigma as a convergence points has a couple problems with it. The main one you've solved - which is that by using a nonsymmetric distribution for your confidence intervals, you can avoid the problem of having uncertainty create the obvious error of penalizing you for winning. However, the distribution in general still behaves in some odd ways because you've placed the convergence point on a bound rather than in the heart of the distribution. Consider this: if your nonsymmetric distribution has the effect of creating more uncertainty on the upper bound when you win games you shouldn't, it should create more uncertainty on the lower bound when you lose games you shouldn't also, correct? Effectively, doesn't this mean you'd be gaining only a little in your convergence point for winning games, while losing a lot in your convergence point for losses? Of course, this wouldn't effect matchmaking or anything (as points are an entirely separate game IIRC), but doesn't this have the effect of allowing a small number of sloppy losses to introduce a disproportionately large impact on your convergence point? How can you be sure of your 95% confidence interval if in the last 30 matches you happened to have 2 random d/cs against 2 really poor players? See what I'm driving at? MMR is used as a convergence point because it's more stable.
You could potentially fix this by letting points converge to MMR+1.96*sigma (so that you're giving players the benefit of the doubt, kinda, instead of being conservative). But really, the point is that MMR is already relatively unknown. By using a bound of a 95% confidence interval, you're making something that looks more and more like noise and less and less like something useful.
I think this is the "problem" you describe:
Suppose that your skill distribution looks like this:
2.5% quantile (points) = 1600
mean = 1700
97.5% quantile = 1800
If you win against a more skill player with mean skill 1750, then your skill should update to something like:
2.5% quantile (points) = 1650
mean = 1780
97.5% quantile = 1900
But if instead you had lost against a player with mean skill 1650, then your skill should update to something like:
2.5% quantile (points) = 1500
mean = 1620
97.5% quantile = 1750
So in this example, winning against a higher skill player increases your points a bit, but losing to a lower skill player decreases your points much more.
However, this is not a problem because:
1. The same thing would happen to everyone, so ranks would still be correct.
2. This is just a matter of numbers and scale. It can easily be fixed by applying some monotonically increasing and convex function (so that correct ranks are unchanged) to the lower bound of the 95% CI.
![[image loading]](http://i.imgur.com/5ASV7.png)
3. So it's just a matter of tweaking numbers to make it feel right.
4. There is nothing "wrong" here. If your points decrease a lot for losing that's because the system legitimately isn't sure enough that you're as good as you were.
5. Making points converge to MMR doesn't account for uncertainty about MMR and the bonus pool is a bad proxy to the uncertainty about MMR as explained in 2.4, whereas sigma is already directly measured.
In fact, it's even possible to transform my points idea, (which explicitly accounts for uncertainty about MMR, unlike the current system) back to a linear scale without any distortions in ranks, simply by making points converge to 2000*p, where p is the percentile of the player when ranked using the lower bound of the 95% CI for skill
Hence, if this is even a problem at all, there are many simple ways to solve it without distorting ranks (as the current system does) and without really changing any of my arguments.
|
Very interesting read! One question: Could you explain exactly how league placement (at the start of the season) work with this system? Currently players exit their placement matches with 0 points, but in a global ladder-based league system, 0 points would be bronze league. Would your system award points at the end of placement to start people in their placed league on the global ladder? If so, how many points? (Maybe the bottommost MMR of the league, or the MMR after 5 matches?)
Or would they start with 0 points in bronze, regardless of their MMR after placement, and have to grind possibly 2k or 3k points to get to their MMR? (Every season? I guess you could award points for more points for wins the further someone is from their MMR+sigma equation, ala the old WoW arena system.)
If everything can be figured out, I hope Blizzard implements your global ladder ideas for HotS.
|
On December 22 2012 14:54 TiberiusAk wrote:
Very interesting read! One question: Could you explain exactly how league placement (at the start of the season) work with this system? Currently players exit their placement matches with 0 points, but in a global ladder-based league system, 0 points would be bronze league. Would your system award points at the end of placement to start people in their placed league on the global ladder? If so, how many points? (Maybe the bottommost MMR of the league, or the MMR after 5 matches?)
Or would they start with 0 points in bronze, regardless of their MMR after placement, and have to grind possibly 2k or 3k points to get to their MMR? (Every season? I guess you could award points for more points for wins the further someone is from their MMR+sigma equation, ala the old WoW arena system.)
If everything can be figured out, I hope Blizzard implements your global ladder ideas for HotS.
This is a very thoughtful and interesting question. For new players, playing 5 placement games will produce some value of MMR-1.96*sigma, and they should be placed in whatever league that falls in. They should not start at 0 points. I said that MMR-1.96*sigma is chosen to be 0 for a new account, but after 5 placement games it won't be 0, and that should be set as the points. Starting at 0 points bloats Bronze league.
For the purposes of accurate ranking, the idea of a season reset where everyone goes back to 0 points doesn't make sense. If one day, you're 1800, it isn't more accurate that you're 0 the next day because of a season reset. So ideally, season resets should keep everyone's points and ranking the same and only wipe bonus pool.
However, people do expect some sort of "clean slate" on a season reset so I don't suggest the above. There are 3 possible solutions.
Solution 1:
Place players into appropriate leagues based on MMR, like the current system. Have a separate global ladder for each league. Start players at 0 points. Since the global ladder within the league is still ranked by MMR-1.96*sigma, hitting the top of your league's global ladder is still approximately equivalent to the current league promotion criteria as explained in 4.4.
Solution 2:
Place players into leagues based on MMR-1.96*sigma. Still have 1 global ladder. Players don't start with 0 points, instead points start at the average MMR of the league.
Solution 3:
Do solution 2, but scrap the idea of leagues. As Apolo explains, Bronze league is just a big a "I suck" sticker which Blizzard forces 42% of players to wear on their heads. It doesn't help people suffering from ladder anxiety, it hurts them instead. They still might want to put a league icon next to a player's name, based on their global ladder percentile, but they should consider possibly ditching all fanfare associated with promotion and demotion and labeling people.
I'm OK with any of these solutions, but I probably prefer solution 2 or 3, because not starting at 0 points is closer to the ideal of changing nothing (and changing nothing in a season reset is the most accurate).
As explained in 2.4, there is no reason to reward activity and force players to grind back up to their previous rating, since the system already knows the uncertainty about their MMR and the leveling system already rewards and encourages activity. Thus, no matter which solution is chosen, points should be designed to not take too long to converge back to MMR-1.96*sigma, like how it's done in WoW arenas as you mentioned.
Ultimately, there is a trade-off between accuracy vs wanting a "clean slate" which Blizzard must decide.
|
Bonus pool has been hurting the ladder for a long time imho.
thumbs up for the great post
|
On December 22 2012 06:50 Treehead wrote:
On Ladder Anxiety:
I'm sorry to say that you've missed the point a bit on ladder anxiety in 5.1-5.3 (though I like the inclusion of team games, though, that would do a ton). People aren't afraid to queue because they see and understand the potential volatility in their ladder ranking, or their points, or whatever. People are afraid to queue for a much more basic reason - they're afraid they'll do something which makes them look silly.
It's like this - you can explain to someone that by asking a girl to a dance, they can't actually possibly lose anything except the indifference of a girl who's probabilistically unlikely to date you anyway, unless you count the fact that you could get married to said girl and she could rule your life with an iron fist, leaving you a crushed and hopeless shell of a man who has to ask permission to use the bathr... oh look, it appears I've wandered - but anyway, the point of going with said girl in the first place is to initiate such a relationship, so I wouldn't think one should count that. And you can explain that if for some reason said girl does decide to go to the dance with you, it could be a great boon to the rest of your life.
But explaining the logic of the matter doesn't deal with the anxiety. If it promotes an action one anyone's part, psychologically speaking it isn't generally because you've dealt with the anxiety, but rather because you've created a social situation in which people feel silly for being afraid of feeling silly - you can see the possible issues this might present people. But really, any "tutorial" about how someone is supposed to feel is most likely to be ignored or misunderstood anyway.
The only ways I know of to deal with anxiety is through practice and discipline, through medication, or through evasion (hence my like of team games, where everything wrong can always be someone else's fault).
The impression that I usually get from reading threads on this forum is that it's more about promotions and demotions and having bad losing streaks than it is about what's happening in the actual game.
If the cause of ladder anxiety is that people are afraid of getting hellion harassed or getting killed by a cannon rush or annihilated by a protoss deathball after spending 20 minutes in game, getting outmatched by zerg etc, then there really isn't too much that game designers can do other than dumbing down the game, which should definitely not happen.
I don't think there is any one cause of ladder anxiety or any one solution. It's probably impossible to completely fix, which is why I've made several suggestions to help reduce it. As for the tutorial, I think it's a good idea not because it tells people how to feel, but rather because it would explain to them that MMR is self-correcting. It's something everyone should be told. How they feel is up to them. It obviously won't be a silver bullet.
Throughout WoL, the awful design of the ladder system seemed to have been completely driven by ladder anxiety, hiding losses, making points incomparable, refusing to implement a global ladder etc. Yet Blizzard doesn't seem to realize that the league system and the bonus pool system, while not the direct cause of ladder anxiety, still makes the problem worse.
What do you suggest?
|
You touched on this in your Battle.net post, but in relation to ladder anxiety the entire "Favored" concept comes into play here as well. For someone with ladder anxiety, seeing their opponent as "Favored" gives you the feeling that you're already losing before you even start the game. On the flip side, seeing yourself as "Favored" give you the feeling that you must win this game or you will fall at a faster pace down the ladder. Both are negative psychological effects that happen before the game even starts.
I'm also glad you brought up WoW's rested XP. You are correct that it works in WoW because you are leveling as a single character and not against other characters as you are in SC2. An extreme example of the treadmill in fact is also from WoW. The original PvP ladder was simply based on points. Points were given simply by playing games. There was no way to lose points, only ways to gain them, with extra points given for playing specific games on "PvP holidays". This became a race to see who could play the game the most, not who was actually the best at PvP. Blizzard scrapped this for obvious reasons. The bonus pool in SC2 is not as bad as the original WoW PvP system, but it has similarities, and the same issues apply.
|
I certainly see how bonus pool helps you catch up. If you are active, playing 5 games a day you will have no bonus pool to draw from for most of the season as you use it regularly. That means that most of the time your wins and losses cost you the same amount of points, someone who is less active however will earn twice the amount of points for a win than they do for a loss (in hots, you don't lose any points for a loss if u have bonus pool, it is simply removed from the bonus pool instead) which inflates their points a lot faster.
I have successfully gained top8 in my division every season by using the bonus pool i get from being inactive most of the season. While everyone else is losing and gaining points at an almost equal rate (MM system making you go 50/50) I am gaining points all the time until I use up my bonus pool, which incidentally always works out to be the exact amount of points I need to get to rank 1 or 2 in my division. Using your bonus pool up every day actually means you get less use from it because most of the time you aren't actually getting any boost from it.
|
Do solution 2, but scrap the idea of leagues. As Apolo explains, Bronze league is just a big a "I suck" sticker which Blizzard forces 42% of players to wear on their heads. It doesn't help people suffering from ladder anxiety, it hurts them instead. They still might want to put a league icon next to a player's name, based on their global ladder percentile, but they should consider possibly ditching all fanfare associated with promotion and demotion and labeling people.
I disagree with scrapping leagues. I think Apolo's post may overestimate the harm and ignoring the benefits of leagues (though he was talking about hiding, not removing them). What evidence is there that bronze players actually feel this way? How many of them do? I don't hear many people saying that ICCUP should have scrapped their D/C/B/A system because people felt stigmatized at being a "D" player. I think players are emotionally more robust than that--players accept that in a competition, there is going to be a bottom level.
Also, if players felt stigma over being in bronze, then why have there been so many posts on the blizzard forums asking for demotions? (I've 2 friends that were independently terrified of being in silver when they started laddering.) I think players who feel like they aren't skilled want to be in a lower league because they perceive the matches would be more fair. They feel scared playing non-bronze players; they feel safer after demotion. On the flipside, players who post "where's my promotion" argue they are worthy of the next league. (The global ladder addition would make promotion/demotion more transparent, possibly solving this problem.)
A couple of positive things:
Improving to the point where one moves up to the next league is a big motivation to play competitively. I would bet cash money that going from Bronze to Silver with some fanfare is a lot more gratifying than going from say, 18% to 22% percentile. Players also like to wear their achievements; I think the HotS feature of displaying level and league on players icons in chat is a great idea.
Leagues may also help the community by making it easier for the social players (the ones that hang out in chats, and might be concerned about stigma in the first place) to find new friends or practice partners near their skill level to invite to games at a glance. (Granted, you might be seeing someone's team league, but if an icon is bronze they are most certainly bronze! -- It'd be nice if blizzard let you choose to have chat display other player's highest league, team league, or 1v1 league.) You could argue that we could show percentile number and level number instead. But would players know what to do with that information? Should a 10% player bother playing at 15%? A 20%? You'd have to educate, or everyone might get unnecessarily afraid of playing people they'd actually have a decent chance against.
Aside: If 20/20/20/20 isn't the best distribution for the lower four leagues for social (i.e. non-automated) matchmaking purposes, it could be adjusted. Proving what the "ideal" distribution might be an interesting problem.
Conclusion: I don't think one can justify scrapping leagues entirely just because some (how many?) people might feel bad about being in bronze. The harm is probably overstated, and can be overcome through other incentive structures (e.g. leveling system) discussed in this thread.
|
United States12235 Posts
This is a good post. I'm gonna pass this along when the devs get back from holiday. I did have a few notes:
- There is no "sigma threshold" for promotion. We theorized that one existed in order to explain the outliers like the 50-0 CauthonLuck, but in reality this was because of a stabilization threshold rather than a sigma one. The league boundaries are so close together (particularly for Silver through Diamond) that sigma would not be an acceptable measurement for determining promotion eligibility. To elaborate on this further, these outliers did not exist after around Season 3 when a separate rule was enacted whereby if your MMR overshoots a league completely, you get moved into that league. Since the division tier removal in Season 9, this rule may have been discarded and it appears the ladder is behaving more like it did at launch.
- You do a good job explaining the goals of the bonus pool. It also serves one other purpose not directly addressed in your counterproposal. It's functionally a decay mechanism however it's more flexible because there are no enforced deadlines. If I want to go on vacation for a couple of weeks or just take a break from SC2, I can do that in the current bonus pool system without falling behind whereas I would have suffered two weeks' worth of decay penalties if there were weekly deadlines. Also--and this could be considered either a good or a bad thing about the bonus pool--if I start the season late I get a ton of bonus pool that I have to spend in order to catch up to the active players, whereas in a decay system I wouldn't necessarily have to play as many games.
- I agree with the promotion criteria change being linked to percentiles. I imagine it already is to some extent under the current system, but there's also the stability element that sort of throws a wrench in how cleanly things operate now. I'd like improved visibility and now that division tiers are out of the picture, there's virtually no reason not to do this. You're in the top 1%, boom, up to Master league you go, where you'll stay until you fall down to 3%. If it was a mistake to promote you because you were on a lucky streak, then at least you won't be bouncing between Diamond and Master constantly because you still have to traverse that 1%-to-3% gap. The tricky part here is identifying "active players" and communicating that effectively to the entire ladder population.
- Blizzard has already shown that they are taking the ladder in a new direction with HotS. It will probably evolve further in LotV. The WoL approach was "let's put everyone into one ladder but have stuff for casuals too", while the HotS approach seems to want to separate the casuals from the competitive players via things like unranked play and leveling. I think there's still time before they feature-lock so this would be a good opportunity to implement some of your suggestions for more accurate competitive rankings.
- The point that Tiberius mentions about placement is interesting and the way a new ladder would address it varies based upon your vision for the new bonus pool. At first I thought you were in favor of scrapping bonus pool completely and enforcing a decay system like War3, but you also said that bonus pool points should be given in bulk at the beginning of a week. In any event, placement matches appear to give you a provisional rating just like the same concept in chess, where your rating is more volatile during those matches to place you as accurately as possible after only 5 games. If there were no bonus pool at all, then you couldn't have season wipes because your points are your MMR, meaning you just play one game in the new season and you're back where you were. If the bonus pool just gets wiped, you could use season wipes in much the same way they're used now.
- I agree about the Favored thing vesicular mentioned. During beta and early Season 1 it was kind of neat, where I thought "oh cool the game knows who is favored to win" kind of as a novelty (which it doesn't even actually do because it's different for both teams, but I digress). After enough games that novelty wears off and you're right: if you're favored then you better win or else, and if the other team is favored then uh oh I'm screwed, it sucks both ways. I mean, this sort of thing will happen as long as there is any visible data, so it's not just the SC2 loading screen that's intimidating. In Street Fighter if I'm a 3000PP player and I face a 1000PP player I think "I'd better not lose" but if I face a 4000PP player I think "uh oh I'm screwed", so there's pressure either way. Dota 2 doesn't tell you anything.
On December 23 2012 07:12 TiberiusAk wrote:
Also, if players felt stigma over being in bronze, then why have there been so many posts on the blizzard forums asking for demotions? (I've 2 friends that were independently terrified of being in silver when they started laddering.) I think players who feel like they aren't skilled want to be in a lower league because they perceive the matches would be more fair. They feel scared playing non-bronze players; they feel safer after demotion. On the flipside, players who post "where's my promotion" argue they are worthy of the next league. (The global ladder addition would make promotion/demotion more transparent, possibly solving this problem.)
I agree that leagues are what make the SC2 ladder what it is. They're fun and interesting and I don't think they need to go away. I would say the top complaint from lower-league players is "why am I facing Diamonds when I'm in Bronze?" Under paralleluniverse's proposal that problem disappears almost completely. You would still see small pockets of these pop up where someone is Bronze in 2v2 but Diamond in 1v1 because they tanked their 2v2 rating or are playing with someone who's new or whatever, but for the most part this wouldn't be an issue.
|
On December 23 2012 07:53 Excalibur_Z wrote:
- I agree about the Favored thing vesicular mentioned. During beta and early Season 1 it was kind of neat, where I thought "oh cool the game knows who is favored to win" kind of as a novelty (which it doesn't even actually do because it's different for both teams, but I digress). After enough games that novelty wears off and you're right: if you're favored then you better win or else, and if the other team is favored then uh oh I'm screwed, it sucks both ways. I mean, this sort of thing will happen as long as there is any visible data, so it's not just the SC2 loading screen that's intimidating. In Street Fighter if I'm a 3000PP player and I face a 1000PP player I think "I'd better not lose" but if I face a 4000PP player I think "uh oh I'm screwed", so there's pressure either way. Dota 2 doesn't tell you anything.
Good point that it's not just "Favored" vs not, it's really any comparison. In fact, I'd say you could fix a lot of this simply through language. When you queue for a match, the system could say something akin to "Searching for a player with comparable skill to yours", followed by "A match has been found!". And then nothing on the loading screen other than the player name and race.
This informs you that the player is at (or around) your skill level, making the match feel fair from the start. It doesn't even necessarily have to be 100% true, it just has to feel true. If the match making service and MMR are working properly, then it should always feel true, and that's what matters.
|
i disagree with this thread in it's entirety. It is solely because an inactive player does become rusty which therein results in a drop of skill. Keep in mind that the skill level in ladder rises as time goes by. E.g., the skill level of WOL Beta Platinum is probably equal to today's EU Bronze or Silver League.
Therefore, your proposition of improvement is flawed.
|
On December 23 2012 03:16 emythrel wrote:
I certainly see how bonus pool helps you catch up. If you are active, playing 5 games a day you will have no bonus pool to draw from for most of the season as you use it regularly. That means that most of the time your wins and losses cost you the same amount of points, someone who is less active however will earn twice the amount of points for a win than they do for a loss (in hots, you don't lose any points for a loss if u have bonus pool, it is simply removed from the bonus pool instead) which inflates their points a lot faster.
I have successfully gained top8 in my division every season by using the bonus pool i get from being inactive most of the season. While everyone else is losing and gaining points at an almost equal rate (MM system making you go 50/50) I am gaining points all the time until I use up my bonus pool, which incidentally always works out to be the exact amount of points I need to get to rank 1 or 2 in my division. Using your bonus pool up every day actually means you get less use from it because most of the time you aren't actually getting any boost from it.
false. Everyone get the same benefit from the bonus pool if they use it up. If I play 100 games a day get a certain amount of points and you wait to the end of the season and use the minimum games to use up the bonus pool we each benefit the exact same amount which is the size of the bonus pool.
The only reason you were further back is because the bonus pool was added as I played each day and you got yours at the day you 'caught up'. This is just a fake catching up that is entirely created by the bonus pool.
|
On December 23 2012 12:22 Unshapely wrote:
the skill level of WOL Beta Platinum is probably equal to today's EU Bronze or Silver League.
That's laughable at best.
|
On December 23 2012 12:49 vesicular wrote:Show nested quote +On December 23 2012 12:22 Unshapely wrote:
the skill level of WOL Beta Platinum is probably equal to today's EU Bronze or Silver League. That's laughable at best.
Indeed. Perhaps I missed your point, but how does your remark pertain to this thread?
|
I just really want the global percentile rating. That's really the only thing that matters. And yes it definitely needs to only include active players that play at least 10 games, maybe 15.
I'm not sure it would work without global servers, but it would be interesting. It should at least be implemented for server percentiles. I think it would be nice if they allowed cross server playing. This would allow for players on the American ladder to play on the Korean or European ladder and vice versa. This would be fun, helpful for serious gamers who want to learn meta game from areas around the world, and allow for an estimation of server comparisons and possibly a foundation for global percentile rankings. (ie MMR would eventually be established that 84 percentile players on the European server are comparable to 30 percentile players on Korea.
Can you imagine the awesomeness?
|
On December 23 2012 07:53 Excalibur_Z wrote:
- You do a good job explaining the goals of the bonus pool. It also serves one other purpose not directly addressed in your counterproposal. It's functionally a decay mechanism however it's more flexible because there are no enforced deadlines. If I want to go on vacation for a couple of weeks or just take a break from SC2, I can do that in the current bonus pool system without falling behind whereas I would have suffered two weeks' worth of decay penalties if there were weekly deadlines. Also--and this could be considered either a good or a bad thing about the bonus pool--if I start the season late I get a ton of bonus pool that I have to spend in order to catch up to the active players, whereas in a decay system I wouldn't necessarily have to play as many games.
You say it can be a good or bad thing that someone who joins later has a mass of bonus pool. This is exactly the problem I outlined in the "Jan 6 vs Feb 20" example in 2.4. How can it possibly be a good thing? How is it fair that Bob has to play extra games because of the bonus pool when the system already knows his MMR and sigma to be the same as Alice's?
The bonus pool doesn't prevent you from falling behind when you vacation for a couple weeks. Your rank will fall because other people have used their bonus pool. It's not a bad thing that your rank falls in this case. What is bad is that it falls even if you take 24 hours off, and it distorts the ladder as explained in 2.2. I did acknowledge the decay mechanism goal in 2.4. And my proposal does include a better "decay mechanism":
-In 3.2, I said sigma should decrease after a week of inactivity.
-In 3.3, I said that points should converge to (as opposed to being set equal to) MMR-1.96*sigma, (so points won't fall if you go on vacation).
-In 3.3, I also suggest how bonus pool can be fixed, which has the side-effect of accounting for inactivity: "she was inactive for the week, so it could be justified that her rank should fall as a small penalty for the chance that her skill has decreased due to prolonged inactivity. However, such an argument cannot be applied to the current bonus pool system because Alice would not lose any skill due to not having played for one hour or one day."
Note that this doesn't "double-count" or "double penalize" inactivity. The change that sigma should decay if you go inactive is needed regardless, otherwise you can get a "garbage in, garbage out" problem due to sigma being used in the MMR update equations not being as accurate as it could be.
When you come back from vacation, my suggestion allows you to very quickly catch up, and consume 2 weeks worth of bonus pool in a few games, because now sigma is explicitly used to account for uncertainty, so there's no need to make you play heaps of games just because the bonus pool is bad at factoring in uncertainty about MMR.
- The point that Tiberius mentions about placement is interesting and the way a new ladder would address it varies based upon your vision for the new bonus pool. At first I thought you were in favor of scrapping bonus pool completely and enforcing a decay system like War3, but you also said that bonus pool points should be given in bulk at the beginning of a week. In any event, placement matches appear to give you a provisional rating just like the same concept in chess, where your rating is more volatile during those matches to place you as accurately as possible after only 5 games. If there were no bonus pool at all, then you couldn't have season wipes because your points are your MMR, meaning you just play one game in the new season and you're back where you were. If the bonus pool just gets wiped, you could use season wipes in much the same way they're used now.
I have been preaching the benefits of decay systems for a while. But I don't think that it would be acceptable to Blizzard. So when I was writing this post, I thought up of a way to prevent stagnation and account for decay, without using a decay system.
To be clear, if we want perfectly accurate ranks and don't care a bit for psychology, I still strongly believe decay systems are superior. However, the proposed overhaul of the bonus pool system in section 3.3 is a far superior way to design a ladder system that accounts for decay, explicitly accounts for uncertainty about MMR, prevents stagnation, with the least distortion to ranks as possible. I do think that this new suggestion is on balance better than a decay system.
The idea of bonus pool being given in bulk at the start of the week and only updating the ladder once weekly, is to capture the good properties of a decay system, while still letting points trend upwards. I actually surprised myself by being able to think of so many benefits that updating the ladder once weekly would bring: it takes a snapshot when bonus pool doesn't really have a distortionary effect (paragraph 4 in 3.3), so it displays ladder ranks at the only time that they're "correct", it reduces noise, it greatly reduces the treadmill effect, it might even reduce ladder anxiety.
You can still have season wipes. The proposal was that points should converge to MMR-1.96*sigma, so in a season wipe points can still be reset to 0, or reset to some average depending on the players league. But as I explained to TiberiusAk, the idea of a season wipe makes no sense in terms of accurately ranking players, but players do expect some sort of "clean slate" at the start of a new system, so it's mostly a matter of choosing a point along this tradeoff.
- I agree about the Favored thing vesicular mentioned. During beta and early Season 1 it was kind of neat, where I thought "oh cool the game knows who is favored to win" kind of as a novelty (which it doesn't even actually do because it's different for both teams, but I digress). After enough games that novelty wears off and you're right: if you're favored then you better win or else, and if the other team is favored then uh oh I'm screwed, it sucks both ways. I mean, this sort of thing will happen as long as there is any visible data, so it's not just the SC2 loading screen that's intimidating. In Street Fighter if I'm a 3000PP player and I face a 1000PP player I think "I'd better not lose" but if I face a 4000PP player I think "uh oh I'm screwed", so there's pressure either way. Dota 2 doesn't tell you anything.
Actually, Blizzard must know who's more likely to win, because this information is needed in the MMR update equation:
posterior distribution of player 1's skill = P(Player 1 wins | player 1's skill)*P(player 1 skill) / normalizing constant.
So P(Player 1 wins | player 1's skill) must be known in order for them to do the calculations. In fact, in TrueSkill, the formula is quite simple:
P(Player 1 wins | player 1's skill) = Phi((mu1-mu2)/sqrt(2*beta^2 + sigma1^2 + sigma^2)), where mu is mean skill, beta is a constant that depends on the type of game, and Phi is the normal cumulative distribution function
But, yes the favored system should be scrapped. It's unnecessary. They should just stop showing it.
|
On December 23 2012 14:29 theinfamousone wrote:
I just really want the global percentile rating. That's really the only thing that matters. And yes it definitely needs to only include active players that play at least 10 games, maybe 15.
I'm not sure it would work without global servers, but it would be interesting. It should at least be implemented for server percentiles. I think it would be nice if they allowed cross server playing. This would allow for players on the American ladder to play on the Korean or European ladder and vice versa. This would be fun, helpful for serious gamers who want to learn meta game from areas around the world, and allow for an estimation of server comparisons and possibly a foundation for global percentile rankings. (ie MMR would eventually be established that 84 percentile players on the European server are comparable to 30 percentile players on Korea.
Can you imagine the awesomeness?
Yes, this is a very good idea, which Blizzard should aim for. However, when I say "global" ladder, I mean a ladder for everyone on the server, as opposed to everyone in the world.
There's actually a bunch of research debating (particularly for chess) whether it's valid to use ELO, TrueSkill, MMR, etc, to compare the skill of players from different eras. Comparing ranks across the world faces the same problem, due to accounts being separate for each server.
|
On December 23 2012 12:22 Unshapely wrote:
i disagree with this thread in it's entirety. It is solely because an inactive player does become rusty which therein results in a drop of skill. Keep in mind that the skill level in ladder rises as time goes by. E.g., the skill level of WOL Beta Platinum is probably equal to today's EU Bronze or Silver League.
Therefore, your proposition of improvement is flawed.
What?
1. Inactive player loses skill.
2. ???
3. Therefore, everything paralleluniverse said is completely and utterly wrong.
Does not compute.
How does this change the validity of anything I've said?
|
On December 23 2012 07:12 TiberiusAk wrote:Show nested quote +Do solution 2, but scrap the idea of leagues. As Apolo explains, Bronze league is just a big a "I suck" sticker which Blizzard forces 42% of players to wear on their heads. It doesn't help people suffering from ladder anxiety, it hurts them instead. They still might want to put a league icon next to a player's name, based on their global ladder percentile, but they should consider possibly ditching all fanfare associated with promotion and demotion and labeling people. I disagree with scrapping leagues. I think Apolo's post may overestimate the harm and ignoring the benefits of leagues (though he was talking about hiding, not removing them). What evidence is there that bronze players actually feel this way? How many of them do? I don't hear many people saying that ICCUP should have scrapped their D/C/B/A system because people felt stigmatized at being a "D" player. I think players are emotionally more robust than that--players accept that in a competition, there is going to be a bottom level. Also, if players felt stigma over being in bronze, then why have there been so many posts on the blizzard forums asking for demotions? (I've 2 friends that were independently terrified of being in silver when they started laddering.) I think players who feel like they aren't skilled want to be in a lower league because they perceive the matches would be more fair. They feel scared playing non-bronze players; they feel safer after demotion. On the flipside, players who post "where's my promotion" argue they are worthy of the next league. (The global ladder addition would make promotion/demotion more transparent, possibly solving this problem.) A couple of positive things:Improving to the point where one moves up to the next league is a big motivation to play competitively. I would bet cash money that going from Bronze to Silver with some fanfare is a lot more gratifying than going from say, 18% to 22% percentile. Players also like to wear their achievements; I think the HotS feature of displaying level and league on players icons in chat is a great idea. Leagues may also help the community by making it easier for the social players (the ones that hang out in chats, and might be concerned about stigma in the first place) to find new friends or practice partners near their skill level to invite to games at a glance. (Granted, you might be seeing someone's team league, but if an icon is bronze they are most certainly bronze! -- It'd be nice if blizzard let you choose to have chat display other player's highest league, team league, or 1v1 league.) You could argue that we could show percentile number and level number instead. But would players know what to do with that information? Should a 10% player bother playing at 15%? A 20%? You'd have to educate, or everyone might get unnecessarily afraid of playing people they'd actually have a decent chance against. Aside: If 20/20/20/20 isn't the best distribution for the lower four leagues for social (i.e. non-automated) matchmaking purposes, it could be adjusted. Proving what the "ideal" distribution might be an interesting problem. Conclusion: I don't think one can justify scrapping leagues entirely just because some (how many?) people might feel bad about being in bronze. The harm is probably overstated, and can be overcome through other incentive structures (e.g. leveling system) discussed in this thread.
A part of the problem with ladder anxiety is "where's the evidence for anything?". Why doesn't Blizzard do random surveys and research on ladder anxiety given that the whole ladder system design seems to have been driven by it? Here I'm assuming there's no research because I haven't heard of anyone ever getting surveyed by Blizzard, and there doesn't seem to be any other way to conduct large-sample ladder anxiety research (but maybe they have secret research which we don't know about).
So a lot of the arguments on this issue are just mostly based on reading what people are writing and just using "common sense" (which isn't always so reliable). Your comment about getting demoted to get easier queues and people demanding promotions highlights a major problem with the SC2 ladder system: most people have no idea how it works. I've also flagged this as a big problem, and Blizzard's attempts to explain it have been a very poor.
I agree with you that there are some benefits to having leagues and you've outline those. I didn't write that leagues should be removed in the OP because I thought it would be a very radical change and unlikely to happen. And I'm not sure whether the benefits are greater than the cost.
|
On December 23 2012 07:53 Excalibur_Z wrote:
Blizzard has already shown that they are taking the ladder in a new direction with HotS.
I don't see it.
I see that Battle.net has very very significantly improved. But the ladder system is virtually identical to WoL.
|
On December 23 2012 07:53 Excalibur_Z wrote:
- I agree about the Favored thing vesicular mentioned. During beta and early Season 1 it was kind of neat, where I thought "oh cool the game knows who is favored to win" kind of as a novelty (which it doesn't even actually do because it's different for both teams, but I digress). After enough games that novelty wears off and you're right: if you're favored then you better win or else, and if the other team is favored then uh oh I'm screwed, it sucks both ways.
I didn't have a problem with this in the early days when it happened very rarely, but now with the wider match search it is as you say.
|
Allow me to make your comprehend.
The bonus pool perpetually inflates the points of more active players, which causes your rank to fall and continually requires you to play on a daily basis, even to maintain the same position. This creates a "treadmill effect”. If Alice and Bob are equally skilled and Alice becomes less active, then her points will diverge from Bob's.
I would like this system to remain unchanged - read my reasoning.
Keep in mind that the skill level in ladder rises as time goes by.
So, I have two points. Firstly, Alice loses her `groove.' Secondly, there is potential for the skill level of the entire ladder to rise. According to your article, the bonus pool doesn't serve it's purpose. I think it does, because if Bob has been playing in Alice's absence, then there is potential for him to improve and become better within this duration. He is learning new tactics while alice's quick hands are not so quick anymore. Nota bene: every company would like their player base to be as active as possible. I do not see any problem with active players being rewarded for keeping the game alive and thriving.
With the inclusion of the leveling system that rewards all types of players, the issue you're trying to address doesn't really bother casuals as much as you claim it does. Therefore the system is fine as it is.
Now, about global ladder. Blizzard has announced that they'll make it possible for people from Europe to be able to play on American and Korean servers (and vice versa). It is a bit unclear how this will be implemented, because I'm almost certain that there will be a variance in latency. I'm hoping for this to be implemented separately, with a separate set of statistics which will enable us to directly compare global statistics to that of our own continent. I must first witness the change before I can criticise it.
remove the GM league
I do not recommend that GM league be removed. It is the hallmark of the greatest achievement in your continent. It not only gives you bragging rights, but also gives you a sense of mental euphoria. I am not against the proposal you made for possible technical corrigendums in the algorithm used, but rather the idea of removing GM altogether.
if a player during the middle of the season becomes more skill than the players in the top 200, than he will be unable to enter the GM league
Then why have seasons at all? Shouldn't there be some significance of having different seasons, aside from a different map pool?
Ladder Anxiety
This is the point where, perhaps, most people would disagree with me. Be it as it may, I find it absurd that certain people feel anxiety from playing a video game. Perhaps it is because I never felt the same way when playing Starcraft.
Blizzard should remove the division ranks,
I like the current system better because it indirectly relates to the ranks in real-world. Just like we have a Soldier, Major, General, etc. People are not ranked upon numbers. You're basically trying to bring the ICCup/Fish like points from Brood War right? I fail to see how this would make a difference, because people would inevitably classify a set of points into a rank.
Exempli gratia,
1-100 (Bronze)
100-200 (Silver)
200-300 (Mid Silver)
...
1000-1500 (Mid Masters or whatever)
So on and so forth. It wouldn't bring about the change you're hoping it otherwise would. Because the rank mentality will remain.
Some of your points are quite general, as people have been wanting to know the correct system used to determine ranks for a long time. You seem to view the bonus pool as a big problem, whereas I don't. This is where we differ. Your proposal for improving the algorithm is better served at Battle.net forums because the likelihood of a developer coming across it would be greater.
I admit, I did not read the rest of your article. I see no problem with the last two points you made. Let me know if I missed anything worthwhile.
|
On December 23 2012 17:42 Unshapely wrote:Allow me to make your comprehend. Show nested quote +The bonus pool perpetually inflates the points of more active players, which causes your rank to fall and continually requires you to play on a daily basis, even to maintain the same position. This creates a "treadmill effect”. If Alice and Bob are equally skilled and Alice becomes less active, then her points will diverge from Bob's. I would like this system to remain unchanged - read my reasoning. So, I have two points. Firstly, Alice loses her `groove.' Secondly, there is potential for the skill level of the entire ladder to rise. According to your article, the bonus pool doesn't serve it's purpose. I think it does, because if Bob has been playing in Alice's absence, then there is potential for him to improve and become better within this duration. He is learning new tactics while alice's quick hands are not so quick anymore.
As I've already explained in the OP:
"In that example, Alice is more skilled than Bob. She doesn't play for a day and falls below Bob's rank as a result of Bob's bonus pool. So the ladder ranks have become wrong. Now if the ladder were to update only once weekly and bonus pool were changed as I've suggested, then Alice would be able to get back ahead of Bob, before the next ladder snapshot. If she didn't, it would be because she was inactive for the week, so it could be justified that her rank should fall as a small penalty for the chance that her skill has decreased due to prolonged inactivity. However, such an argument cannot be applied to the current bonus pool system because Alice would not lose any skill due to not having played for one hour or one day. The skill lost for 2 weeks of inactivity is far more than 14 times the skill lost in 1 day of inactivity."
While players may lose skill for not having played for a week, they don't lose skill for not having played for a day. In fact, the sleep and rest should make them play better. Yet the bonus pool still penalizes them for inactivity on an hourly basis, causing ranks to be distorted. It creates a treadmill effect where your rank continually falls each day you log in.
Nota bene: every company would like their player base to be as active as possible. I do not see any problem with active players being rewarded for keeping the game alive and thriving.
Ask yourself why activity should be rewarded instead of just taking it as an article of faith.
There are 2 possible reasons:
1. As a proxy to uncertainty about MMR. In the above example Alice may have lost some skill, we're not sure so we would like to take that possibility into account.
2. To encourage players to play the game.
On 1, Blizzard should directly use the uncertainty about MMR, sigma, instead of using the bonus pool as a way to indirectly estimate uncertainty about MMR. And it isn't even an accurate proxy as shown by my "Jan 6 vs Feb 20" example in 2.4. How can you explain this example using an "inactivity causes loss of skill argument"? You can't. What Blizzard's bonus pool does is like estimating the temperature of the room indirectly by using air inflows and outflows and the rate at which a bowl of hot water cools down. Whereas what I'm proposing (making points converge to MMR-1.96*sigma) is like measuring the temperature of the room directly by looking at the thermometer.
On 2, encouraging people to play should be achieved using a leveling system that gives cosmetic rewards. Doing it through the bonus pool distorts ranks in ways that cannot be explained by loss of skill, delays points from self-correcting when the player has legitimately loss skill (as shown by the examples in 2.2) and fails to correctly account for uncertainty about MMR (as shown in the "Jan 6 vs Feb 20" example in 2.4).
With the inclusion of the leveling system that rewards all types of players, the issue you're trying to address doesn't really bother casuals as much as you claim it does. Therefore the system is fine as it is.
What are you talking about? What issue? The leveling system is for rewarding activity. My point is that the bonus pool is not needed for this goal since the leveling system already does it.
Now, about global ladder. Blizzard has announced that they'll make it possible for people from Europe to be able to play on American and Korean servers (and vice versa). It is a bit unclear how this will be implemented, because I'm almost certain that there will be a variance in latency. I'm hoping for this to be implemented separately, with a separate set of statistics which will enable us to directly compare global statistics to that of our own continent. I must first witness the change before I can criticise it.
It is crystal clear how this will be implemented. Blizzard has continually said that you'll get a completely and absolutely new account for each region. Nothing is shared between regions.
But that's not what I mean by a global ladder. I'm talking about a ladder that ranks all active players on the server, i.e. all North American players on 1 ladder, instead of hundreds of meaningless division ladders.
I also have no idea where you quote-mined the statement: "Blizzard should remove the division ranks,". That phrase doesn't appear in my OP. The closest thing I said is that ranks should be taken from the matchmaking page and score summary page and put into the ladder page within the profile, in order to reduce the treadmill effect and to maximize the positive psychology associated with points that trend upwards.
I do not recommend that GM league be removed. It is the hallmark of the greatest achievement in your continent. It not only gives you bragging rights, but also gives you a sense of mental euphoria. I am not against the proposal you made for possible technical corrigendums in the algorithm used, but rather the idea of removing GM altogether. Show nested quote +if a player during the middle of the season becomes more skill than the players in the top 200, than he will be unable to enter the GM league Then why have seasons at all? Shouldn't there be some significance of having different seasons, aside from a different map pool?
I said that the GM league should be removed, if there is a global ladder. If all active players are on a single ladder, from 1 to 40,238, then what's the point of GM league? What additional information does it provide? None, because everyone, not just the top 200, are ranked together.
You call the GM league the greatest achievement. But you fail to understand that the design of GM is flawed. Once you get in the GM league you don't leave unless you're inactive. You can't get into the GM league unless someone leaves. So why have a GM league at all when it doesn't even give the top 200 players at the current moment. It gives the top 200 players at the start of the season who have remained active. The GM league does not reflect current information about the top 200, hence there are people in GM who shouldn't be, and there are people who should be in GM but aren't.
Information on how GM league works: http://us.battle.net/sc2/en/blog/2452060
This is the point where, perhaps, most people would disagree with me. Be it as it may, I find it absurd that certain people feel anxiety from playing a video game. Perhaps it is because I never felt the same way when playing Starcraft.
You can find it absurd all you want. To deny that it exists is to ignore reality. Why do you think Blizzard hid losses in WoL? Because of ladder anxiety.
I like the current system better because it indirectly relates to the ranks in real-world. Just like we have a Soldier, Major, General, etc. People are not ranked upon numbers. You're basically trying to bring the ICCup/Fish like points from Brood War right? I fail to see how this would make a difference, because people would inevitably classify a set of points into a rank. Exempli gratia, Show nested quote +
1-100 (Bronze)
100-200 (Silver)
200-300 (Mid Silver)
...
1000-1500 (Mid Masters or whatever)
So on and so forth. It wouldn't bring about the change you're hoping it otherwise would. Because the rank mentality will remain.
I wasn't aware that there's a real world rank called #47 or #18. You're confusing "league" with "division ranks". I suggest you read up on the current ladder system: http://wiki.teamliquid.net/starcraft2/Battle.net_Leagues
I said that division ranks are meaningless, not leagues. You haven't addressed any of the substance of my reasoning on why division ranks are meaningless in 4.1. Moreover, I didn't say that division ranks should be removed, in fact, I argued that they should stay: "This does not necessarily mean that the division ladders should be removed, just that division ranks are meaningless. Both global and division ladders can coexist."
Where are you getting this idea that I'm trying to bring back Iccup? I said there should be a global rank as a percentile, i.e. if you're in the top 4% of all active players on the server, you get a percentile of 96, i.e. you're better than 96% of all active players on the server.
If you don't believe in ladder anxiety, then what is your problem with my suggestion of adding a global ladder.
Some of your points are quite general, as people have been wanting to know the correct system used to determine ranks for a long time. You seem to view the bonus pool as a big problem, whereas I don't. This is where we differ. Your proposal for improving the algorithm is better served at Battle.net forums because the likelihood of a developer coming across it would be greater.
I admit, I did not read the rest of your article. I see no problem with the last two points you made. Let me know if I missed anything worthwhile.
It appears that you didn't read much of my post as you've misinterpreted most of what I said, while having ignored sections showing that your arguments are wrong and that bonus pool is a major problem. Moreover, you haven't adequately addressed why my reasons against the bonus pool are wrong and how my changes to the bonus pool system would make it worse. No one else who has replied to this thread has so misunderstood what I've written.
|
Loss aversion: decay system vs bonus pool
An interesting point to consider is that humans are psychologically loss adverse, i.e. the hurt caused by losing $1 is greater than the happiness caused by getting $1. This very recent article (not political) makes some points about this fact: http://www.bloomberg.com/news/2012-12-24/people-hate-losses-and-that-affects-u-s-budget-talks.html
Although the discussion in the article looks at motivation, rather than "fun", it would seem to imply that a decay system is a greater motivator than a bonus pool system. I still think my revamped bonus pool system is probably overall better than a decay system, which in turn is light years better than the current bonus pool system, but this is interesting food for thought.
Division system in a global ladder
If a global ladder were implemented, the division system could have great potential:
1. Divisions could be a mini-ladder with players in your skill range (i.e. division tiers without the point modifiers).
2. Divisions can be completely divorced from leagues (if you change league, you don't change division).
3. Divisions could even be a "create-your-own ladder" with friends and others.
Under the current system:
1 and 2 are not possible since it would imply that getting to the top of division ladder would no longer be a (crude) approximation to being promoted.
3 is not possible because there would be no real ranking system at all.
So if there's a global ladder to handle "serious" ranking, the division system could be freed up to be more fun and welcoming. It could be a system that allows you to make your own division to compete with friends. This could even be tied into the group and clan feature. And it could also be a system where you can compete with 100 players of near equal skill. If a tournament system were added, good players would always win. But if divisions had tournaments, it would allow the average player to win, despite the fact that such a tournament win can't really be taken seriously. But that would be OK, because the global ladder is for the more serious business.
|
I just want to add that, imo, the biggest factor that makes people stop laddering is fear of demotion.
When your MMR is in-between 2 leagues and you manage to get a promotion to the higher league, you have all reasons to stop playing right there.
The fact that many people don't realize that the threshold for demotion is lower than the threshold for promotion doesn't help.
Just picture this example for a second:
- you've been recently promoted to plat, a long term goal! It took you so much effort to get there, but people now finally respect you a little! And, of course, you told all your friends about it.
- Problem is now you're on a losing streak, so you start worrying about demotion to gold again.
- You have no idea how close it could be... how do demotions work anyway? You have no idea... but you have a feeling it could be next loss or 2.
- So what do you do? You stop laddering to keep your precious plat icon.
(Note: I don't mean everyone is like this)
Also, the whole ladder system is obscure and not knowing / being in control of things make people uncomfortable and anxious.
This is why I actually think a very good measure would be to make MMR visible.
What would this achieve?
The social importance of being in a certain league would decrease a lot, as people would be focusing more on each other's MMR, so players wouldn't worry so much about demotion.
In terms of MMR, any loss is just another 10-20 points you loose. Not a big deal. The loss aversion remains a lot more constant, instead of reaching huge proportions near demotion.
Also, everyone would understand how ladder works a lot better, which is good.
I feel that Blizzard doesn't want people asking questions like: 'why does this guy with -100 MMR is in a higher league than me?' But making things clear is always better. In chess, GMs having lower ELO than masters happens all the time and no one complains about that, because people understand the system.
Also, I believe MMR is a constant motivation to play. Instead of just having very distant league promotions, you can set a number of intermediate goals for yourself: reach 1500, then reach 1600, etc...
I've had a chess ELO since I was a kid and chess players love their ELOs and are constantly working to improve them for the next 50 or 100 points.
This little number is best motivation one can have to play.
|
On December 22 2012 21:06 paralleluniverse wrote:Show nested quote +On December 22 2012 06:50 Treehead wrote:
On Ladder Anxiety:
I'm sorry to say that you've missed the point a bit on ladder anxiety in 5.1-5.3 (though I like the inclusion of team games, though, that would do a ton). People aren't afraid to queue because they see and understand the potential volatility in their ladder ranking, or their points, or whatever. People are afraid to queue for a much more basic reason - they're afraid they'll do something which makes them look silly.
It's like this - you can explain to someone that by asking a girl to a dance, they can't actually possibly lose anything except the indifference of a girl who's probabilistically unlikely to date you anyway, unless you count the fact that you could get married to said girl and she could rule your life with an iron fist, leaving you a crushed and hopeless shell of a man who has to ask permission to use the bathr... oh look, it appears I've wandered - but anyway, the point of going with said girl in the first place is to initiate such a relationship, so I wouldn't think one should count that. And you can explain that if for some reason said girl does decide to go to the dance with you, it could be a great boon to the rest of your life.
But explaining the logic of the matter doesn't deal with the anxiety. If it promotes an action one anyone's part, psychologically speaking it isn't generally because you've dealt with the anxiety, but rather because you've created a social situation in which people feel silly for being afraid of feeling silly - you can see the possible issues this might present people. But really, any "tutorial" about how someone is supposed to feel is most likely to be ignored or misunderstood anyway.
The only ways I know of to deal with anxiety is through practice and discipline, through medication, or through evasion (hence my like of team games, where everything wrong can always be someone else's fault). The impression that I usually get from reading threads on this forum is that it's more about promotions and demotions and having bad losing streaks than it is about what's happening in the actual game. If the cause of ladder anxiety is that people are afraid of getting hellion harassed or getting killed by a cannon rush or annihilated by a protoss deathball after spending 20 minutes in game, getting outmatched by zerg etc, then there really isn't too much that game designers can do other than dumbing down the game, which should definitely not happen. I don't think there is any one cause of ladder anxiety or any one solution. It's probably impossible to completely fix, which is why I've made several suggestions to help reduce it. As for the tutorial, I think it's a good idea not because it tells people how to feel, but rather because it would explain to them that MMR is self-correcting. It's something everyone should be told. How they feel is up to them. It obviously won't be a silver bullet. Throughout WoL, the awful design of the ladder system seemed to have been completely driven by ladder anxiety, hiding losses, making points incomparable, refusing to implement a global ladder etc. Yet Blizzard doesn't seem to realize that the league system and the bonus pool system, while not the direct cause of ladder anxiety, still makes the problem worse. What do you suggest?
The fact that you're reading threads on ladder anxiety and taking what is written there at face value exemplifies the problem. Take said example with asking a girl to the dance - assume guy asks girl to dance and gets rejected. Is he likely to respond to the anxiety itself? Is he likely to think with a cool and logical mindset about it (and just say "oh well, it happens - I'll find someone else" without feeling bad)? Or is he more likely to find random negative things about himself or his potential date and attribute the blame for his rejection entirely on a small set of likely irrelevant factors?
If you said the third option, you've probably been rejected by a girl before. If you ask this person what the problems were which led to this rejection, he'll likely complain about her, himself, or the environment in which this occurred. That is what these people are doing. They're taking something arbitrary (the inner workings of the promotion and demotion system, which they usually do not understand, or balance which they understand even less, or maybe even the nature of the game, or people, etc.), and attributing the entirety of the cause for their anxiety on these things.
You ask what I suggest doing for anxiety? I have a few ideas, but on the whole overcoming anxiety is more about a person's awareness than it is about those around them.
1. Training - people on the ladder hate losing because they're not used to it and they're not sure what to make of it. They expect to win because "they're awesome", and much like inertia, they'll remain in this state of mind unless you break them of it. The simplest and most straightforward way to do this - make something that'll beat them a lot of the time. You have to make it beatable, but you have to make it hard. Then you have to give them incentives to go back over and over again. Then you have to somehow make it fun.
Concretely, make an AI trainer that actually is capable of playing like a normal player would and makes decisions (if only randomly) like another player. Make this AI notably better than each player, but not so much better that it leaves them no chance to win. Reward them with wins against these bots by giving them points, but don't change their MMR. The points awarded should roughly be the same as point inflation (which may or may not fade away).
This gives people a way of playing and improving and most importantly getting used to losing in an environment free of judgment, but is probably not the best option because such an AI would likely be impossible to code correctly.
2. Give them external factors to blame - these can be chance factors (as in card games), or it can be other players (you already mentioned doing this a la WC3-like multiplayer).
As an alternative to the standard multiplayer, I'd propose a new kind of multiplayer where you can simply have multiple players controlling the same base. This accomplishes a couple of key things. First, it provides an environment where 3 or 4 people can team up and work together to beat their opponent(s). Second of all, it gives us an environment that we are already trying to balance so that no additional balance work needs to be done to make multiplayer "balanced". Third, and perhaps most importantly of all, it makes the multiplayer that people stick around for still related to the pro scene - so that players who are into 2v2, 3v3 and 4v4 are still able to look to 1v1 pros to discover build orders, micro techniques and compositions.
This is infinitely more achievable that 1 above - but I still doubt they'll do this.
3. Give the people reasons they lost at the end of the game. These do not need to be terribly overt, but some form of measuring a players macro, micro, strategic thinking and multitasking would give each player something to work on after each loss. When you lose to a cloaked banshee early on before you have an observer, the lesson is obvious, and people alter their build orders to make an earlier observer. But often times, the reason people lose are less obvious. Maybe they missed a bit of micro or got supply blocked for a long period of time, while their opponent did not. By rating things like micro, multitasking and macro, we can get an idea of why our opponent was better than we were and it gives us something to improve on in our next game. When we don't know what went wrong, it can be a frustrating experience.
Let's look at another analogy. When a child is about 2 or 3 years of age, they begin to purposefully get into trouble in a period known by most parents as the "terrible twos". In general, they do this to determine precisely what it is that parents expect of them. By giving them repeated punishments or reprieval of privileges when they do certain activities, a child learns that these activities are things they are *not* expected to do. Through repeated rewards or removal of negative consequences, a child learns that these activities are things they are expected to do (and praised for doing). It is when these punishments and rewards get mixed that we see anxiety. A child may be momentarily happy that he is rewarded despite performing a behavior which previously earned punishment, but the overall effect is uncertainty and anxiety as to whether the action he is performing is appropriate or not.
Now, I'm not saying we're all 2-year-olds, but at some level human beings learn behaviors similarly throughout life. If you win some games and lose others while appearing internally to be doing the same thing, the result is that you feel uncertain about if the way you are playing is good or bad - creating anxiety. Especially amongst people with poor game sense where the reason they lost feels unknown, giving a person ratings compared to their opponent on different things gives them concrete things they need to be doing better instead of the generic "win or loss" mentality, we can reward their macro while punishing their micro, or reward their use of strategy while punishing them for their multitasking.
The biggest hurdle here is actually measuring these things in a way that is both meaningful and will very seldomly produce a result where the winning player actually played worse in all categories than their opponent.
|
United Kingdom36161 Posts
I don't have much deep to add, I just read it because it was interesting and made good points. I'm a fairly casual low masters player, I just play a few games a week. The one suggestion I didn't like was only updating the ladder weekly. Even though I appreciate it's pointless, I like seeing my points go up and down as I play (even knowing that with my bonus pool and the bonus people of other people in my division, the ranks are mostly meaningless). Basically what I'm saying is that I like immediate feedback when I play games, and it would irritate me not to have it, however meaningless it is ^^
|
I honestly wish we hada true global ladder. I mean i really dont even care about sliver/gold/plat. I jsut want to know where i'm at without having to go to sc2 ranks. idc if its ever 400,000/4 million or whatever.
|
@OP:
This is an interesting post, and you obviously put a lot of thought into it. I would like to respond to certain points about it, but have a question for you, first, since you seem to understand TrueSkill better than most I've been able to talk to.
Halo Wars, an RTS created by Microsoft Game Studios for Xbox 360, used the TrueSkill rating system for online ladder. The developers linked to the same Microsoft research article that you linked to in your OP and claimed that it was this implementation. I would assume that if TrueSkill was to be implemented correctly anywhere, it would probably be in a Microsoft Game Studios game for Xbox 360.
This rating system had serious problems for 1v1 gameplay, and was widely regarded as 'completely unusably horrible' for team play. To give you an example of some of the 1v1 problems, I'll just share my experience:
---------------------------
At release, the best player in the game (a game balance tester who started with a bit of an advantage) reached a TrueSkill (TS) rating of 48, and then quit. He was #1 on the ladder until the ladder reset months later, unable to be surpassed by anyone. Since the system was not well-understood, people just continued to play games, and were not sure why he couldn't be surpassed. The best players reached TS 47 but could not increase. I was one of those players.
Then a ladder reset came. Everyone started playing games, and after around a week, people couldn't get above TS 46. Nobody understood why. About a month later, I created a new account out of boredom, and hit up the 1v1 ladder. I rocketed to TS 48 in 17 games. Other top players saw the same and created new accounts as well, and met me at TS 48.
We started noticing interesting (read: bad) characteristics of the TrueSkill system. One such characteristic was that if you played a player with significantly lower TS than you (let's say 48 against 37), and you won the game, you would actually lose TrueSkill. Sometimes you would drop from 48 to 47. Someone found a bug that allowed you to view your exact TrueSkill rating (out to many decimal points, so for example, 47.271642), after which we could confirm that playing a player below your skill level definitely made you lose TrueSkill 100% of the time, win or lose (if you lost, your TrueSkill would tank).
Another characteristic was that it had a very, very strong weighting component. If you were TS 48, and you played 150 games, then you aren't getting to 49 TS, ever. Beating an equally-ranked opponent at 48 will net you around 0.002 TS points. If you've played 14 games, it will net you about 0.1, a 50-fold difference.
Everyone began to race for TS 50, the end-accomplishment on the solo ladder. Many people didn't think it was even possible to do. I made this my goal. The game became "get a new Xbox account, play 10-16 games, if you lose any games or if you play anyone bad and win after ~16 games, you quit and restart". Losing more than a few games was an insurmountable challenge because your TS became nearly impossible to move. Beating a bad opponent was the same thing as losing.
My second account was un-tainted by losses or games with bad players, so I stayed on it, playing judiciously. I would add all the top players to my friends list, see when they were searching for games, and snipe them. This kept me from playing anyone bad and weighting my account, and somewhat hilariously, made it impossible for them to ever improve, forcing them to create another account. After about two months or so, I was the first to hit TS50, and quit (playing anyone else at that point, even if I won, would drop me back down to 49. This is the silliest thing ever).
Following this, people figured out that you can just boost to 50+ by getting two accounts to ~45 TS, which is easy to do because of the weighting factor. Then you attempt to get matched up with eachother by searching at the same time and forfeit. Months later, there were 5-6 TS 50s from this (even though it's essentially cheating).
---------------------------
As you can see, the rating system became the focal point of the game, and you had to game it in order to progress on the ladder.
I'm a bit tired of typing to write a full experience with the team TrueSkill system, but just trust me when I say that it's unplayable. My team had a 118-0 record (yes, seriously) on the in-game leaderboard. We were ranked 956th or something with 42 TS. The #1 team was like 30-10 with 49 TS, because they learned that the TrueSkill system gives insane gains when you pair a highly-ranked player with lowly-ranked players and intentionally exploited that fact. The teams in the between unintentionally exploited that fact.
Your endorsement of the TrueSkill system, and inclination to move Starcraft's system closer towards it, makes me question how good of an idea these suggestions would be. If Halo Wars' implementation of the TrueSkill system is any indication, implementing TrueSkill would literally ruin the Starcraft II ladder. I've laddered competitively on Starcraft II as well as an RTS with TrueSkill and I can say with confidence that Starcraft's ladder system is superior.
Do you have personal experience with a system where TrueSkill works well?
|
I would have to say that bonus pool is fine where it is. This is because matchmaking is made to place people with you who are equal in skill, resulting in a ~50% winrate. If bonus pool didnt exist, everyone would have 0 points, except for smurfs, and people who are on the verge of promotion. While you are right in that it does not help inactive players, creating a treadmill effect, it, instead of helping them, makes them play more. It is more of an incentive than an actual tool in helping inactive players.
And yet, this treadmill effect is still a problem. I have to play to keep my rank, because someone else played. I definately think the best way to rank someone is to reveal MMR and rank accordingly. Eg. a higher mmr player is higher in his division. Of course, there is an uncertainty value. The MMR should subtract the average of the upper and lower bounds of a player's uncertainty. So that way, if a player will occasionally play higher than his mmr more often than he plays lower than his mmr, his mmr rises slightly. And if a player's uncertainty shifts to a tendancy to play less than their mmr, he loses some mmr. If blizz wants to hide ppls mmr to hide their patented algorythms, they simply dont reveal the uncertainty, and the mmr algorythm is impossible to decode.
Another idea i have is just to make active players have no bonus pool. Bonus pool only accrues after 2 days of ladder incativity. If a person is active, he instead loses less points for losing a match, equal to the number of points the game is worth + or - the number of consecutive active days. and wins more points for winning a match. This means that inactive players can catch up, and that the ability to gain points along with mmr gets rid of the need for a treadmill effect.
This way:
- a person cant log In once a day to farm his bonus pool and then get off.
- Active players are rewarded.
- Inactive players can catch up, but the playing field is leveled once they are caught up.
(edited to avoid double post with second idea)
|
I am extremely impressed by how well written the OP is and how extensively paralleluniverse answers all questions and objections. I just read through everything very thoroughly, looking up hard words like "converge" and working hard to understand all the math and statistics.
All the posts have been impressing too, except for one which was so unclear it caused a laugh attack-welcome in all the seriousness.
I do not actually have anything to contribute with but felt that I had to share my appreciation for all this. The work put into making ladder a better experience for all and the thread with the best written OP and highest overall quality of posts I have ever seen on TL.
Thank you.
|
On December 31 2012 05:23 Azoryen wrote:I just want to add that, imo, the biggest factor that makes people stop laddering is fear of demotion. When your MMR is in-between 2 leagues and you manage to get a promotion to the higher league, you have all reasons to stop playing right there. The fact that many people don't realize that the threshold for demotion is lower than the threshold for promotion doesn't help. Just picture this example for a second: - you've been recently promoted to plat, a long term goal! It took you so much effort to get there, but people now finally respect you a little! And, of course, you told all your friends about it. - Problem is now you're on a losing streak, so you start worrying about demotion to gold again. - You have no idea how close it could be... how do demotions work anyway? You have no idea... but you have a feeling it could be next loss or 2. - So what do you do? You stop laddering to keep your precious plat icon. (Note: I don't mean everyone is like this) + Show Spoiler +
Also, the whole ladder system is obscure and not knowing / being in control of things make people uncomfortable and anxious.
This is why I actually think a very good measure would be to make MMR visible.
What would this achieve?
The social importance of being in a certain league would decrease a lot, as people would be focusing more on each other's MMR, so players wouldn't worry so much about demotion.
In terms of MMR, any loss is just another 10-20 points you loose. Not a big deal. The loss aversion remains a lot more constant, instead of reaching huge proportions near demotion.
Also, everyone would understand how ladder works a lot better, which is good.
I feel that Blizzard doesn't want people asking questions like: 'why does this guy with -100 MMR is in a higher league than me?' But making things clear is always better. In chess, GMs having lower ELO than masters happens all the time and no one complains about that, because people understand the system.
Also, I believe MMR is a constant motivation to play. Instead of just having very distant league promotions, you can set a number of intermediate goals for yourself: reach 1500, then reach 1600, etc...
I've had a chess ELO since I was a kid and chess players love their ELOs and are constantly working to improve them for the next 50 or 100 points.
This little number is best motivation one can have to play.
Excellent point. Paralleluniverse makes a great case for the competitive player. And its a point to be made now that non-competitive options are available (unranked play and arcade). There are still remnants of the ladder casuals (like me) who have suffered in the hands of the current system and would like to rest on their laurels rather than face the facts that:
1) Inactivity decays your skill,
2) The ladder average gets better,
3) Maintaining your platinum/diamond symbol can be difficult as you might not have deserved those wins fully (good times for your race, lucky anti-meta build, luck...) if you're not focused on playing sc2 enough, it's better to not play at all...
But despite this reality, full speed ahead, Pallaleluniverse. I hope Blizz listens to you.
|
I think there should be an MMR instead of divisions. EX 3000MMR diamond with a visible "instability factor" and a clear line to show when you get promoted or demoted.
Then make all MMR slowly retrograde over time so if you don't play you will slowly get lower MMR. This encourages occasional playing. Overall i'd like to see that system rather than a meaningless 100 random person division and bonus pool.
TBH bonus pool is okay but we really need a universal ranking instead of devisions :/ we are separating casuals from hardcore players anyhow.
|
On January 02 2013 03:50 iEchoic wrote:
@OP:
This is an interesting post, and you obviously put a lot of thought into it. I would like to respond to certain points about it, but have a question for you, first, since you seem to understand TrueSkill better than most I've been able to talk to.
Halo Wars, an RTS created by Microsoft Game Studios for Xbox 360, used the TrueSkill rating system for online ladder. The developers linked to the same Microsoft research article that you linked to in your OP and claimed that it was this implementation. I would assume that if TrueSkill was to be implemented correctly anywhere, it would probably be in a Microsoft Game Studios game for Xbox 360.
This rating system had serious problems for 1v1 gameplay, and was widely regarded as 'completely unusably horrible' for team play. To give you an example of some of the 1v1 problems, I'll just share my experience:
---------------------------
I've never played the game, but I can explain the situations you've described below, and why they won't be a problem in SC2.
At release, the best player in the game (a game balance tester who started with a bit of an advantage) reached a TrueSkill (TS) rating of 48, and then quit. He was #1 on the ladder until the ladder reset months later, unable to be surpassed by anyone. Since the system was not well-understood, people just continued to play games, and were not sure why he couldn't be surpassed. The best players reached TS 47 but could not increase. I was one of those players.
Then a ladder reset came. Everyone started playing games, and after around a week, people couldn't get above TS 46. Nobody understood why. About a month later, I created a new account out of boredom, and hit up the 1v1 ladder. I rocketed to TS 48 in 17 games. Other top players saw the same and created new accounts as well, and met me at TS 48.
This appears to be a combination of 2 problems. Firstly, TrueSkill has no decay system. If he quit on 48, then in a few weeks, it's likely that his true skill is no longer 48. That's why it's important to have a decay system or something to account for the fact that the certainty about a player's true skill decreases over time due to inactivity.
Secondly, the update equations for mu (true skill) and sigma (the uncertainty about your true skill) assuming player 1 wins and player 2 loses is:
updated_mu1 = mu1+(sigma1^2/c)*v((mu1-mu2)/c)
updated_mu2 = mu2-(sigma2^2/c)*v((mu1-mu2)/c)
updated_sigma1^2 = sigma1^2*[1-(sigma1^2/c^2)*v((mu1-mu2)/c)]
updated_sigma2^2 = sigma2^2*[1-(sigma2^2/c^2)*v((mu1-mu2)/c)],
where c = sqrt(2*beta^2 + sigma1^2 + sigma2^2), beta is a constant, w is a function with values between 0 and 1, and v is a function that's not important for the discussion.
The important thing to note is that from these equations it's obvious that:
(a) The factor in the square brackets is a number between 0 and 1 for both players, i.e. win or lose uncertainty about true skill decreases
(b) The greater your sigma, the larger the update to mu, i.e. when uncertainty about true skill is low, the change to your true skill is low.
Therefore, the combination of (a) and (b) means that if you play a lot of games, with your TS at around 46, it is harder to get it to increase to 48 because sigma is small, than if you start a new account where your sigma is large.
This does not mean that anyone can just start a new account and hit 48 TS. You have to be good enough to be able to reach a 48 TS. In your case, it seems you initially had a correct TS of 46, but then your skill increased such that your legitimate TS became 48. However, since the system was quite sure you were 46, it was more difficult to increase it to 48.
In conclusion, because sigma tends to decrease, TrueSkill is good at finding your true skill if it doesn't change, but is less capable of tracking your true skill if it were to increase.
We started noticing interesting (read: bad) characteristics of the TrueSkill system. One such characteristic was that if you played a player with significantly lower TS than you (let's say 48 against 37), and you won the game, you would actually lose TrueSkill. Sometimes you would drop from 48 to 47. Someone found a bug that allowed you to view your exact TrueSkill rating (out to many decimal points, so for example, 47.271642), after which we could confirm that playing a player below your skill level definitely made you lose TrueSkill 100% of the time, win or lose (if you lost, your TrueSkill would tank).
Because sigma decreases regardless of a win or loss, the value of updated_sigma^2 in the equations above isn't what's actually used, they add a small number gamma^2 to it (see last page here), reflecting the increase in uncertainty about a player's skill since the end of the previous match, and preventing sigma from decreasing to 0.
So sigma doesn't really go to 0, but goes to gamma (the reasoning in the previous paragraphs is still correct despite this). Now since true skill ranks by mu-3*sigma, assuming gamma = 1, the following could happen when you beat a player with a lower skill than you:
Before game: mu = 47, sigma = 2, mu-3*sigma = 41.
After game (you win): mu = 47.1, sigma = 1.8, sigma after accounting for gamma = 2.0591, mu-3*sigma = 40.9227.
Thus your rank has decreased, because the increase in true skill (mu) was so small that it is outweighed by the increase in uncertainty due to gamma. When I said that this wouldn't happened in TrueSkill in the OP, I was assuming that gamma wouldn't be taken into account for the purposes of ranking.
Another characteristic was that it had a very, very strong weighting component. If you were TS 48, and you played 150 games, then you aren't getting to 49 TS, ever. Beating an equally-ranked opponent at 48 will net you around 0.002 TS points. If you've played 14 games, it will net you about 0.1, a 50-fold difference.
Everyone began to race for TS 50, the end-accomplishment on the solo ladder. Many people didn't think it was even possible to do. I made this my goal. The game became "get a new Xbox account, play 10-16 games, if you lose any games or if you play anyone bad and win after ~16 games, you quit and restart". Losing more than a few games was an insurmountable challenge because your TS became nearly impossible to move. Beating a bad opponent was the same thing as losing.
My second account was un-tainted by losses or games with bad players, so I stayed on it, playing judiciously. I would add all the top players to my friends list, see when they were searching for games, and snipe them. This kept me from playing anyone bad and weighting my account, and somewhat hilariously, made it impossible for them to ever improve, forcing them to create another account. After about two months or so, I was the first to hit TS50, and quit (playing anyone else at that point, even if I won, would drop me back down to 49. This is the silliest thing ever).
A few points:
1. Part of the problem here is that since sigma decreases, it's hard to change your TS.
2. Due to gamma preventing sigma from decreasing to 0, there's nothing stopping you from reaching 50 TS if you win enough.
3. People don't have a right to reach 50 TS. If you're not good enough to get 50 TS, then you shouldn't have 50 TS.
4. Even if it were true that nobody could reach 50 TS, that in itself is not a flaw with the ranking system. Ranks are relative, if no one can do it, it doesn't necessarily imply that the ranks are wrong.
While part of the problem is due to sigma always decreasing in TrueSkill, the abuse you've outlined here is more so a problem with multiple accounts. What should your legitimate and true TS be? Mathematically, it should be the value of TS calculated had all those games that you played on hundreds of accounts occurred on a single account. Such an abuse is possible in any game that allows for multiple accounts. Suppose it takes around 150 wins in a row to reach the top of the ladder. If you had a sufficiently large number of accounts, then by the Infinite Monkey Theorem, even a bronze league player will eventually get a 150-0 record on at least one of those accounts given enough attempts
Following this, people figured out that you can just boost to 50+ by getting two accounts to ~45 TS, which is easy to do because of the weighting factor. Then you attempt to get matched up with eachother by searching at the same time and forfeit. Months later, there were 5-6 TS 50s from this (even though it's essentially cheating).
---------------------------
This is a problem with win trading. Such an abuse is common in WoW arenas. Although in that case it wasn't completely terrible, because people used it to quickly get back to a 3000 team rating after a ladder reset, but it only worked if you had a 3000 MMR before the ladder reset.
As you can see, the rating system became the focal point of the game, and you had to game it in order to progress on the ladder.
I'm a bit tired of typing to write a full experience with the team TrueSkill system, but just trust me when I say that it's unplayable. My team had a 118-0 record (yes, seriously) on the in-game leaderboard. We were ranked 956th or something with 42 TS. The #1 team was like 30-10 with 49 TS, because they learned that the TrueSkill system gives insane gains when you pair a highly-ranked player with lowly-ranked players and intentionally exploited that fact. The teams in the between unintentionally exploited that fact.
Your endorsement of the TrueSkill system, and inclination to move Starcraft's system closer towards it, makes me question how good of an idea these suggestions would be. If Halo Wars' implementation of the TrueSkill system is any indication, implementing TrueSkill would literally ruin the Starcraft II ladder. I've laddered competitively on Starcraft II as well as an RTS with TrueSkill and I can say with confidence that Starcraft's ladder system is superior.
Do you have personal experience with a system where TrueSkill works well?
The problems you've outline here are either caused by sigma virtually always decreasing in TrueSkill, or external factors such as multiple accounts and win trading. On the former, this won't be a problem in SC2 because sigma increases when the result of a game is surprising (e.g. you beat a higher skilled player or you lose to a lower skilled player). On the former, these are issues that aren't directly related or fixable by the ladder system. It's also hard to get multiple accounts in SC2, because it costs lots of money.
It should be noted that I'm not saying to switch to the TrueSkill system and use all their equations. I'm just saying to rank by mu-1.96*sigma, like TrueSkill. This isn't even a TrueSkill concept, it's the lower bound of a 95% confidence interval for a normally distributed quantity. It's a concept that is ubiquitous in statistics.
I've outlined how the problem of losing ranks after winning games can be avoided in the "technical issue" section in 3.2. And my suggested changes to the ladder system wouldn't make it any more abuseable than current, as it doesn't involve a change to TrueSkill's equations.
I'd be interested if you come up with an abuse that is possible under my suggested ladder system (as opposed to TrueSkill), that isn't currently possible. As explained above, none of the abuses you've listed fit this criteria.
|
bnets ladder really makes me miss iccup. Having a MOTW with a point bonus and interchangeable maps was so cool. What blizzard needs to do is implement something like that into their AMM system instead of this dated ladder that they have (when I say dated i am speaking in terms of maps and updates).
If you look at it, with iccup, the best players always occupied the top spots anyway and it encouraged activity because you needed to play a ton of games to get there.
Going 30-0 and being on top the ladder SHOULD be discouraged. Yea that person may be the best player, but thats kinda dumb. They can just sit there forever at their 3000 ELO rating and never play a game again.
|
On January 02 2013 12:16 Sadist wrote:
bnets ladder really makes me miss iccup. Having a MOTW with a point bonus and interchangeable maps was so cool. What blizzard needs to do is implement something like that into their AMM system instead of this dated ladder that they have (when I say dated i am speaking in terms of maps and updates).
If you look at it, with iccup, the best players always occupied the top spots anyway and it encouraged activity because you needed to play a ton of games to get there.
Going 30-0 and being on top the ladder SHOULD be discouraged. Yea that person may be the best player, but thats kinda dumb. They can just sit there forever at their 3000 ELO rating and never play a game again.
I'm not familiar with the ICCUP ladder. But from what I've heard of it, it sounds horrible.
I believe people are given a grade, like A, B, C, D, etc, and that like 80% of players aren't good enough to get above a D. If this is true, then this is just a completely messed up league system. A system that is incapable of distinguishing the skill of 80% of players, because they're all given the exact same D.
Giving bonus points for MotW is also a ridiculous idea. The purpose of a ladder system is to correctly rank, giving extra points for things that don't increase the accuracy of ranks (e.g. bonus pool in SC2 or bonus points for MotW) wrecks this. If you beat a player on map A in a week where map A is MotW, you'll get extra points, but not if you beat him on map A in a week where it's not MotW. But the system is no more sure that the winner is better than the loser in the first case than in the second case. So why should be first case be disproportionately rewarded with extra ladder points? It makes no sense.
Now Blizzard is going to reward activity through the level system, they even have days where players get bonus XP. And that's exactly how it should work. Encouragement to play should be taken out of the ladder system and put into a system that rewards cosmetic items so that ladder ranks aren't screwed up by these gimmicks.
|
On January 02 2013 12:30 paralleluniverse wrote:Show nested quote +On January 02 2013 12:16 Sadist wrote:
bnets ladder really makes me miss iccup. Having a MOTW with a point bonus and interchangeable maps was so cool. What blizzard needs to do is implement something like that into their AMM system instead of this dated ladder that they have (when I say dated i am speaking in terms of maps and updates).
If you look at it, with iccup, the best players always occupied the top spots anyway and it encouraged activity because you needed to play a ton of games to get there.
Going 30-0 and being on top the ladder SHOULD be discouraged. Yea that person may be the best player, but thats kinda dumb. They can just sit there forever at their 3000 ELO rating and never play a game again. I'm not familiar with the ICCUP ladder. But from what I've heard of it, it sounds horrible. I believe people are given a grade, like A, B, C, D, etc, and that like 80% of players aren't good enough to get above a D. If this is true, then this is just a completely messed up league system. A system that is incapable of distinguishing the skill of 80% players, because they're all given a D. Giving bonus points for MotW is also a ridiculous idea. The purpose of a ladder system is to correctly rank, giving extra points for things that don't increase the accuracy of ranks (e.g. bonus pool in SC2 or bonus points for MotW) wrecks this. If you beat a player on map A in a week where map A is MotW, you'll get extra points, but not if you beat him on map A in a week where it's not MotW. But the system is no more sure that the winner is better than the loser in the first case than in the second case. So why should be first case be disproportionately rewarded by ladder points? It makes no sense. Now Blizzard is going to reward activity through the level system, they even have days where players get bonus XP. And that's exactly how it should work. Encouragement to play should be taken out of the ladder system and into a system that rewards cosmetic items so that ladder ranks aren't mess up by these gimmicks.
giving bonus for a MOTW gives people incentive to play new maps and allows maps to be introduced throughout the season. It actually is better for the longterm health of the game itself because it can promote creativity.
We have tournaments to decide the best players. I probably shouldn't be in this thread because it seems to be strictly about statistics, but from a players perspective iccup's system was much superior to this. You could add more ranking systems at the bottom if you so liked to widen out the lower tiers of players. The reason there was a larger percentage at D on iccup is because you could make a new account whenevr you wanted and people played like 5 games and stopped. VERY rarely did you find people at like 100 games played stuck at D. It just didnt happen often.\
This hidden MMR system is pretty stupid to a player IMO. A visible point system (however inaccurate you may deem it) is much better to a player because at least you know where you stand.
If you want to find the best players why not just due pure ELO? Who cares about point decay if that is your main objective.
|
On January 02 2013 12:34 Sadist wrote:Show nested quote +On January 02 2013 12:30 paralleluniverse wrote:On January 02 2013 12:16 Sadist wrote:
bnets ladder really makes me miss iccup. Having a MOTW with a point bonus and interchangeable maps was so cool. What blizzard needs to do is implement something like that into their AMM system instead of this dated ladder that they have (when I say dated i am speaking in terms of maps and updates).
If you look at it, with iccup, the best players always occupied the top spots anyway and it encouraged activity because you needed to play a ton of games to get there.
Going 30-0 and being on top the ladder SHOULD be discouraged. Yea that person may be the best player, but thats kinda dumb. They can just sit there forever at their 3000 ELO rating and never play a game again. I'm not familiar with the ICCUP ladder. But from what I've heard of it, it sounds horrible. I believe people are given a grade, like A, B, C, D, etc, and that like 80% of players aren't good enough to get above a D. If this is true, then this is just a completely messed up league system. A system that is incapable of distinguishing the skill of 80% players, because they're all given a D. Giving bonus points for MotW is also a ridiculous idea. The purpose of a ladder system is to correctly rank, giving extra points for things that don't increase the accuracy of ranks (e.g. bonus pool in SC2 or bonus points for MotW) wrecks this. If you beat a player on map A in a week where map A is MotW, you'll get extra points, but not if you beat him on map A in a week where it's not MotW. But the system is no more sure that the winner is better than the loser in the first case than in the second case. So why should be first case be disproportionately rewarded by ladder points? It makes no sense. Now Blizzard is going to reward activity through the level system, they even have days where players get bonus XP. And that's exactly how it should work. Encouragement to play should be taken out of the ladder system and into a system that rewards cosmetic items so that ladder ranks aren't mess up by these gimmicks. giving bonus for a MOTW gives people incentive to play new maps and allows maps to be introduced throughout the season. It actually is better for the longterm health of the game itself because it can promote creativity. We have tournaments to decide the best players. I probably shouldn't be in this thread because it seems to be strictly about statistics, but from a players perspective iccup's system was much superior to this. You could add more ranking systems at the bottom if you so liked to widen out the lower tiers of players. The reason there was a larger percentage at D on iccup is because you could make a new account whenevr you wanted and people played like 5 games and stopped. VERY rarely did you find people at like 100 games played stuck at D. It just didnt happen often.\ This hidden MMR system is pretty stupid to a player IMO. A visible point system (however inaccurate you may deem it) is much better to a player because at least you know where you stand. If you want to find the best players why not just due pure ELO? Who cares about point decay if that is your main objective.
If you want to give people a bonus for playing new maps, then the correct way to do it is through a level system that rewards activity. The incorrect way to do it is to distort ranks by giving extra points for MotW, which stuffs up the ladder in the same way that the bonus pool does by rewarding people more than is mathematically correct.
To argue that we have tournaments to decide the best players is to fall into the exact same fallacy that led to the current system where only the top 2% of players deserve to see their losses, where only the top 2% before the removal of division tiers deserve to have comparable points, and where ICCUP says that we won't bother differentiating the skill of 80% of players. Ladder systems aren't just for pro gamers who attend live tournaments. In fact, they don't need ladder systems, it's the rest who do.
I never deemed a "visible point system" as inaccurate. In fact, a system that displays MMR, while having a few minor flaws, would be superior to the current system. And the reason not to do pure ELO, or just to rank using displayed MMR, is because it doesn't take into account uncertainty about a player's ELO/MMR. If we're 99% sure a player's MMR is 1900, then he should be ranked higher than a player who's MMR is 1901, but we're only 50% sure of that. And because player skill changes over time, we need to take into account that the certainty about MMR decreases with inactivity.
That's why I've stated in the OP that the idea is to make a system that ranks as accurately as possible, explicitly accounts for uncertainty about MMR, and that's psychologically positive to the extend that it doesn't materially jeopardize accurate ranks. And to do this, encouragement needs to be moved entirely into a leveling system that rewards cosmetic items.
You cannot just randomly give people bonus ladder points for partaking in some sort of gimmick, under the dubious justification that it promotes playing. It's mathematically wrong.
|
United States12235 Posts
ICCup's system started every player at a base value and there were breakpoints that defined new letter grades. Furthermore, point changes per game were weighted based on the grade, so it was harder to lose points as a D than it was as a B. The MotW added bonus points to wins. This was fun for players because it was easy to rank up early on. However it creates an inflation problem because extra points are pumped into the system. So, you can't really use that as a pure skill ranking. Also, every season points were reset which caused problems like a true D-level player playing a Day 1 game of the new season against NaDa.
|
On January 02 2013 03:53 NonameAI wrote:
I would have to say that bonus pool is fine where it is. This is because matchmaking is made to place people with you who are equal in skill, resulting in a ~50% winrate. If bonus pool didnt exist, everyone would have 0 points, except for smurfs, and people who are on the verge of promotion. While you are right in that it does not help inactive players, creating a treadmill effect, it, instead of helping them, makes them play more. It is more of an incentive than an actual tool in helping inactive players.
As explained here, this has nothing to with bonus pool. It's just a matter of scale.
Suppose for example, without a bonus pool, points under the current system, or my suggested system or whatever, turn out to fall between -5000 and 100. This doesn't look good. Further suppose that we want points to fall between 0 and 2000, because that would look a lot prettier.
Then one way to solve this is to use a simple linear transformation:
new_points = (2/5.1)*old_points + (10000/5.1),
which will map -5000 to 1000 into 0 to 2000.
Another way to do it is to use new_points = 2000*p, where p is the percentile when all players are ranked by the current system or my suggested system or whatever. You can even add bonus pool to this directly (e.g. new_points = 2000*p + bonus_pool), regardless of whether it's the current bonus pool system or the revamped bonus pool system I suggested. All of these ways of scaling that exclude bonus pool will completely preserve ranks.
And yet, this treadmill effect is still a problem. I have to play to keep my rank, because someone else played. I definately think the best way to rank someone is to reveal MMR and rank accordingly. Eg. a higher mmr player is higher in his division. Of course, there is an uncertainty value. The MMR should subtract the average of the upper and lower bounds of a player's uncertainty. So that way, if a player will occasionally play higher than his mmr more often than he plays lower than his mmr, his mmr rises slightly. And if a player's uncertainty shifts to a tendancy to play less than their mmr, he loses some mmr. If blizz wants to hide ppls mmr to hide their patented algorythms, they simply dont reveal the uncertainty, and the mmr algorythm is impossible to decode.
Another idea i have is just to make active players have no bonus pool. Bonus pool only accrues after 2 days of ladder incativity. If a person is active, he instead loses less points for losing a match, equal to the number of points the game is worth + or - the number of consecutive active days. and wins more points for winning a match. This means that inactive players can catch up, and that the ability to gain points along with mmr gets rid of the need for a treadmill effect.
This way:
- a person cant log In once a day to farm his bonus pool and then get off.
- Active players are rewarded.
- Inactive players can catch up, but the playing field is leveled once they are caught up.
(edited to avoid double post with second idea)
Most of this is basically what I said in the OP. So we almost entirely agree here.
However, I suggest 7 days, not 2, because people schedule their lives around weeks, e.g. some people can only play on weekends. As for your idea to reduce the treadmill effect, what you've suggested is mostly a bonus pool system, where the bonus pool is not displayed. The main difference between this and my suggestion (apart from 7 instead of 2) is that I've also suggested that bonus pool is significantly reduce, and it should be consumed exponentially faster the more you have (because there's no need to make it reward activity, since the leveling system and accounting for sigma serves that role), and it's explained in Section 3.3, why this would reduce the treadmill effect.
|
On January 02 2013 13:26 Excalibur_Z wrote:
ICCup's system started every player at a base value and there were breakpoints that defined new letter grades. Furthermore, point changes per game were weighted based on the grade, so it was harder to lose points as a D than it was as a B. The MotW added bonus points to wins. This was fun for players because it was easy to rank up early on. However it creates an inflation problem because extra points are pumped into the system. So, you can't really use that as a pure skill ranking. Also, every season points were reset which caused problems like a true D-level player playing a Day 1 game of the new season against NaDa.
What defines a pure skill ranking anyway? I agree with you, there was point inflation because of all the new accounts that pumped points into the system. But who is to say inflation is a bad thing? At least you knew where you stood. I agree with your issue about someone like Nada (kinda) but if everyone deosnt start at the same place, what is the point of the ladder anyway? At least if you do it as a point total system, you may not get "the best" (whatever that means) player at the top, but it will be damn close assuming everyone is active (especially with the way iccup changed the point system as you moved up into the ranks). Going from B+ to A- or A was a huge jump because of the competition level and the fact that I believe at A- you lost more points for losses than you gained for wins so it was pretty difficult to stay there if you didn't belong.
I still say something like the motw is good for the community as a whole. It encourages playing new maps or maps you dont like which can lead to an overall skill increase for everyone since people are forced out of their comfort zones.
I know half of the maps on iccup I woulnd't have played had there not been that added incentive of bonus points for playing motw (that and most people wanted the bonus points so it was harder to get games on the regular ladder maps).
It made it cool because it felt like every week or 2 you were trying out a new map which was good because it was fairly easy to get into a rut of playing the same maps OVER and OVER (which I believe is what we see on sc2 now because of the ladder system).
Theres very little experimentation with what we have now. Im glad Kespa will have their own people working on making new maps. Maybe we will see some creativity and things like non buildable (or creepable?) areas on the map. Maybe this restricts warp ins and forces P to build more warp prisms. Things like that.
|
On January 02 2013 01:44 marvellosity wrote:
I don't have much deep to add, I just read it because it was interesting and made good points. I'm a fairly casual low masters player, I just play a few games a week. The one suggestion I didn't like was only updating the ladder weekly. Even though I appreciate it's pointless, I like seeing my points go up and down as I play (even knowing that with my bonus pool and the bonus people of other people in my division, the ranks are mostly meaningless). Basically what I'm saying is that I like immediate feedback when I play games, and it would irritate me not to have it, however meaningless it is ^^
The suggestion only pertains to updating the ladder ranks.
So points would still go up and down. I think that's important so you get feedback about wins and losses and how much they matter.
I think it would be OK if the division ladder were updated in real time, as long as division ranks are removed from the matchmaking page and put into the ladder page in the profile to reduce the treadmill effect. But the main reason for updating a global ladder once weekly is because it takes the ladder snapshot when bonus pool doesn't really have a distortionary effect (paragraph 4 in 3.3), so it displays ladder ranks at the only time that they're "correct", and it greatly reduces the treadmill effect. As for division ladder it's OK to update them in real time because the first point doesn't matter due to division ranks being meaningless.
|
On January 02 2013 13:40 Sadist wrote:Show nested quote +On January 02 2013 13:26 Excalibur_Z wrote:
ICCup's system started every player at a base value and there were breakpoints that defined new letter grades. Furthermore, point changes per game were weighted based on the grade, so it was harder to lose points as a D than it was as a B. The MotW added bonus points to wins. This was fun for players because it was easy to rank up early on. However it creates an inflation problem because extra points are pumped into the system. So, you can't really use that as a pure skill ranking. Also, every season points were reset which caused problems like a true D-level player playing a Day 1 game of the new season against NaDa. What defines a pure skill ranking anyway? I agree with you, there was point inflation because of all the new accounts that pumped points into the system. But who is to say inflation is a bad thing? At least you knew where you stood. I agree with your issue about someone like Nada (kinda) but if everyone deosnt start at the same place, what is the point of the ladder anyway? At least if you do it as a point total system, you may not get "the best" (whatever that means) player at the top, but it will be damn close assuming everyone is active (especially with the way iccup changed the point system as you moved up into the ranks). Going from B+ to A- or A was a huge jump because of the competition level and the fact that I believe at A- you lost more points for losses than you gained for wins so it was pretty difficult to stay there if you didn't belong. I still say something like the motw is good for the community as a whole. It encourages playing new maps or maps you dont like which can lead to an overall skill increase for everyone since people are forced out of their comfort zones. I know half of the maps on iccup I woulnd't have played had there not been that added incentive of bonus points for playing motw (that and most people wanted the bonus points so it was harder to get games on the regular ladder maps). It made it cool because it felt like every week or 2 you were trying out a new map which was good because it was fairly easy to get into a rut of playing the same maps OVER and OVER (which I believe is what we see on sc2 now because of the ladder system). Theres very little experimentation with what we have now. Im glad Kespa will have their own people working on making new maps. Maybe we will see some creativity and things like non buildable (or creepable?) areas on the map. Maybe this restricts warp ins and forces P to build more warp prisms. Things like that.
A pure skill rating system would be one which isn't distorted by whatever artificial bonus or gimmick the designers want to put in. For example, one which isn't distorted by bonus pool or bonus points for MotW.
A pure skill rating system should have certain properties. For example, if the system believes a players skill has increased by X, then it should always go up by X. Giving bonus points for MotW will reward you with X if you beat a player on that map when it's not MotW, but reward you with X+bonus if you beat the exact same player in exactly the same conditions, in a week where that map is MotW. So it fails this very basic property.
Another example, in a pure skill rating system, the idea of a ladder reset makes zero sense. It simply does not make sense to believe a player has 2400 MMR and 93 percentile on one day, and then to believe everyone has 0 MMR and 0 percentile the very next day, because someone has arbitrarily decided that the ladder should be reset. As I've said, things should be somewhat partly reset in a ladder reset because people expect a "clean slate", but it's worth noting that doing so deviates from a pure skill rating system.
As I've said in the OP, the bonus pool, and hence point inflation, is only a bad thing to the extend that it distorts ranks and creates a treadmill effect. And it does so for virtually no valid reason. You talk about a huge jump from B+ to A-, but these types of discontinuities are both pointless and artificial. Why should there be a big jump between B+ and A-? This only make sense if in the real world, there is no one with an intermediate skill level, which is definitely not true. Skill ratings fall in a continuum, to artificially create jumps because someone thought it would be a cool thing to do, violates this basic fact. Furthermore, assuming that 80% of players are in D, is ICCUP suggesting that 80% of players are exactly the same skill? Is that why it doesn't differentiate the skill between these players? Again, clearly this is not true. It's merely an attempt to shoehorn skill ratings into these artificial grades. Again this violates common knowledge about how skill is distributed. This system fails to display basic properties that a system that models skill correctly and purely should have.
You say you know that such a system can be trusted. I don't see why anyone would take such a flawed system seriously. Probably because it's all you had at the time.
The correct way to reward activity and to encourage people to try out whatever you want them to try out is let them work towards cosmetic, non-ladder rewards. For example, through the new level system or by tokens used to buy portraits, units skins, decals, B.net backgrounds, etc.
|
On January 02 2013 14:06 paralleluniverse wrote:Show nested quote +On January 02 2013 13:40 Sadist wrote:On January 02 2013 13:26 Excalibur_Z wrote:
ICCup's system started every player at a base value and there were breakpoints that defined new letter grades. Furthermore, point changes per game were weighted based on the grade, so it was harder to lose points as a D than it was as a B. The MotW added bonus points to wins. This was fun for players because it was easy to rank up early on. However it creates an inflation problem because extra points are pumped into the system. So, you can't really use that as a pure skill ranking. Also, every season points were reset which caused problems like a true D-level player playing a Day 1 game of the new season against NaDa. What defines a pure skill ranking anyway? I agree with you, there was point inflation because of all the new accounts that pumped points into the system. But who is to say inflation is a bad thing? At least you knew where you stood. I agree with your issue about someone like Nada (kinda) but if everyone deosnt start at the same place, what is the point of the ladder anyway? At least if you do it as a point total system, you may not get "the best" (whatever that means) player at the top, but it will be damn close assuming everyone is active (especially with the way iccup changed the point system as you moved up into the ranks). Going from B+ to A- or A was a huge jump because of the competition level and the fact that I believe at A- you lost more points for losses than you gained for wins so it was pretty difficult to stay there if you didn't belong. I still say something like the motw is good for the community as a whole. It encourages playing new maps or maps you dont like which can lead to an overall skill increase for everyone since people are forced out of their comfort zones. I know half of the maps on iccup I woulnd't have played had there not been that added incentive of bonus points for playing motw (that and most people wanted the bonus points so it was harder to get games on the regular ladder maps). It made it cool because it felt like every week or 2 you were trying out a new map which was good because it was fairly easy to get into a rut of playing the same maps OVER and OVER (which I believe is what we see on sc2 now because of the ladder system). Theres very little experimentation with what we have now. Im glad Kespa will have their own people working on making new maps. Maybe we will see some creativity and things like non buildable (or creepable?) areas on the map. Maybe this restricts warp ins and forces P to build more warp prisms. Things like that. A pure skill rating system would be one which isn't distorted by whatever artificial bonus or gimmick the designers want to put in. For example, one which isn't distorted by bonus pool or bonus points for MotW. A pure skill rating system should have certain properties. For example, if the system believes a players skill has increased by X, then it should always go up by X. Giving bonus points for MotW will reward you with X if you beat a player on that map when it's not MotW, but reward you with X+bonus if you beat the exact same player in exactly the same conditions, in a week where that map is MotW. So it fails this very basic property. Another example, in a pure skill rating system, the idea of a ladder reset makes zero sense. It simply does not make sense to believe a player has 2400 MMR and 93 percentile on one day, and then to believe everyone has 0 MMR and 0 percentile the very next day, because someone has arbitrarily decided that the ladder should be reset. As I've said, things should be somewhat partly reset in a ladder reset because people expect a "clean slate", but it's worth noting that doing so deviates from a pure skill rating system. As I've said in the OP, the bonus pool, and hence point inflation, is only a bad thing to the extend that it distorts ranks and creates a treadmill effect. And it does so for virtually no valid reason. You talk about a huge jump from B+ to A-, but these types of discontinuities are both pointless and artificial. Why should there be a big jump between B+ and A-? This only make sense if in the real world, there is no one with an intermediate skill level, which is definitely not true. Skill ratings fall in a continuum, to artificially create jumps because someone thought it would be a cool thing to do, violates this basic fact. Furthermore, assuming that 80% of players are in D, is ICCUP suggesting that 80% of players are exactly the same skill? Is that why it doesn't differentiate the skill between these players? Again, clearly this is not true. It's merely an attempt to shoehorn skill ratings into these artificial grades. Again this violates common knowledge about how skill is distributed. This system fails to display basic properties that a system that models skill correctly and purely should have. You say you know that such a system can be trusted. I don't see why anyone would take such a flawed system seriously. Probably because it's all you had at the time. The correct way to reward activity and to encourage people to try out whatever you want them to try out is let them work towards cosmetic, non-ladder rewards. For example, through the new level system or by tokens used to buy portraits, units skins, decals, B.net backgrounds, etc.
Whats the point of the ladder if everyone is already slotted into a specific place? Don't you run into the problem that iechoic mentioned where the longer you played at a certain level, it becomes tougher to move up when you improve? A system like this doesn't take into account that for most players, skill doesnt increase linearly, often it increases in large jumps. So because I played for a long time before I had a big epiphany and rapidly became a lot better, I move up slower on the ladder than someone who has played fewer games because my sigma is lower? Thats stupid. You aren't measuring my skill at what it is now, you are basically screwing me over because of what my skill once was.
At least in a purely points based system I wouldn't be held back on the ladder from my previous 1000 games. My previous 1000 games have nothing to do with my skill now and shouldnt hold me back from increasing my rank on the ladder.
If you truly are trying to find the best/better players, you are talking about competitive players who don't really care about portraits. Using portraits/achievemetns for things like MOTW would be irrelevant. Why not just separate the ladder into casuals and people who care about the game?
Also this idea about 80% of the ladder being grouped into D is completely arbitrary (the ladder was based on your total number of points, if we are both at D, but I have 1300 pts and you have 1200 pts, I am higher than you on the ladder obv.). You could set the ranking wherever you want. 80% of the ladder could have been B if they were good enough and won enough points. Or you could create an infinite amount of rankings so that each person occupied their own slot. Its irrelevant. The point is you knew were you stood and you moved up the ladder by accumulating points, not some fluxuating MMR system you can't see.
This idea that the ladder measures skill is also kinda laughable. There are so many variables that go into getting wins against someone, the map you played them on, the build order, were they on tilt, were you playing exceptionally well, were they playing exceptionally bad? Did they get cheesed the game before so now they are looking out for cheese? Blind build order luck........
sure if we all play enough games things like that even out......but I dont believe your system is calling for that. Calling someone the best or something like skill is purely arbitrary anyway. What exactly is skill? Say i'm a guy who is a steady player and beats everyone on the ladder except 1 person 99% of the time. That 1 person has my number and just kicks my ass and I never beat him, but he only beats the rest of the ladder 75% of the time. Who is the better player? What does that even mean?
|
On January 02 2013 14:32 Sadist wrote:Show nested quote +On January 02 2013 14:06 paralleluniverse wrote:On January 02 2013 13:40 Sadist wrote:On January 02 2013 13:26 Excalibur_Z wrote:
ICCup's system started every player at a base value and there were breakpoints that defined new letter grades. Furthermore, point changes per game were weighted based on the grade, so it was harder to lose points as a D than it was as a B. The MotW added bonus points to wins. This was fun for players because it was easy to rank up early on. However it creates an inflation problem because extra points are pumped into the system. So, you can't really use that as a pure skill ranking. Also, every season points were reset which caused problems like a true D-level player playing a Day 1 game of the new season against NaDa. What defines a pure skill ranking anyway? I agree with you, there was point inflation because of all the new accounts that pumped points into the system. But who is to say inflation is a bad thing? At least you knew where you stood. I agree with your issue about someone like Nada (kinda) but if everyone deosnt start at the same place, what is the point of the ladder anyway? At least if you do it as a point total system, you may not get "the best" (whatever that means) player at the top, but it will be damn close assuming everyone is active (especially with the way iccup changed the point system as you moved up into the ranks). Going from B+ to A- or A was a huge jump because of the competition level and the fact that I believe at A- you lost more points for losses than you gained for wins so it was pretty difficult to stay there if you didn't belong. I still say something like the motw is good for the community as a whole. It encourages playing new maps or maps you dont like which can lead to an overall skill increase for everyone since people are forced out of their comfort zones. I know half of the maps on iccup I woulnd't have played had there not been that added incentive of bonus points for playing motw (that and most people wanted the bonus points so it was harder to get games on the regular ladder maps). It made it cool because it felt like every week or 2 you were trying out a new map which was good because it was fairly easy to get into a rut of playing the same maps OVER and OVER (which I believe is what we see on sc2 now because of the ladder system). Theres very little experimentation with what we have now. Im glad Kespa will have their own people working on making new maps. Maybe we will see some creativity and things like non buildable (or creepable?) areas on the map. Maybe this restricts warp ins and forces P to build more warp prisms. Things like that. A pure skill rating system would be one which isn't distorted by whatever artificial bonus or gimmick the designers want to put in. For example, one which isn't distorted by bonus pool or bonus points for MotW. A pure skill rating system should have certain properties. For example, if the system believes a players skill has increased by X, then it should always go up by X. Giving bonus points for MotW will reward you with X if you beat a player on that map when it's not MotW, but reward you with X+bonus if you beat the exact same player in exactly the same conditions, in a week where that map is MotW. So it fails this very basic property. Another example, in a pure skill rating system, the idea of a ladder reset makes zero sense. It simply does not make sense to believe a player has 2400 MMR and 93 percentile on one day, and then to believe everyone has 0 MMR and 0 percentile the very next day, because someone has arbitrarily decided that the ladder should be reset. As I've said, things should be somewhat partly reset in a ladder reset because people expect a "clean slate", but it's worth noting that doing so deviates from a pure skill rating system. As I've said in the OP, the bonus pool, and hence point inflation, is only a bad thing to the extend that it distorts ranks and creates a treadmill effect. And it does so for virtually no valid reason. You talk about a huge jump from B+ to A-, but these types of discontinuities are both pointless and artificial. Why should there be a big jump between B+ and A-? This only make sense if in the real world, there is no one with an intermediate skill level, which is definitely not true. Skill ratings fall in a continuum, to artificially create jumps because someone thought it would be a cool thing to do, violates this basic fact. Furthermore, assuming that 80% of players are in D, is ICCUP suggesting that 80% of players are exactly the same skill? Is that why it doesn't differentiate the skill between these players? Again, clearly this is not true. It's merely an attempt to shoehorn skill ratings into these artificial grades. Again this violates common knowledge about how skill is distributed. This system fails to display basic properties that a system that models skill correctly and purely should have. You say you know that such a system can be trusted. I don't see why anyone would take such a flawed system seriously. Probably because it's all you had at the time. The correct way to reward activity and to encourage people to try out whatever you want them to try out is let them work towards cosmetic, non-ladder rewards. For example, through the new level system or by tokens used to buy portraits, units skins, decals, B.net backgrounds, etc. Whats the point of the ladder if everyone is already slotted into a specific place? Don't you run into the problem that iechoic mentioned where the longer you played at a certain level, it becomes tougher to move up when you improve? A system like this doesn't take into account that for most players, skill doesnt increase linearly, often it increases in large jumps. So because I played for a long time before I had a big epiphany and rapidly became a lot better, I move up slower on the ladder than someone who has played fewer games because my sigma is lower? Thats stupid. You aren't measuring my skill at what it is now, you are basically screwing me over because of what my skill once was. At least in a purely points based system I wouldn't be held back on the ladder from my previous 1000 games. My previous 1000 games have nothing to do with my skill now and shouldnt hold me back from increasing my rank on the ladder. If you truly are trying to find the best/better players, you are talking about competitive players who don't really care about portraits. Using portraits/achievemetns for things like MOTW would be irrelevant. Why not just separate the ladder into casuals and people who care about the game? Also this idea about 80% of the ladder being grouped into D is completely arbitrary (the ladder was based on your total number of points, if we are both at D, but I have 1300 pts and you have 1200 pts, I am higher than you on the ladder obv.). You could set the ranking wherever you want. 80% of the ladder could have been B if they were good enough and won enough points. Or you could create an infinite amount of rankings so that each person occupied their own slot. Its irrelevant. The point is you knew were you stood and you moved up the ladder by accumulating points, not some fluxuating MMR system you can't see. This idea that the ladder measures skill is also kinda laughable. There are so many variables that go into getting wins against someone, the map you played them on, the build order, were they on tilt, were you playing exceptionally well, were they playing exceptionally bad? Did they get cheesed the game before so now they are looking out for cheese? Blind build order luck........ sure if we all play enough games things like that even out......but I dont believe your system is calling for that. Calling someone the best or something like skill is purely arbitrary anyway. What exactly is skill? Say i'm a guy who is a steady player and beats everyone on the ladder except 1 person 99% of the time. That 1 person has my number and just kicks my ass and I never beat him, but he only beats the rest of the ladder 75% of the time. Who is the better player? What does that even mean?
What is your point? Who said that everyone is already slotted into a specific place and can't move?
The post I made addressing iEchoic's concerns with TrueSkill is for TrueSkill, not the SC2 ladder system. The complaint that you can't move in the ladder because of your previous history doesn't exist in the SC2 ladder system because sigma increases when the outcome of a game is surprising, unlike TrueSkill where it decreases. So this point that you've made is irrelevant to SC2.
So ICCUP has both points and a grade like D? If this is true, the argument that 80% of the players that are in D are ranked the same, is no longer valid. But the argument that it's dumb that 80% of players are in D or C or whatever is still valid. Why is it suboptimal for it to be evenly spread like in SC2 (approximately 20/20/20/20/20 of active player)? In addition, you have not addressed the other complaints I've made against the ICCUP system.
Really, it's laughable that the ladder measures skill? I've already addressed this point in the OP:
There are also claims that skill simply cannot be measured accurately with MMR and that there’s no way to account for uncertainty about MMR, so that global ladders are meaningless. But this is just completely wrong. The uncertainty about MMR, is already measured by the system, and I’ve suggested that it should be explicitly included into points by using MMR-1.96*sigma. And even if sigma isn't directly used to calculate points it still tends to reduce as games are played. Skill can be measured with good accuracy, as shown by the near 50-50 matchmaking SC2 achieves using MMR, and the empirical evidence from similar skill rating system such as TrueSkill which have remarkable success.
You base the fact that skill can't be measured accurately because of all the vast number of factors that go into winning a game. That's like saying, the trajectory of a rocket cannot be calculated correctly due to the trillions of particle interactions that constitutes the motion of a rocket and its surroundings. The only thing that matters for skill is whether you're able to win. If you can beat 95% of players with nothing but a cannon rush, and that is the only trick you can pull, then guess what? How well you macro doesn't matter. The depth of your game knowledge doesn't matter. You're better than 95% of all players and deserve a percentile of 95, simply because you can beat 95% of players with a cannon rush.
And even if it were true that we can't measure skill correctly, then what? So we shouldn't try? We should knowingly and deliberately fudge the best measure of skill that we have because you find it incredulous that the MMR could possibly be correct?
You talk about a situation where A > B > C > A. Does it make sense to rank these players and say that A is better than B or C? Well if you account for how these players match up against other players instead of just each other, then you can rank them. And that's what MMR does.
And then you argue that competitive players don't care about portraits. So what? What's your point? That competitive players will be unwilling to play the ladder unless they are rewarded with ladder points beyond what they deserve and beyond what is mathematically correct? And you base this on what? If they don't play then their ranks will fall. If they're competitive then they should play to prevent that. And you're solution to this non-problem is that we should knowingly screw up the ranks just to get these people to try the MotW. Really.
Again, I'm highly confused by what your point is. Replace the SC2 ladder system with the ICCUP system? That it's desirable to reward people for playing by giving them ladder points with no regard for how that affects the accuracy of ranks?
|
United States12235 Posts
The SC2 ladder is more fluid than TrueSkill in that your sigma never shrinks so much that you get locked somewhere. Lose 100 games and you'll find yourself a few leagues lower, win 100 games and you'll move up higher.
The "80% of ICCup is D" simply follows the logic that everyone has to start somewhere. In every system, the starting value is going to be the most common just because people will create an account and not do anything with it, or they'll play one game or two games and then quit. The spread from the lowest player to the highest depends on how frequently people are playing games, and the higher your rating, the higher your win percentage has to be to maintain that rating gap over the next-highest player. I believe in ICCup there is a rule where you can't earn points from someone who is X grades below you, and that's not uncommon for most ranking systems. You're right that it's arbitrary, but then the entire grade system is arbitrary as well (weighting wins more heavily than losses in D than in C than in B than in A).
I think unranked ladder and race levels are just a couple of ways Blizzard is trying to separate the ladder into casual and hardcore in HotS.
You're right that players are prone to streaks, but yes, over enough games that balances out, and that's essentially the idea. Everybody is going to plateau somewhere, and realistically they're not going to fall very far below that either. Therefore their actual skill is somewhere in the middle. MMR isn't there to say "your skill is 1738, period.", it changes with every game and maybe you'll hover between 1725-1750 or 1700-1800 or 1600-1900.
|
Thanks for the detailed reply about my TrueSkill question, was enlightening. I've taken a full read through the OP with your points in mind and agree with everything you've said.
I do think there's value in creating discrete skill rankings (such as in ICCUP's A/B/C/D system), but think there should be higher granularity. I like TrueSkill's 1-50 ranking system, and think it's more enjoyable to play when your rating is bucketized into 50 displayed values, simply because it fosters easier discussion with people and is easier to conceptualize ("How good are you? I'm a 42 TS").
My ideal rating system would essentially be OP's, but with MMR bucketized into ~50 rankings, similar to TrueSkill. This has more to do with branding and psychological response more than anything functional, though.
|
United States12235 Posts
On January 02 2013 17:54 iEchoic wrote:
Thanks for the detailed reply about my TrueSkill question, was enlightening. I've taken a full read through the OP with your points in mind and agree with everything you've said.
I do think there's value in creating discrete skill rankings (such as in ICCUP's A/B/C/D system), but think there should be higher granularity. I like TrueSkill's 1-50 ranking system, and think it's more enjoyable to play when your rating is bucketized into 50 displayed values, simply because it fosters easier discussion with people and is easier to conceptualize ("How good are you? I'm a 42 TS").
My ideal rating system would essentially be OP's, but with MMR bucketized into ~50 rankings, similar to TrueSkill. This has more to do with branding and psychological response more than anything functional, though.
There's something to be said for granularity that would reduce some of the confusion and frustration present in the SC2 ladder system. Technically speaking, originally the ladder had 24 "buckets" via division tiers whereas now there are only 7, but the biggest problem with the tiers was they were completely opaque. Most commonly I've heard people complain that they're "high X/low Y" because they either actually teeter on the border between leagues or they at least think they do because of a larger matchmaking range. I remember Dustin Browder himself in an interview said he's "high Plat/low Diamond" which really defeats the purpose of the league system in my opinion because the Silver-through-Plat buckets are narrow enough that there's matchmaking overlap, but the Bronze and Diamond buckets are large enough that players can feel stuck. It's possible they changed divisions from fixed/tiered buckets to a more organic granular approach (where on the aggregate, top 3 in a division would equate to the top 3% of a league) but there's so much volatility from one division to the next that this doesn't neatly translate.
|
On January 02 2013 12:16 Sadist wrote:Going 30-0 and being on top the ladder SHOULD be discouraged. Yea that person may be the best player, but thats kinda dumb. They can just sit there forever at their 3000 ELO rating and never play a game again. This brings up another issue I have with ladder (It actually isn't related to ELO, but the current system), which is the whole placement thing. Before all my friends quit this game, a few were in Master league and all they did for most of the last year or two after achieving Master was do their placement game and then not play all season, then do the same thing the next season. My one friend did this for the better part of a year, staying in Master playing probably 10 1v1s in total. If you look at most divisions the bottom 10-20 spots are like this almost every time. It seems dumb to me that people can do this. If Blizzard insists on keeping this convoluted ladder system then they should have something in place to make sure people that are placed are actually playing so the bottom quarter of each league isn't filled with people who only played one game. It is pretty demoralizing trying to get into Master league and working my butt off when the bottom chunk of that league doesn't even play yet can maintain their spots by loading up the game once per season, getting matched once then either cheesing or F10-Ning then going on with their day.
I would rather have ELO personally as well. I like having a solid metric to tell where I am at rather than "Oh I sometimes play people in a higher league so I might be getting better but it might actually be that they are playing horribly and might be demoted soon but I don't know".
|
On December 22 2012 12:30 paralleluniverse wrote:Show nested quote +On December 22 2012 06:48 Apolo wrote:
Not only should we be allowed to hide our season history but also our current stats and rank. A part of ladder anxiety is the feeling of others judging our lack of skill. The bad feeling of being in bronze or silver etc is exacerbated by the fact that you wear it like a badge on top of your hat for everyone to see. It's like having a banner saying "I suck".
Not only that, but you play not knowing when, if you will be demoted. Each loss can mean demotion. Actually with this kind of system it would be surprising if there wasn't ladder anxiety. I completely disagree with hiding any current season stats. I agree that previous season history should be hidable because it's the only way to permanently stuff up your account, and then it can never be changed. However, current season stats cannot be screwed up because MMR is self-correcting, so why hide it? Previous season history is old information, so it's not so important. The most fundamental purpose of a ladder system is to rank correctly, not to hide stats because of the hurt feelings of those players with ladder anxiety. These suggested changes and unranked play should already solve a lot of ladder anxiety. You're absolutely right about Blizzard's league system stigmatizing bad players. I find it puzzling when people defend the current system as good for casuals even though Blizzard brands a mediocrity badge called “Bronze League” on 42.1% of players forehead. One of the advantages of a global ladder is that you could scrap the league system or make promotion and demotion criteria clearly stated in terms of percentiles. While in a global ladder, a person with a 0-20 percentile would be equivalent to Bronze, removing the league system and confining the percentile to the ladder summary page prevents these players from being force to wear an "I suck" on their heads, and it removes the anxiety of possibly getting demoted. No leagues system means no demotion. It's also deprives them of a derogatory label. tI might not make much of a difference, but it definitely wouldn't make things worst than they already are.
How would being ranked 999,999 out of 1,200,000 people make your more likely to ladder than being ranked 50th in bronze? It actually gives them a more derogatory label of being worse than 1million others rather than how ever many people are in their division. When you suck, there is no ranking that will make you feel better about it. If you are diamond you might prefer to say i'm ranked 7,000th in the world rather than rank 1 diamond, but then you are in the top 3% of players and while you still suck ass compared to pro's, compared to the other 97% you rock.
There is no way to make a bronze player feel better about being bronze, there simply isn't. You can rank them however you want, they are still ranked and still have that stigma attached to them. The moment you rank someone, in any way, they will care about that ranking and won't want to ruin it. I personally don't care whether i'm rank 1 diamond or rank 50 master, I play because I enjoy the competition and losing my current spot cause I go on a loss streak doesn't bother me.
I've experienced a global ladder and watched as I climbed those thousands of ranks, it was fun for me, not because of anything other than I like the competition. If you don't like to see how you rank against others, then you aren't going to like it. Doesn't matter how its presented. I played LotR: Battle for Middle Earth and at one point was ranked in the top 200 in the world.... didn't matter in the slightest to me, I played cause I loved the competition. I didn't all of a sudden stop playing to keep my spot... because if i did that, I'd lose it anyways lol. Thats the nature of competition.
Stop trying to coddle newbies, give them unranked play and leave them to it.... there is nothing you can do that will make them feel better about being ranked, they will always have "ladder anxiety" because they care more about epeen than competition.
|
lol
LOL (League of Legends) has "copied" SC2's ladder system. Details here: http://na.leagueoflegends.com/news/new-league-system-coming-soon-ranked
I've never played LOL. SC2's ladder system has ample flaws as documented in the OP. Yet, despite this, LOL managed to vastly improve on SC2's ladder system. However, this new ladder system has major blunders of its own.
The article starts with the classical strawman argument against a global ladder:
We decided to move to the new league system for a few reasons. For starters, having a single ladder with all ranked players doesn't provide a lot of incentive for advancement. When you’re ranked 290,000 and have 289,999 opponents left to pass on the way up, that process can seem meaningless and interminable. Tiers and divisions also provide milestones and manageable goals you can strive to achieve at your skill level. Through leagues we can move away from focusing on a single number as the core indicator of a player’s skill, and instead move toward something more compelling: competition on a small ladder with a relatable number of opponents.
It fails to mention that a percentile can simply be used to overcome this problem, as I've explained in the OP:
A percentile should be used. For example, 84.2 instead of 3756 out of 20000. This number is very easy to understand, 84.2 simply means that you’re better than 84.2% of all players that are ranked (this is exactly how university admission ranks work in Australia).
So this isn't even a problem with a global ladder, it's an issue with how your place in the global ladder is displayed. In fact, if they're so determined to stop using numbers, they can simply rename 90 percentile as Diamond/Tier 3/50 or 39 percentile as Silver/Tier 5/75. Oh wait, their new ladder system uses these labels. But then, why not use them on the Elo percentile? That is, why not have a global ladder or call it that?
Now, this new ladder system is essentially a global ladder. Everyone is ranked comparably on a line, unlike the SC2 league system. Their ladder is broken into 6 leagues (named Bronze to Diamond, with Masters called Challenger instead). Within each league (except Challenger), there are 5 division tiers, where each division tier is higher skilled than the previous division tier. So in effect there's really 26 SC2-style leagues, and it is intended that we can infer that Tier 1 is better than Tier 2, etc. Within each division, players are ranked by points from 0 to 100, essentially a percentile. It's unclear whether division points are effectively a percentile for the division or for all divisions in the division tier. If it's the former, then division points are as meaningless as they are in SC2 (as explained in the OP). If it's the latter, then that's good and this is basically a rebranded global ladder. And the following argument I've used against the SC2's ladder system cannot be applied to LOL.
Leagues do not solve the problem of division ranks being meaningless and there being no way to get a reasonable measure of your skill relative to all active players. The 5 leagues other than Masters and GM cover an approximately 20% skill range, in the sense that Platinum league contains players in the top 20%-40%. This is a very large skill gap.
In LOL, a 3.8% skill range is small enough, and you can use division points to calculate a finer global percentile.
Also, there's a clearly articulated promotion criteria, unlike the SC2 ladder system. To get promoted in LOL, you just need to get 100 division points (i.e. get to the top of your division), and then win a best of 3. This is an imperfect promotion criteria. What's the purpose of requiring the player to win a best of 3 when the system can already determined the player's skill through probably hundreds of games in the history? What is the mathematical rationale that these 3 games should be given greater weight? What statistical evidence is there to suggest that this approach will model player skill more accurately? Indeed, singling out 3 particular games is completely arbitrary. They probably chose this just because it sounds like something cool to do, despite there being no mathematical rationale behind it. In fact, this is a distortionary factor. What if you win against 1 higher skilled team and lost against 2 even higher skilled teams, so that on net, you gain rank? Like the SC2 ladder system, the goal should be to promote when the system is reasonably sure of the players skill as measured by uncertainty about Elo (or MMR), and this should just be baked into the calculation of division points. But wait, Elo has no uncertainty. This is one of the reasons why Elo is flawed. Taking a Bayesian approach, like the SC2 ladder system, TrueSkill, or Glicko would be far better than Elo. But despite these imperfections, at least LOL has an explicit and transparent promotion criteria.
Now, this is just crazy:
Losing a ranked game in the league system will cost some of your League Points. If you’re already at the bottom of your division, this may mean falling back to the previous division . Once you’ve earned a skill tier, however, you can never be demoted to the previous tier unless you stop playing for a prolonged period of time.
In LOL, you can't be demoted to a lower league, only to a lower division. Why not? This will be a problem when the system has erroneously placed a new player into the wrong league, or if a player just manages to scrape into a league and becomes relatively worse afterwards. This is a needless, arbitrary, and artificial gimmick that prevents the ladder from self-correcting when player skill changes or when the ladder has made a mistake. It distorts the ranks, messes up the ladder, and for absolutely no reason at all. This is just baffling. What if a player, after being promoted to Diamond, decides to smurf by losing 50 games? He can smurf Bronze players while remaining in Diamond. This is absurd. At least LOL avoids the problem in SC2 whereby players can play 1 game at the start of the season to get put into Diamond or Masters and camp on that league regardless of inactivity.
And finally, there's the fact that Elo will now be hidden:
Our matchmaking system still matches you by skill level, but this “rating” is no longer visible and does not have any bearing on your seasonal rewards or ladder standing. Your standing in your league is now determined by your tier, division and League Points, not your matchmaking rating.
Why? To measure skill, why should the matchmaking system use Elo and the ranking system use leagues and divisions instead? Obviously, one of these is a better measure of skill. So why not use that one everywhere? Well, the answer to this question in SC2 is that using different skill measure for ranking and matchmaking allows for uncertainty about skill to be taken into account (although points and bonus pool do a really bad job as explained in the OP), to allow points to inflate via the bonus pool (this could be done far more optimally as explained in the OP), or to allow points to reset in a season reset without effecting MMR.
But none of these reasons apply to LOL. Elo can't take into account the uncertainty about skill because it sucks. Points don't inflate in LOL because Riot is smart enough to not copy the egregiously terrible SC2 bonus pool system that most of the OP is devoted to demolishing. And since LOL's league and divisions break players into 3.8% skill range, a season reset will have virtually no effect.
So there appears to be no reason why Riot couldn't have, as I've explained above, simply relabeled an Elo percentile of 90 as Diamond/Tier 3/50 or a percentile of 39 as Silver/Tier 5/75. Then there would be no reason to hide Elo, as the players league, division tier and division points would directly correspond to Elo. Problems of flip-flopping between leagues could be solved by the suboptimal, but workable, idea of having a best of 3 when the player hits 100 division points. Nothing more needs to be done.
In conclusion, in a hilarious and unexpected move LOL has copied SC2's league system. It has disowned it's previous global ladder system, despite it's current ladder system being effectively a rebranded global ladder system. Riot has avoided the flaws of the current SC2 ladder system, while still offering a global ladder, clear promotion criteria, and not implementing the bonus pool system. Yet, the LOL league system makes several major mistakes. Some design choices seemingly have no defensible rationale. A lot of these problems with the SC2 and LOL ladder system could have been avoided by asking 3 questions:
1. What is the purpose of <obfuscating factor X>?
2. Is the above purpose legitimate and desirable?
3. If so, is there a better way to achieve this without introducing artificial factors that distort ranks?
And that's what I've set out to do in the OP.
On balance, the new LOL ladder system (at least according to the description given), if it stopped using Elo, would probably be better than SC2's current ladder system. But it makes mistakes that Blizzard should avoid. In my opinion, the ladder system I've outlined in the OP is still preferable to both.
|
It seems that this post is not having the desired effect, because today it has been revealed that the terrible bonus pool system will be added to WoW. While Riot decided to copy parts of SC2's league system a few days ago, they were smart enough to not copy the most counterproductive and distortionary part of the ladder system: the bonus pool. Yet, WoW has.
But what's even worse is that Blizzard continues to use the same arguments, debunked in the OP, to defend this really bad system.
Mid-season Start Players getting into PvP mid-season face a very significant challenge to gear up and become competitive because players that started earlier in the season are so far ahead in terms of gear. We want to make it more reasonable for a player to join in on organized PvP mid-season, while still rewarding the commitment of players that have remained invested in PvP throughout the season. Source: http://us.battle.net/wow/en/blog/8397168/PvP_Gear_in_52_and_Beyond-1_17_2013
Team Rating Inflation As we’ve mentioned previously, we want to see the ladder rankings decided toward the end of a season, and not a foregone conclusion dictated by what happens at the start. As discussed in the PvP in Mists of Pandaria Dev Watercooler, as of patch 5.2 Team Rating will gradually increase as players participate in PvP over the course of the season. We expect the new system to help ensure that the ladder remains active at all ratings and that the competition stays fierce from start to finish. Source: http://us.battle.net/wow/en/blog/8397168/PvP_Gear_in_52_and_Beyond-1_17_2013
Blizzard claims that they want to help people who start midseason. Yet the bonus pool punishes these players, by requiring them to play more games than is necessary to stabilize MMR, because they must also spend all accrued bonus pool to be placed correctly. As explained in the OP:
Suppose it takes about 30 games for the ladder system to calculate a player’s MMR to within an acceptably small uncertainty of 50, any more games would just cause uncertainty to fluctuate a little around 50. Alice joins the ladder on Jan 6. By Feb 20 she has played 90 games, just enough to consume her bonus pool, giving her 1500 MMR with an uncertainty of 50. Bob joins the ladder on Feb 20 and plays 30 games that day, ending with 1500 MMR and the uncertainty about his MMR would also be 50. Bob will have fewer points than Alice because he is 60 games short of consuming his bonus pool. But for the purposes of accurate ranking, there is absolutely no reason why Alice should have more points, since they both have equal MMR and equal uncertainty about MMR. Note that a decay system does not face this problem. Therefore, activity as measured by consumed bonus pool can be a bad proxy to uncertainty about MMR.
They say that they want to encourage activity, instead of camping on a rating. But there's no reason to reward activity for the sake of rewarding activity, as the above example shows. More active players should be rewarded for activity only to the extent that it leads to a lower uncertainty about MMR. But there's no reason to use bonus pool as a proxy to measure this uncertainty, as it is already explicitly measured by sigma, and sigma increases with inactivity and should instead be explicitly taken into account, as suggested in section 3.2 (quoted below) of the OP. In fact, section 3.3 explains how bonus pool can be revamped and done correctly. Unfortunately for WoW, it seems that they've fallen into the same flawed system as SC2.
They further say:
We believe this adjustment to the Team Rating formula will have a similar benefit to the “rating decay” that some of you have been asking for, but will feel more positive – rather than feeling like you must keep running just to stay in place (i.e. keep your current score), players that continue participating will be rewarded with higher Team Ratings. As a season wears on, this should also make upper brackets more active as well. Source: http://us.battle.net/wow/en/blog/7702178/Dev_Watercooler_Mists_of_Pandaria_PvP-10_31_2012
Now they make the treadmill argument, while failing to see that the bonus pool creates the treadmill by causing your rank to continuously fall each day you log in. However, unlike SC2, WoW does not have a crazed and fanatic obsession with ranks. So in that sense, the suggestion in section 3.3 of the OP to deemphasize ranks because of the bonus pool is already in WoW.
And again, they make the completely fallacious claim that the bonus pool is a psychological feel-good decay system. This is an argument already debunked in section 2.3 of the OP. Conveniently, they leave out the fact that the bonus pool distorts ranks (explained in section 2.2 of the OP).
They've also made the claim that this is a good way to encourage activity. But as explained in 2.4, it's a suboptimal way of rewarding activity. In fact, all the sections in the OP on bonus pool deserve to be requoted, because it directly counters every argument they've now made for adding the bonus pool to WoW.
2. The Flaws of the Points and Bonus Pool SystemIn this section, I will explain how the bonus pool system fails as a positive psychological gimmick and distorts points and ranks. 2.1 Bonus Pool is not a catch up mechanismBlizzard claims that the bonus pool is to help casuals keep up on the ladder. Show nested quote +Q. What is the Bonus Pool and how are bonus points acquired? A. The Bonus Pool is an accumulation of points that every player receives whether they're online and playing or not. They're essentially used as a means to help give a player a catch-up boost if they haven't played in a while. The pool does have a cap, but it increases slowly until the end of a season. Source: http://us.battle.net/sc2/en/blog/110519 This is completely wrong. The bonus pool perpetually inflates the points of more active players, which causes your rank to fall and continually requires you to play on a daily basis, even to maintain the same position. This creates a "treadmill effect”. If Alice and Bob are equally skilled and Alice becomes less active, then her points will diverge from Bob's. The bonus pool doesn't help Alice close this divergence, it’s the cause of this divergence in the first place. It’s the problem, not the solution. If there were no bonus pool, after approximately 30 games, players will hit their MMR and fluctuate around this point unless there’s a legitimate change in their skill. This is what happens when bonus pool is used up anyway. In this case, players would stagnate at their true skill level, meaning that there would be no need to catch up. Thus, having no bonus pool system would be far more helpful to players catching up, since they won’t have to chase a moving target. So Blizzard is wrong about bonus pool and their justification for it is completely fallacious. 2.2 Bonus Pool distorts ranks and increases the time taken for points to self-correctThe bonus pool system causes points to inflate until the season lock, so that the ladder never really stabilizes. Suppose that Alice and Bob have used up their bonus pool and are ranked 10 and 15 respectively. Alice is more skilled than Bob. The next day, Alice doesn’t play, so she falls to rank 14 because other players have used their bonus pool. Bob uses his bonus pool moving him to rank 12. Bob is now erroneously ranked higher than Alice, until Alice and the other players use their bonus pool to increase their points and bump Bob down. Until the season lock, this situation is perpetual. The bonus pool also obstructs and hinders MMR from self-correcting, because a player’s points cannot decrease until bonus pool is used, even when skill legitimately does. For example, suppose Alice plays actively, she has true MMR 1000, and bonus pool is given at the rate of 50 per week. In week 5, Alice will correctly have 1250 points. Now suppose that she takes a week off and her true MMR has dropped to 800 (e.g. she had a real life injury, or forgot how to play, etc.). Then in week 6, her correct points would be 1100 (1300 minus 200 MMR for loss skill). But, she’ll still be stuck on 1250 points, which wouldn’t decrease until the bonus pool is used up. Even without the HotS change where lost points are absorbed by the bonus pool, Alice's points will on average change very little, until the bonus pool is spent. Hence, Alice's points have been distorted to be erroneously higher than is correct, with adjustment only happening after the bonus pool is used up. In contrast, adjustment would be immediate had there not been a bonus pool system. These distortions mess up ladder ranks. 2.3 Bonus pool is not a “feel good” decay system, nor a psychological reward Some apologists of the bonus pool system claim that it’s all about positive psychology. Bonus pool prevents stagnation by letting points increase even if skill and MMR plateaus. A decay system is defined as one that deducts points at the end of each week where the player has not played enough games. Instead of a decay system where players are punished for not playing, the bonus pool system "rewards" players for playing. At least that's what it tries to be. Bonus pool was seemingly designed with the same philosophy as WoW’s rested XP system. Back in WoW's beta, instead of punishing players by reducing 50% of XP gain when they've played too much, Blizzard doubled all XP and made rested XP a psychological reward by having it always give a 100% bonus. But such logic cannot be applied to a ranking system, where one player's gain is another player’s loss. Every day you log in, you see your rank fall because of the treadmill effect. Accounting for the fact that other player’s bonus pool causes your rank to continually fall, obviously the reward of increased points is symmetric to the punishment of falling ranks, it’s self-defeating, it cancels itself out. Thus, these positive psychology arguments are also completely wrong. However, the bonus pool system has replaced the traditional decay system. In this section, I’ve shown why bonus pool is a flawed decay system for the purposes of positive psychology. In 2.4, I show why it fails as a decay system for the purposes of accurate ranking. 2.4 Bonus pool rewards activity in a needless and suboptimal wayAnother common argument is that the bonus pool allows for the ladder to reward activity without rewarding mass gaming. While it’s a good idea to encourage activity for the sake of getting people to play the game, this should be done with a levelling system, not a bonus pool system, because the latter distorts ranks as explained in 2.2. The bonus pool tends to increase the points of active players. But for the purpose of accurate ranking, why should activity even matter? If we were psychic and simply knew the skill of each player at a given moment, without needing any games to be played, then we would only use this knowledge for ranking, i.e. in an ideal world ranking will be 100% skill based. However, we don't completely know someone's skill at a given moment, unless they play. This is the only reason to consider factoring activity into points and ranks, as higher activity is usually a good proxy to a higher probability that the player's MMR is correct. To the extent that we have good knowledge of a player's current skill, activity should not matter for the purpose of ranking. This means that ideally, we want to minimize the weight given to activity as a factor, subject to the constraint that the player is active enough to give a reasonably good estimate of his current skill. For example, to have accurate ranks, decay systems that penalize players after a week of inactivity are superior to the current bonus pool system, because they reduce the weight given to activity as it doesn't matter as long as you play a little each week. The bonus pool system, however, requires that you be active always, every single day, so does not satisfy the above-mentioned criteria. Therefore, decay systems result in significantly less distortions than bonus pool systems, particularly for active players. Suppose it takes about 30 games for the ladder system to calculate a player’s MMR to within an acceptably small uncertainty of 50, any more games would just cause uncertainty to fluctuate a little around 50. Alice joins the ladder on Jan 6. By Feb 20 she has played 90 games, just enough to consume her bonus pool, giving her 1500 MMR with an uncertainty of 50. Bob joins the ladder on Feb 20 and plays 30 games that day, ending with 1500 MMR and the uncertainty about his MMR would also be 50. Bob will have fewer points than Alice because he is 60 games short of consuming his bonus pool. But for the purposes of accurate ranking, there is absolutely no reason why Alice should have more points, since they both have equal MMR and equal uncertainty about MMR. Note that a decay system does not face this problem. Therefore, activity as measured by consumed bonus pool can be a bad proxy to uncertainty about MMR. In fact, it’s completely unnecessary to use activity as this proxy, because the system already measures it directly and it’s called sigma. Therefore, bonus pool is flawed because it factors activity into points and ranks, when there is no reason for activity to matter since what we ultimately want from it is uncertainty about the player’s MMR, which is a number the system already knows.
And how to fix the bonus pool:
3. Fixing the Points and Bonus Pool System
Above we have identified 4 goals of bonus pool.
(1) As a catch up mechanism.
(2) To prevent points from stagnating.
(3) To encourage playing more games.
(4) To allow activity to be factored into points and ranks.
It is logically impossible to achieve (1) for any serious ranking system, as explained in 2.1, so this goal will be ignored. So far I have shown that the bonus pool fails at all of these goals, except (2). But worse than failing, I have shown that bonus pool distorts points and ranks thereby screwing up the ladder. In this section, I suggest how to design a ladder system that achieves all of these goals, while only distorting points and ranks to the smallest possible extent.
3.1 Only encourage playing and reward activity through the leveling system.
[...]
3.2 Make a more accurate ranking system by ignoring activity and explicitly including sigma
Note that goals (2) and (3) already address psychology. Therefore, in addressing (4), we are purely concerned with accurate ranking. To explicitly account for uncertainty about MMR, instead of indirectly using consumed bonus pool as an imperfect proxy, points should simply be set to MMR-1.96*sigma (possibly scaled so that the numbers fall into a reasonable range), which is the lower bound of the 95% confidence interval for MMR. Then the current bonus pool system becomes unnecessary. This is essentially what is done in TrueSkill (it uses 3 instead of 1.96).
Here sigma is the uncertainty about MMR, which is initially chosen so that points are equal to 0 for new accounts. It should increase at the end of each week if the player has not played enough games, reflecting the fact that we are less certain about a player’s current skill if he has not recently played.
Currently, the ladder system ranks by points that converge to MMR as long as the player is active enough to spend bonus pool. The proposed system converges to MMR as long as the system gets increasingly sure about the player’s MMR.
3.3 Do the bonus pool correctly: make points converge to MMR-1.96*sigma significantly reduce bonus pool, make bonus pool consume at a faster rate the more bonus pool a player has, give out bonus pool weekly not hourly, update the ladder weekly not in real time, deemphasize ranks and emphasize points
So far we have addressed (3) and (4) without needing the distortionary bonus pool system. The only way to address stagnation without some sort of bonus pool is to increase every player’s points every hour, regardless of their activity. This is not a completely terrible idea. However, this section explains how bonus pool can be redesigned to address (2), while minimizing the distortionary and treadmill effects that are caused by the current system.
The only remaining legitimate reason for bonus pool is to prevent stagnation. Firstly, the suggestion in 3.2 should be slightly amended so that points converge to MMR-1.96*sigma instead of being precisely that as having both a decay system and bonus pool system doesn’t make sense. Next, bonus pool can be significantly reduce, from about 110 per week in HotS, to 20 per week. Additionally, the more bonus pool a player has the faster it should be consume. For example, if you have 100 bonus pool, getting 12 points for winning should use, say, 24 bonus pool, if you have 200 bonus pool, it should use 84 bonus pool. Note that these changes only make sense when implemented together with the suggestions in 3.1 and 3.2 that have already addressed the need to encourage activity and account for uncertainty about MMR. Hence, these changes to trivialize bonus pool have only the purpose of preventing stagnation and nothing more.
These are positive changes because significantly reducing the bonus pool would significantly reduce the distortionary and treadmill effects it creates. Allowing bonus pool to be consumed faster when players have large bonus pools partly addresses the problems in the second example in 2.2 and the “Jan 3 vs Feb 20” example in 2.4. It also partly addresses (1), but no changes in any serious ranking system can (nor should) entirely fix (1).
In addition, bonus pool should be given in bulk, once weekly, instead of in small amounts each hour, and the ladder should only be updated at this time, instead of in real time. Updating the ladder once a week will significantly reduce the treadmill effects since players will no longer see their rank perpetually fall due to other player’s bonus pools. But more importantly, these changes will mostly eliminate the distortionary effects that bonus pool has on ranks as explained in the first example of 2.2. In that example, Alice is more skilled than Bob. She doesn't play for a day and falls below Bob's rank as a result of Bob's bonus pool. So the ladder ranks have become wrong. Now if the ladder were to update only once weekly and bonus pool were changed as I've suggested, then Alice would be able to get back ahead of Bob, before the next ladder snapshot. If she didn't, it would be because she was inactive for the week, so it could be justified that her rank should fall as a small penalty for the chance that her skill has decreased due to prolonged inactivity. However, such an argument cannot be applied to the current bonus pool system because Alice would not lose any skill due to not having played for one hour or one day. The skill lost for 2 weeks of inactivity is far more than 14 times the skill lost in 1 day of inactivity.
Lastly, to further reduce the treadmill effect and to maximize the benefits of having prevented points from stagnating, ranks need to be removed from the matchmaking page and the score screen. Instead, points should be emphasized, as they no longer stagnate. Ranks should be kept in the ladder page in the profile. They are critically important for competition in a competitive game.
|
I'm curious if anyone has heard if they will be changing the recently implemented system whereby you cannot fall out of a league if you're under-performing? I've been playing Master league players on ladder recently, and upon checking their match history I see that they have been playing platinum and diamond players regularly. This means that their MMR is well below master league status.
If so, what exactly is the point of the divisions? While I would love to believe I have improved enough to get into masters it's simply not true, and if players currently in masters are playing people in my division than this implies that there are people in diamond playing at master league level which are not being promoted (I believe).
Anyone have any information on this?
|
The bonus pool also obstructs and hinders MMR from self-correcting, because a player’s points cannot decrease until bonus pool is used, even when skill legitimately does. For example, suppose Alice plays actively, she has true MMR 1000, and bonus pool is given at the rate of 50 per week. In week 5, Alice will correctly have 1250 points. Now suppose that she takes a week off and her true MMR has dropped to 800 (e.g. she had a real life injury, or forgot how to play, etc.). Then in week 6, her correct points would be 1100 (1300 minus 200 MMR for loss skill). But, she’ll still be stuck on 1250 points, which wouldn’t decrease until the bonus pool is used up. Even without the HotS change where lost points are absorbed by the bonus pool, Alice's points will on average change very little, until the bonus pool is spent. Hence, Alice's points have been distorted to be erroneously higher than is correct, with adjustment only happening after the bonus pool is used up. In contrast, adjustment would be immediate had there not been a bonus pool system.
week 5 1000 MMR - 1250 points
week 6 1000 MMR - 1250 points 50 bonus pool
Lets say wins and losses are 1:1 with MMR losses.
So 1 win would be +20 points and +20 mmr, 1 loss would be -20 points, -20 MMR. (This is how you are claiming it to be).
After 3 losses alice would look like this:
940 MMR - 1240 points.Since she lost 60 points but had 50 bonus pool, she only lost 10 points.
Now if she would keep losing it would eventually turn into:
800 MMR - 1100 points.
Which means that 800MMR - 1100 points is Correct.
So by the time she would have bottomed out at 800MMR, she would have reached 1100 points.
Your analysis is flawed, very flawed.
You are claiming that MMR drops instantly without the need of >50% loss ratio. which is not the case. You need to actually lose games to decrease your mmr significantly.
When you are going from 1000mmr to 800mmr you have to lose 10 games. which is -200 points, and with 50 bonus pool, that is an effective 150 point loss.
And if someone loses 100% of the time, but will stop playing after the bonus pool is depleted, the rest of the players will inflate while this guy stands still, which would still make it an accurate measurement.
Now suppose that she takes a week off and her true MMR has dropped to 800 (e.g. she had a real life injury, or forgot how to play, etc.). Then in week 6, her correct points would be 1100 (1300 minus 200 MMR for loss skill). But, she’ll still be stuck on 1250 points, which wouldn’t decrease until the bonus pool is used up.
Using up a 50 point bonus pool takes 2.5 losses. (according to your fictional stats of -20 mmr -20 points per game).
so by the time she used up her bonus pool, she would be at -60 MMR and -10 points.
Which would leave her at 940 MMR and not the 800 you are claiming it to be.
It would be impossible for her to be stuck at 1250 points, and still go any lower than 950 MMR.
Everything about your post is so flawed, I don't even know where to begin - so I'll leave it at this.
|
|
|
|
|
|
EPT
