|
On December 31 2012 05:23 Azoryen wrote:I just want to add that, imo, the biggest factor that makes people stop laddering is fear of demotion. When your MMR is in-between 2 leagues and you manage to get a promotion to the higher league, you have all reasons to stop playing right there. The fact that many people don't realize that the threshold for demotion is lower than the threshold for promotion doesn't help. Just picture this example for a second: - you've been recently promoted to plat, a long term goal! It took you so much effort to get there, but people now finally respect you a little! And, of course, you told all your friends about it. - Problem is now you're on a losing streak, so you start worrying about demotion to gold again. - You have no idea how close it could be... how do demotions work anyway? You have no idea... but you have a feeling it could be next loss or 2. - So what do you do? You stop laddering to keep your precious plat icon. (Note: I don't mean everyone is like this) + Show Spoiler +
Also, the whole ladder system is obscure and not knowing / being in control of things make people uncomfortable and anxious.
This is why I actually think a very good measure would be to make MMR visible.
What would this achieve?
The social importance of being in a certain league would decrease a lot, as people would be focusing more on each other's MMR, so players wouldn't worry so much about demotion.
In terms of MMR, any loss is just another 10-20 points you loose. Not a big deal. The loss aversion remains a lot more constant, instead of reaching huge proportions near demotion.
Also, everyone would understand how ladder works a lot better, which is good.
I feel that Blizzard doesn't want people asking questions like: 'why does this guy with -100 MMR is in a higher league than me?' But making things clear is always better. In chess, GMs having lower ELO than masters happens all the time and no one complains about that, because people understand the system.
Also, I believe MMR is a constant motivation to play. Instead of just having very distant league promotions, you can set a number of intermediate goals for yourself: reach 1500, then reach 1600, etc...
I've had a chess ELO since I was a kid and chess players love their ELOs and are constantly working to improve them for the next 50 or 100 points.
This little number is best motivation one can have to play.
Excellent point. Paralleluniverse makes a great case for the competitive player. And its a point to be made now that non-competitive options are available (unranked play and arcade). There are still remnants of the ladder casuals (like me) who have suffered in the hands of the current system and would like to rest on their laurels rather than face the facts that:
1) Inactivity decays your skill,
2) The ladder average gets better,
3) Maintaining your platinum/diamond symbol can be difficult as you might not have deserved those wins fully (good times for your race, lucky anti-meta build, luck...) if you're not focused on playing sc2 enough, it's better to not play at all...
But despite this reality, full speed ahead, Pallaleluniverse. I hope Blizz listens to you.
|
I think there should be an MMR instead of divisions. EX 3000MMR diamond with a visible "instability factor" and a clear line to show when you get promoted or demoted.
Then make all MMR slowly retrograde over time so if you don't play you will slowly get lower MMR. This encourages occasional playing. Overall i'd like to see that system rather than a meaningless 100 random person division and bonus pool.
TBH bonus pool is okay but we really need a universal ranking instead of devisions :/ we are separating casuals from hardcore players anyhow.
|
On January 02 2013 03:50 iEchoic wrote:
@OP:
This is an interesting post, and you obviously put a lot of thought into it. I would like to respond to certain points about it, but have a question for you, first, since you seem to understand TrueSkill better than most I've been able to talk to.
Halo Wars, an RTS created by Microsoft Game Studios for Xbox 360, used the TrueSkill rating system for online ladder. The developers linked to the same Microsoft research article that you linked to in your OP and claimed that it was this implementation. I would assume that if TrueSkill was to be implemented correctly anywhere, it would probably be in a Microsoft Game Studios game for Xbox 360.
This rating system had serious problems for 1v1 gameplay, and was widely regarded as 'completely unusably horrible' for team play. To give you an example of some of the 1v1 problems, I'll just share my experience:
---------------------------
I've never played the game, but I can explain the situations you've described below, and why they won't be a problem in SC2.
At release, the best player in the game (a game balance tester who started with a bit of an advantage) reached a TrueSkill (TS) rating of 48, and then quit. He was #1 on the ladder until the ladder reset months later, unable to be surpassed by anyone. Since the system was not well-understood, people just continued to play games, and were not sure why he couldn't be surpassed. The best players reached TS 47 but could not increase. I was one of those players.
Then a ladder reset came. Everyone started playing games, and after around a week, people couldn't get above TS 46. Nobody understood why. About a month later, I created a new account out of boredom, and hit up the 1v1 ladder. I rocketed to TS 48 in 17 games. Other top players saw the same and created new accounts as well, and met me at TS 48.
This appears to be a combination of 2 problems. Firstly, TrueSkill has no decay system. If he quit on 48, then in a few weeks, it's likely that his true skill is no longer 48. That's why it's important to have a decay system or something to account for the fact that the certainty about a player's true skill decreases over time due to inactivity.
Secondly, the update equations for mu (true skill) and sigma (the uncertainty about your true skill) assuming player 1 wins and player 2 loses is:
updated_mu1 = mu1+(sigma1^2/c)*v((mu1-mu2)/c)
updated_mu2 = mu2-(sigma2^2/c)*v((mu1-mu2)/c)
updated_sigma1^2 = sigma1^2*[1-(sigma1^2/c^2)*v((mu1-mu2)/c)]
updated_sigma2^2 = sigma2^2*[1-(sigma2^2/c^2)*v((mu1-mu2)/c)],
where c = sqrt(2*beta^2 + sigma1^2 + sigma2^2), beta is a constant, w is a function with values between 0 and 1, and v is a function that's not important for the discussion.
The important thing to note is that from these equations it's obvious that:
(a) The factor in the square brackets is a number between 0 and 1 for both players, i.e. win or lose uncertainty about true skill decreases
(b) The greater your sigma, the larger the update to mu, i.e. when uncertainty about true skill is low, the change to your true skill is low.
Therefore, the combination of (a) and (b) means that if you play a lot of games, with your TS at around 46, it is harder to get it to increase to 48 because sigma is small, than if you start a new account where your sigma is large.
This does not mean that anyone can just start a new account and hit 48 TS. You have to be good enough to be able to reach a 48 TS. In your case, it seems you initially had a correct TS of 46, but then your skill increased such that your legitimate TS became 48. However, since the system was quite sure you were 46, it was more difficult to increase it to 48.
In conclusion, because sigma tends to decrease, TrueSkill is good at finding your true skill if it doesn't change, but is less capable of tracking your true skill if it were to increase.
We started noticing interesting (read: bad) characteristics of the TrueSkill system. One such characteristic was that if you played a player with significantly lower TS than you (let's say 48 against 37), and you won the game, you would actually lose TrueSkill. Sometimes you would drop from 48 to 47. Someone found a bug that allowed you to view your exact TrueSkill rating (out to many decimal points, so for example, 47.271642), after which we could confirm that playing a player below your skill level definitely made you lose TrueSkill 100% of the time, win or lose (if you lost, your TrueSkill would tank).
Because sigma decreases regardless of a win or loss, the value of updated_sigma^2 in the equations above isn't what's actually used, they add a small number gamma^2 to it (see last page here), reflecting the increase in uncertainty about a player's skill since the end of the previous match, and preventing sigma from decreasing to 0.
So sigma doesn't really go to 0, but goes to gamma (the reasoning in the previous paragraphs is still correct despite this). Now since true skill ranks by mu-3*sigma, assuming gamma = 1, the following could happen when you beat a player with a lower skill than you:
Before game: mu = 47, sigma = 2, mu-3*sigma = 41.
After game (you win): mu = 47.1, sigma = 1.8, sigma after accounting for gamma = 2.0591, mu-3*sigma = 40.9227.
Thus your rank has decreased, because the increase in true skill (mu) was so small that it is outweighed by the increase in uncertainty due to gamma. When I said that this wouldn't happened in TrueSkill in the OP, I was assuming that gamma wouldn't be taken into account for the purposes of ranking.
Another characteristic was that it had a very, very strong weighting component. If you were TS 48, and you played 150 games, then you aren't getting to 49 TS, ever. Beating an equally-ranked opponent at 48 will net you around 0.002 TS points. If you've played 14 games, it will net you about 0.1, a 50-fold difference.
Everyone began to race for TS 50, the end-accomplishment on the solo ladder. Many people didn't think it was even possible to do. I made this my goal. The game became "get a new Xbox account, play 10-16 games, if you lose any games or if you play anyone bad and win after ~16 games, you quit and restart". Losing more than a few games was an insurmountable challenge because your TS became nearly impossible to move. Beating a bad opponent was the same thing as losing.
My second account was un-tainted by losses or games with bad players, so I stayed on it, playing judiciously. I would add all the top players to my friends list, see when they were searching for games, and snipe them. This kept me from playing anyone bad and weighting my account, and somewhat hilariously, made it impossible for them to ever improve, forcing them to create another account. After about two months or so, I was the first to hit TS50, and quit (playing anyone else at that point, even if I won, would drop me back down to 49. This is the silliest thing ever).
A few points:
1. Part of the problem here is that since sigma decreases, it's hard to change your TS.
2. Due to gamma preventing sigma from decreasing to 0, there's nothing stopping you from reaching 50 TS if you win enough.
3. People don't have a right to reach 50 TS. If you're not good enough to get 50 TS, then you shouldn't have 50 TS.
4. Even if it were true that nobody could reach 50 TS, that in itself is not a flaw with the ranking system. Ranks are relative, if no one can do it, it doesn't necessarily imply that the ranks are wrong.
While part of the problem is due to sigma always decreasing in TrueSkill, the abuse you've outlined here is more so a problem with multiple accounts. What should your legitimate and true TS be? Mathematically, it should be the value of TS calculated had all those games that you played on hundreds of accounts occurred on a single account. Such an abuse is possible in any game that allows for multiple accounts. Suppose it takes around 150 wins in a row to reach the top of the ladder. If you had a sufficiently large number of accounts, then by the Infinite Monkey Theorem, even a bronze league player will eventually get a 150-0 record on at least one of those accounts given enough attempts
Following this, people figured out that you can just boost to 50+ by getting two accounts to ~45 TS, which is easy to do because of the weighting factor. Then you attempt to get matched up with eachother by searching at the same time and forfeit. Months later, there were 5-6 TS 50s from this (even though it's essentially cheating).
---------------------------
This is a problem with win trading. Such an abuse is common in WoW arenas. Although in that case it wasn't completely terrible, because people used it to quickly get back to a 3000 team rating after a ladder reset, but it only worked if you had a 3000 MMR before the ladder reset.
As you can see, the rating system became the focal point of the game, and you had to game it in order to progress on the ladder.
I'm a bit tired of typing to write a full experience with the team TrueSkill system, but just trust me when I say that it's unplayable. My team had a 118-0 record (yes, seriously) on the in-game leaderboard. We were ranked 956th or something with 42 TS. The #1 team was like 30-10 with 49 TS, because they learned that the TrueSkill system gives insane gains when you pair a highly-ranked player with lowly-ranked players and intentionally exploited that fact. The teams in the between unintentionally exploited that fact.
Your endorsement of the TrueSkill system, and inclination to move Starcraft's system closer towards it, makes me question how good of an idea these suggestions would be. If Halo Wars' implementation of the TrueSkill system is any indication, implementing TrueSkill would literally ruin the Starcraft II ladder. I've laddered competitively on Starcraft II as well as an RTS with TrueSkill and I can say with confidence that Starcraft's ladder system is superior.
Do you have personal experience with a system where TrueSkill works well?
The problems you've outline here are either caused by sigma virtually always decreasing in TrueSkill, or external factors such as multiple accounts and win trading. On the former, this won't be a problem in SC2 because sigma increases when the result of a game is surprising (e.g. you beat a higher skilled player or you lose to a lower skilled player). On the former, these are issues that aren't directly related or fixable by the ladder system. It's also hard to get multiple accounts in SC2, because it costs lots of money.
It should be noted that I'm not saying to switch to the TrueSkill system and use all their equations. I'm just saying to rank by mu-1.96*sigma, like TrueSkill. This isn't even a TrueSkill concept, it's the lower bound of a 95% confidence interval for a normally distributed quantity. It's a concept that is ubiquitous in statistics.
I've outlined how the problem of losing ranks after winning games can be avoided in the "technical issue" section in 3.2. And my suggested changes to the ladder system wouldn't make it any more abuseable than current, as it doesn't involve a change to TrueSkill's equations.
I'd be interested if you come up with an abuse that is possible under my suggested ladder system (as opposed to TrueSkill), that isn't currently possible. As explained above, none of the abuses you've listed fit this criteria.
|
bnets ladder really makes me miss iccup. Having a MOTW with a point bonus and interchangeable maps was so cool. What blizzard needs to do is implement something like that into their AMM system instead of this dated ladder that they have (when I say dated i am speaking in terms of maps and updates).
If you look at it, with iccup, the best players always occupied the top spots anyway and it encouraged activity because you needed to play a ton of games to get there.
Going 30-0 and being on top the ladder SHOULD be discouraged. Yea that person may be the best player, but thats kinda dumb. They can just sit there forever at their 3000 ELO rating and never play a game again.
|
On January 02 2013 12:16 Sadist wrote:
bnets ladder really makes me miss iccup. Having a MOTW with a point bonus and interchangeable maps was so cool. What blizzard needs to do is implement something like that into their AMM system instead of this dated ladder that they have (when I say dated i am speaking in terms of maps and updates).
If you look at it, with iccup, the best players always occupied the top spots anyway and it encouraged activity because you needed to play a ton of games to get there.
Going 30-0 and being on top the ladder SHOULD be discouraged. Yea that person may be the best player, but thats kinda dumb. They can just sit there forever at their 3000 ELO rating and never play a game again.
I'm not familiar with the ICCUP ladder. But from what I've heard of it, it sounds horrible.
I believe people are given a grade, like A, B, C, D, etc, and that like 80% of players aren't good enough to get above a D. If this is true, then this is just a completely messed up league system. A system that is incapable of distinguishing the skill of 80% of players, because they're all given the exact same D.
Giving bonus points for MotW is also a ridiculous idea. The purpose of a ladder system is to correctly rank, giving extra points for things that don't increase the accuracy of ranks (e.g. bonus pool in SC2 or bonus points for MotW) wrecks this. If you beat a player on map A in a week where map A is MotW, you'll get extra points, but not if you beat him on map A in a week where it's not MotW. But the system is no more sure that the winner is better than the loser in the first case than in the second case. So why should be first case be disproportionately rewarded with extra ladder points? It makes no sense.
Now Blizzard is going to reward activity through the level system, they even have days where players get bonus XP. And that's exactly how it should work. Encouragement to play should be taken out of the ladder system and put into a system that rewards cosmetic items so that ladder ranks aren't screwed up by these gimmicks.
|
On January 02 2013 12:30 paralleluniverse wrote:Show nested quote +On January 02 2013 12:16 Sadist wrote:
bnets ladder really makes me miss iccup. Having a MOTW with a point bonus and interchangeable maps was so cool. What blizzard needs to do is implement something like that into their AMM system instead of this dated ladder that they have (when I say dated i am speaking in terms of maps and updates).
If you look at it, with iccup, the best players always occupied the top spots anyway and it encouraged activity because you needed to play a ton of games to get there.
Going 30-0 and being on top the ladder SHOULD be discouraged. Yea that person may be the best player, but thats kinda dumb. They can just sit there forever at their 3000 ELO rating and never play a game again. I'm not familiar with the ICCUP ladder. But from what I've heard of it, it sounds horrible. I believe people are given a grade, like A, B, C, D, etc, and that like 80% of players aren't good enough to get above a D. If this is true, then this is just a completely messed up league system. A system that is incapable of distinguishing the skill of 80% players, because they're all given a D. Giving bonus points for MotW is also a ridiculous idea. The purpose of a ladder system is to correctly rank, giving extra points for things that don't increase the accuracy of ranks (e.g. bonus pool in SC2 or bonus points for MotW) wrecks this. If you beat a player on map A in a week where map A is MotW, you'll get extra points, but not if you beat him on map A in a week where it's not MotW. But the system is no more sure that the winner is better than the loser in the first case than in the second case. So why should be first case be disproportionately rewarded by ladder points? It makes no sense. Now Blizzard is going to reward activity through the level system, they even have days where players get bonus XP. And that's exactly how it should work. Encouragement to play should be taken out of the ladder system and into a system that rewards cosmetic items so that ladder ranks aren't mess up by these gimmicks.
giving bonus for a MOTW gives people incentive to play new maps and allows maps to be introduced throughout the season. It actually is better for the longterm health of the game itself because it can promote creativity.
We have tournaments to decide the best players. I probably shouldn't be in this thread because it seems to be strictly about statistics, but from a players perspective iccup's system was much superior to this. You could add more ranking systems at the bottom if you so liked to widen out the lower tiers of players. The reason there was a larger percentage at D on iccup is because you could make a new account whenevr you wanted and people played like 5 games and stopped. VERY rarely did you find people at like 100 games played stuck at D. It just didnt happen often.\
This hidden MMR system is pretty stupid to a player IMO. A visible point system (however inaccurate you may deem it) is much better to a player because at least you know where you stand.
If you want to find the best players why not just due pure ELO? Who cares about point decay if that is your main objective.
|
On January 02 2013 12:34 Sadist wrote:Show nested quote +On January 02 2013 12:30 paralleluniverse wrote:On January 02 2013 12:16 Sadist wrote:
bnets ladder really makes me miss iccup. Having a MOTW with a point bonus and interchangeable maps was so cool. What blizzard needs to do is implement something like that into their AMM system instead of this dated ladder that they have (when I say dated i am speaking in terms of maps and updates).
If you look at it, with iccup, the best players always occupied the top spots anyway and it encouraged activity because you needed to play a ton of games to get there.
Going 30-0 and being on top the ladder SHOULD be discouraged. Yea that person may be the best player, but thats kinda dumb. They can just sit there forever at their 3000 ELO rating and never play a game again. I'm not familiar with the ICCUP ladder. But from what I've heard of it, it sounds horrible. I believe people are given a grade, like A, B, C, D, etc, and that like 80% of players aren't good enough to get above a D. If this is true, then this is just a completely messed up league system. A system that is incapable of distinguishing the skill of 80% players, because they're all given a D. Giving bonus points for MotW is also a ridiculous idea. The purpose of a ladder system is to correctly rank, giving extra points for things that don't increase the accuracy of ranks (e.g. bonus pool in SC2 or bonus points for MotW) wrecks this. If you beat a player on map A in a week where map A is MotW, you'll get extra points, but not if you beat him on map A in a week where it's not MotW. But the system is no more sure that the winner is better than the loser in the first case than in the second case. So why should be first case be disproportionately rewarded by ladder points? It makes no sense. Now Blizzard is going to reward activity through the level system, they even have days where players get bonus XP. And that's exactly how it should work. Encouragement to play should be taken out of the ladder system and into a system that rewards cosmetic items so that ladder ranks aren't mess up by these gimmicks. giving bonus for a MOTW gives people incentive to play new maps and allows maps to be introduced throughout the season. It actually is better for the longterm health of the game itself because it can promote creativity. We have tournaments to decide the best players. I probably shouldn't be in this thread because it seems to be strictly about statistics, but from a players perspective iccup's system was much superior to this. You could add more ranking systems at the bottom if you so liked to widen out the lower tiers of players. The reason there was a larger percentage at D on iccup is because you could make a new account whenevr you wanted and people played like 5 games and stopped. VERY rarely did you find people at like 100 games played stuck at D. It just didnt happen often.\ This hidden MMR system is pretty stupid to a player IMO. A visible point system (however inaccurate you may deem it) is much better to a player because at least you know where you stand. If you want to find the best players why not just due pure ELO? Who cares about point decay if that is your main objective.
If you want to give people a bonus for playing new maps, then the correct way to do it is through a level system that rewards activity. The incorrect way to do it is to distort ranks by giving extra points for MotW, which stuffs up the ladder in the same way that the bonus pool does by rewarding people more than is mathematically correct.
To argue that we have tournaments to decide the best players is to fall into the exact same fallacy that led to the current system where only the top 2% of players deserve to see their losses, where only the top 2% before the removal of division tiers deserve to have comparable points, and where ICCUP says that we won't bother differentiating the skill of 80% of players. Ladder systems aren't just for pro gamers who attend live tournaments. In fact, they don't need ladder systems, it's the rest who do.
I never deemed a "visible point system" as inaccurate. In fact, a system that displays MMR, while having a few minor flaws, would be superior to the current system. And the reason not to do pure ELO, or just to rank using displayed MMR, is because it doesn't take into account uncertainty about a player's ELO/MMR. If we're 99% sure a player's MMR is 1900, then he should be ranked higher than a player who's MMR is 1901, but we're only 50% sure of that. And because player skill changes over time, we need to take into account that the certainty about MMR decreases with inactivity.
That's why I've stated in the OP that the idea is to make a system that ranks as accurately as possible, explicitly accounts for uncertainty about MMR, and that's psychologically positive to the extend that it doesn't materially jeopardize accurate ranks. And to do this, encouragement needs to be moved entirely into a leveling system that rewards cosmetic items.
You cannot just randomly give people bonus ladder points for partaking in some sort of gimmick, under the dubious justification that it promotes playing. It's mathematically wrong.
|
United States12235 Posts
ICCup's system started every player at a base value and there were breakpoints that defined new letter grades. Furthermore, point changes per game were weighted based on the grade, so it was harder to lose points as a D than it was as a B. The MotW added bonus points to wins. This was fun for players because it was easy to rank up early on. However it creates an inflation problem because extra points are pumped into the system. So, you can't really use that as a pure skill ranking. Also, every season points were reset which caused problems like a true D-level player playing a Day 1 game of the new season against NaDa.
|
On January 02 2013 03:53 NonameAI wrote:
I would have to say that bonus pool is fine where it is. This is because matchmaking is made to place people with you who are equal in skill, resulting in a ~50% winrate. If bonus pool didnt exist, everyone would have 0 points, except for smurfs, and people who are on the verge of promotion. While you are right in that it does not help inactive players, creating a treadmill effect, it, instead of helping them, makes them play more. It is more of an incentive than an actual tool in helping inactive players.
As explained here, this has nothing to with bonus pool. It's just a matter of scale.
Suppose for example, without a bonus pool, points under the current system, or my suggested system or whatever, turn out to fall between -5000 and 100. This doesn't look good. Further suppose that we want points to fall between 0 and 2000, because that would look a lot prettier.
Then one way to solve this is to use a simple linear transformation:
new_points = (2/5.1)*old_points + (10000/5.1),
which will map -5000 to 1000 into 0 to 2000.
Another way to do it is to use new_points = 2000*p, where p is the percentile when all players are ranked by the current system or my suggested system or whatever. You can even add bonus pool to this directly (e.g. new_points = 2000*p + bonus_pool), regardless of whether it's the current bonus pool system or the revamped bonus pool system I suggested. All of these ways of scaling that exclude bonus pool will completely preserve ranks.
And yet, this treadmill effect is still a problem. I have to play to keep my rank, because someone else played. I definately think the best way to rank someone is to reveal MMR and rank accordingly. Eg. a higher mmr player is higher in his division. Of course, there is an uncertainty value. The MMR should subtract the average of the upper and lower bounds of a player's uncertainty. So that way, if a player will occasionally play higher than his mmr more often than he plays lower than his mmr, his mmr rises slightly. And if a player's uncertainty shifts to a tendancy to play less than their mmr, he loses some mmr. If blizz wants to hide ppls mmr to hide their patented algorythms, they simply dont reveal the uncertainty, and the mmr algorythm is impossible to decode.
Another idea i have is just to make active players have no bonus pool. Bonus pool only accrues after 2 days of ladder incativity. If a person is active, he instead loses less points for losing a match, equal to the number of points the game is worth + or - the number of consecutive active days. and wins more points for winning a match. This means that inactive players can catch up, and that the ability to gain points along with mmr gets rid of the need for a treadmill effect.
This way:
- a person cant log In once a day to farm his bonus pool and then get off.
- Active players are rewarded.
- Inactive players can catch up, but the playing field is leveled once they are caught up.
(edited to avoid double post with second idea)
Most of this is basically what I said in the OP. So we almost entirely agree here.
However, I suggest 7 days, not 2, because people schedule their lives around weeks, e.g. some people can only play on weekends. As for your idea to reduce the treadmill effect, what you've suggested is mostly a bonus pool system, where the bonus pool is not displayed. The main difference between this and my suggestion (apart from 7 instead of 2) is that I've also suggested that bonus pool is significantly reduce, and it should be consumed exponentially faster the more you have (because there's no need to make it reward activity, since the leveling system and accounting for sigma serves that role), and it's explained in Section 3.3, why this would reduce the treadmill effect.
|
On January 02 2013 13:26 Excalibur_Z wrote:
ICCup's system started every player at a base value and there were breakpoints that defined new letter grades. Furthermore, point changes per game were weighted based on the grade, so it was harder to lose points as a D than it was as a B. The MotW added bonus points to wins. This was fun for players because it was easy to rank up early on. However it creates an inflation problem because extra points are pumped into the system. So, you can't really use that as a pure skill ranking. Also, every season points were reset which caused problems like a true D-level player playing a Day 1 game of the new season against NaDa.
What defines a pure skill ranking anyway? I agree with you, there was point inflation because of all the new accounts that pumped points into the system. But who is to say inflation is a bad thing? At least you knew where you stood. I agree with your issue about someone like Nada (kinda) but if everyone deosnt start at the same place, what is the point of the ladder anyway? At least if you do it as a point total system, you may not get "the best" (whatever that means) player at the top, but it will be damn close assuming everyone is active (especially with the way iccup changed the point system as you moved up into the ranks). Going from B+ to A- or A was a huge jump because of the competition level and the fact that I believe at A- you lost more points for losses than you gained for wins so it was pretty difficult to stay there if you didn't belong.
I still say something like the motw is good for the community as a whole. It encourages playing new maps or maps you dont like which can lead to an overall skill increase for everyone since people are forced out of their comfort zones.
I know half of the maps on iccup I woulnd't have played had there not been that added incentive of bonus points for playing motw (that and most people wanted the bonus points so it was harder to get games on the regular ladder maps).
It made it cool because it felt like every week or 2 you were trying out a new map which was good because it was fairly easy to get into a rut of playing the same maps OVER and OVER (which I believe is what we see on sc2 now because of the ladder system).
Theres very little experimentation with what we have now. Im glad Kespa will have their own people working on making new maps. Maybe we will see some creativity and things like non buildable (or creepable?) areas on the map. Maybe this restricts warp ins and forces P to build more warp prisms. Things like that.
|
On January 02 2013 01:44 marvellosity wrote:
I don't have much deep to add, I just read it because it was interesting and made good points. I'm a fairly casual low masters player, I just play a few games a week. The one suggestion I didn't like was only updating the ladder weekly. Even though I appreciate it's pointless, I like seeing my points go up and down as I play (even knowing that with my bonus pool and the bonus people of other people in my division, the ranks are mostly meaningless). Basically what I'm saying is that I like immediate feedback when I play games, and it would irritate me not to have it, however meaningless it is ^^
The suggestion only pertains to updating the ladder ranks.
So points would still go up and down. I think that's important so you get feedback about wins and losses and how much they matter.
I think it would be OK if the division ladder were updated in real time, as long as division ranks are removed from the matchmaking page and put into the ladder page in the profile to reduce the treadmill effect. But the main reason for updating a global ladder once weekly is because it takes the ladder snapshot when bonus pool doesn't really have a distortionary effect (paragraph 4 in 3.3), so it displays ladder ranks at the only time that they're "correct", and it greatly reduces the treadmill effect. As for division ladder it's OK to update them in real time because the first point doesn't matter due to division ranks being meaningless.
|
On January 02 2013 13:40 Sadist wrote:Show nested quote +On January 02 2013 13:26 Excalibur_Z wrote:
ICCup's system started every player at a base value and there were breakpoints that defined new letter grades. Furthermore, point changes per game were weighted based on the grade, so it was harder to lose points as a D than it was as a B. The MotW added bonus points to wins. This was fun for players because it was easy to rank up early on. However it creates an inflation problem because extra points are pumped into the system. So, you can't really use that as a pure skill ranking. Also, every season points were reset which caused problems like a true D-level player playing a Day 1 game of the new season against NaDa. What defines a pure skill ranking anyway? I agree with you, there was point inflation because of all the new accounts that pumped points into the system. But who is to say inflation is a bad thing? At least you knew where you stood. I agree with your issue about someone like Nada (kinda) but if everyone deosnt start at the same place, what is the point of the ladder anyway? At least if you do it as a point total system, you may not get "the best" (whatever that means) player at the top, but it will be damn close assuming everyone is active (especially with the way iccup changed the point system as you moved up into the ranks). Going from B+ to A- or A was a huge jump because of the competition level and the fact that I believe at A- you lost more points for losses than you gained for wins so it was pretty difficult to stay there if you didn't belong. I still say something like the motw is good for the community as a whole. It encourages playing new maps or maps you dont like which can lead to an overall skill increase for everyone since people are forced out of their comfort zones. I know half of the maps on iccup I woulnd't have played had there not been that added incentive of bonus points for playing motw (that and most people wanted the bonus points so it was harder to get games on the regular ladder maps). It made it cool because it felt like every week or 2 you were trying out a new map which was good because it was fairly easy to get into a rut of playing the same maps OVER and OVER (which I believe is what we see on sc2 now because of the ladder system). Theres very little experimentation with what we have now. Im glad Kespa will have their own people working on making new maps. Maybe we will see some creativity and things like non buildable (or creepable?) areas on the map. Maybe this restricts warp ins and forces P to build more warp prisms. Things like that.
A pure skill rating system would be one which isn't distorted by whatever artificial bonus or gimmick the designers want to put in. For example, one which isn't distorted by bonus pool or bonus points for MotW.
A pure skill rating system should have certain properties. For example, if the system believes a players skill has increased by X, then it should always go up by X. Giving bonus points for MotW will reward you with X if you beat a player on that map when it's not MotW, but reward you with X+bonus if you beat the exact same player in exactly the same conditions, in a week where that map is MotW. So it fails this very basic property.
Another example, in a pure skill rating system, the idea of a ladder reset makes zero sense. It simply does not make sense to believe a player has 2400 MMR and 93 percentile on one day, and then to believe everyone has 0 MMR and 0 percentile the very next day, because someone has arbitrarily decided that the ladder should be reset. As I've said, things should be somewhat partly reset in a ladder reset because people expect a "clean slate", but it's worth noting that doing so deviates from a pure skill rating system.
As I've said in the OP, the bonus pool, and hence point inflation, is only a bad thing to the extend that it distorts ranks and creates a treadmill effect. And it does so for virtually no valid reason. You talk about a huge jump from B+ to A-, but these types of discontinuities are both pointless and artificial. Why should there be a big jump between B+ and A-? This only make sense if in the real world, there is no one with an intermediate skill level, which is definitely not true. Skill ratings fall in a continuum, to artificially create jumps because someone thought it would be a cool thing to do, violates this basic fact. Furthermore, assuming that 80% of players are in D, is ICCUP suggesting that 80% of players are exactly the same skill? Is that why it doesn't differentiate the skill between these players? Again, clearly this is not true. It's merely an attempt to shoehorn skill ratings into these artificial grades. Again this violates common knowledge about how skill is distributed. This system fails to display basic properties that a system that models skill correctly and purely should have.
You say you know that such a system can be trusted. I don't see why anyone would take such a flawed system seriously. Probably because it's all you had at the time.
The correct way to reward activity and to encourage people to try out whatever you want them to try out is let them work towards cosmetic, non-ladder rewards. For example, through the new level system or by tokens used to buy portraits, units skins, decals, B.net backgrounds, etc.
|
On January 02 2013 14:06 paralleluniverse wrote:Show nested quote +On January 02 2013 13:40 Sadist wrote:On January 02 2013 13:26 Excalibur_Z wrote:
ICCup's system started every player at a base value and there were breakpoints that defined new letter grades. Furthermore, point changes per game were weighted based on the grade, so it was harder to lose points as a D than it was as a B. The MotW added bonus points to wins. This was fun for players because it was easy to rank up early on. However it creates an inflation problem because extra points are pumped into the system. So, you can't really use that as a pure skill ranking. Also, every season points were reset which caused problems like a true D-level player playing a Day 1 game of the new season against NaDa. What defines a pure skill ranking anyway? I agree with you, there was point inflation because of all the new accounts that pumped points into the system. But who is to say inflation is a bad thing? At least you knew where you stood. I agree with your issue about someone like Nada (kinda) but if everyone deosnt start at the same place, what is the point of the ladder anyway? At least if you do it as a point total system, you may not get "the best" (whatever that means) player at the top, but it will be damn close assuming everyone is active (especially with the way iccup changed the point system as you moved up into the ranks). Going from B+ to A- or A was a huge jump because of the competition level and the fact that I believe at A- you lost more points for losses than you gained for wins so it was pretty difficult to stay there if you didn't belong. I still say something like the motw is good for the community as a whole. It encourages playing new maps or maps you dont like which can lead to an overall skill increase for everyone since people are forced out of their comfort zones. I know half of the maps on iccup I woulnd't have played had there not been that added incentive of bonus points for playing motw (that and most people wanted the bonus points so it was harder to get games on the regular ladder maps). It made it cool because it felt like every week or 2 you were trying out a new map which was good because it was fairly easy to get into a rut of playing the same maps OVER and OVER (which I believe is what we see on sc2 now because of the ladder system). Theres very little experimentation with what we have now. Im glad Kespa will have their own people working on making new maps. Maybe we will see some creativity and things like non buildable (or creepable?) areas on the map. Maybe this restricts warp ins and forces P to build more warp prisms. Things like that. A pure skill rating system would be one which isn't distorted by whatever artificial bonus or gimmick the designers want to put in. For example, one which isn't distorted by bonus pool or bonus points for MotW. A pure skill rating system should have certain properties. For example, if the system believes a players skill has increased by X, then it should always go up by X. Giving bonus points for MotW will reward you with X if you beat a player on that map when it's not MotW, but reward you with X+bonus if you beat the exact same player in exactly the same conditions, in a week where that map is MotW. So it fails this very basic property. Another example, in a pure skill rating system, the idea of a ladder reset makes zero sense. It simply does not make sense to believe a player has 2400 MMR and 93 percentile on one day, and then to believe everyone has 0 MMR and 0 percentile the very next day, because someone has arbitrarily decided that the ladder should be reset. As I've said, things should be somewhat partly reset in a ladder reset because people expect a "clean slate", but it's worth noting that doing so deviates from a pure skill rating system. As I've said in the OP, the bonus pool, and hence point inflation, is only a bad thing to the extend that it distorts ranks and creates a treadmill effect. And it does so for virtually no valid reason. You talk about a huge jump from B+ to A-, but these types of discontinuities are both pointless and artificial. Why should there be a big jump between B+ and A-? This only make sense if in the real world, there is no one with an intermediate skill level, which is definitely not true. Skill ratings fall in a continuum, to artificially create jumps because someone thought it would be a cool thing to do, violates this basic fact. Furthermore, assuming that 80% of players are in D, is ICCUP suggesting that 80% of players are exactly the same skill? Is that why it doesn't differentiate the skill between these players? Again, clearly this is not true. It's merely an attempt to shoehorn skill ratings into these artificial grades. Again this violates common knowledge about how skill is distributed. This system fails to display basic properties that a system that models skill correctly and purely should have. You say you know that such a system can be trusted. I don't see why anyone would take such a flawed system seriously. Probably because it's all you had at the time. The correct way to reward activity and to encourage people to try out whatever you want them to try out is let them work towards cosmetic, non-ladder rewards. For example, through the new level system or by tokens used to buy portraits, units skins, decals, B.net backgrounds, etc.
Whats the point of the ladder if everyone is already slotted into a specific place? Don't you run into the problem that iechoic mentioned where the longer you played at a certain level, it becomes tougher to move up when you improve? A system like this doesn't take into account that for most players, skill doesnt increase linearly, often it increases in large jumps. So because I played for a long time before I had a big epiphany and rapidly became a lot better, I move up slower on the ladder than someone who has played fewer games because my sigma is lower? Thats stupid. You aren't measuring my skill at what it is now, you are basically screwing me over because of what my skill once was.
At least in a purely points based system I wouldn't be held back on the ladder from my previous 1000 games. My previous 1000 games have nothing to do with my skill now and shouldnt hold me back from increasing my rank on the ladder.
If you truly are trying to find the best/better players, you are talking about competitive players who don't really care about portraits. Using portraits/achievemetns for things like MOTW would be irrelevant. Why not just separate the ladder into casuals and people who care about the game?
Also this idea about 80% of the ladder being grouped into D is completely arbitrary (the ladder was based on your total number of points, if we are both at D, but I have 1300 pts and you have 1200 pts, I am higher than you on the ladder obv.). You could set the ranking wherever you want. 80% of the ladder could have been B if they were good enough and won enough points. Or you could create an infinite amount of rankings so that each person occupied their own slot. Its irrelevant. The point is you knew were you stood and you moved up the ladder by accumulating points, not some fluxuating MMR system you can't see.
This idea that the ladder measures skill is also kinda laughable. There are so many variables that go into getting wins against someone, the map you played them on, the build order, were they on tilt, were you playing exceptionally well, were they playing exceptionally bad? Did they get cheesed the game before so now they are looking out for cheese? Blind build order luck........
sure if we all play enough games things like that even out......but I dont believe your system is calling for that. Calling someone the best or something like skill is purely arbitrary anyway. What exactly is skill? Say i'm a guy who is a steady player and beats everyone on the ladder except 1 person 99% of the time. That 1 person has my number and just kicks my ass and I never beat him, but he only beats the rest of the ladder 75% of the time. Who is the better player? What does that even mean?
|
On January 02 2013 14:32 Sadist wrote:Show nested quote +On January 02 2013 14:06 paralleluniverse wrote:On January 02 2013 13:40 Sadist wrote:On January 02 2013 13:26 Excalibur_Z wrote:
ICCup's system started every player at a base value and there were breakpoints that defined new letter grades. Furthermore, point changes per game were weighted based on the grade, so it was harder to lose points as a D than it was as a B. The MotW added bonus points to wins. This was fun for players because it was easy to rank up early on. However it creates an inflation problem because extra points are pumped into the system. So, you can't really use that as a pure skill ranking. Also, every season points were reset which caused problems like a true D-level player playing a Day 1 game of the new season against NaDa. What defines a pure skill ranking anyway? I agree with you, there was point inflation because of all the new accounts that pumped points into the system. But who is to say inflation is a bad thing? At least you knew where you stood. I agree with your issue about someone like Nada (kinda) but if everyone deosnt start at the same place, what is the point of the ladder anyway? At least if you do it as a point total system, you may not get "the best" (whatever that means) player at the top, but it will be damn close assuming everyone is active (especially with the way iccup changed the point system as you moved up into the ranks). Going from B+ to A- or A was a huge jump because of the competition level and the fact that I believe at A- you lost more points for losses than you gained for wins so it was pretty difficult to stay there if you didn't belong. I still say something like the motw is good for the community as a whole. It encourages playing new maps or maps you dont like which can lead to an overall skill increase for everyone since people are forced out of their comfort zones. I know half of the maps on iccup I woulnd't have played had there not been that added incentive of bonus points for playing motw (that and most people wanted the bonus points so it was harder to get games on the regular ladder maps). It made it cool because it felt like every week or 2 you were trying out a new map which was good because it was fairly easy to get into a rut of playing the same maps OVER and OVER (which I believe is what we see on sc2 now because of the ladder system). Theres very little experimentation with what we have now. Im glad Kespa will have their own people working on making new maps. Maybe we will see some creativity and things like non buildable (or creepable?) areas on the map. Maybe this restricts warp ins and forces P to build more warp prisms. Things like that. A pure skill rating system would be one which isn't distorted by whatever artificial bonus or gimmick the designers want to put in. For example, one which isn't distorted by bonus pool or bonus points for MotW. A pure skill rating system should have certain properties. For example, if the system believes a players skill has increased by X, then it should always go up by X. Giving bonus points for MotW will reward you with X if you beat a player on that map when it's not MotW, but reward you with X+bonus if you beat the exact same player in exactly the same conditions, in a week where that map is MotW. So it fails this very basic property. Another example, in a pure skill rating system, the idea of a ladder reset makes zero sense. It simply does not make sense to believe a player has 2400 MMR and 93 percentile on one day, and then to believe everyone has 0 MMR and 0 percentile the very next day, because someone has arbitrarily decided that the ladder should be reset. As I've said, things should be somewhat partly reset in a ladder reset because people expect a "clean slate", but it's worth noting that doing so deviates from a pure skill rating system. As I've said in the OP, the bonus pool, and hence point inflation, is only a bad thing to the extend that it distorts ranks and creates a treadmill effect. And it does so for virtually no valid reason. You talk about a huge jump from B+ to A-, but these types of discontinuities are both pointless and artificial. Why should there be a big jump between B+ and A-? This only make sense if in the real world, there is no one with an intermediate skill level, which is definitely not true. Skill ratings fall in a continuum, to artificially create jumps because someone thought it would be a cool thing to do, violates this basic fact. Furthermore, assuming that 80% of players are in D, is ICCUP suggesting that 80% of players are exactly the same skill? Is that why it doesn't differentiate the skill between these players? Again, clearly this is not true. It's merely an attempt to shoehorn skill ratings into these artificial grades. Again this violates common knowledge about how skill is distributed. This system fails to display basic properties that a system that models skill correctly and purely should have. You say you know that such a system can be trusted. I don't see why anyone would take such a flawed system seriously. Probably because it's all you had at the time. The correct way to reward activity and to encourage people to try out whatever you want them to try out is let them work towards cosmetic, non-ladder rewards. For example, through the new level system or by tokens used to buy portraits, units skins, decals, B.net backgrounds, etc. Whats the point of the ladder if everyone is already slotted into a specific place? Don't you run into the problem that iechoic mentioned where the longer you played at a certain level, it becomes tougher to move up when you improve? A system like this doesn't take into account that for most players, skill doesnt increase linearly, often it increases in large jumps. So because I played for a long time before I had a big epiphany and rapidly became a lot better, I move up slower on the ladder than someone who has played fewer games because my sigma is lower? Thats stupid. You aren't measuring my skill at what it is now, you are basically screwing me over because of what my skill once was. At least in a purely points based system I wouldn't be held back on the ladder from my previous 1000 games. My previous 1000 games have nothing to do with my skill now and shouldnt hold me back from increasing my rank on the ladder. If you truly are trying to find the best/better players, you are talking about competitive players who don't really care about portraits. Using portraits/achievemetns for things like MOTW would be irrelevant. Why not just separate the ladder into casuals and people who care about the game? Also this idea about 80% of the ladder being grouped into D is completely arbitrary (the ladder was based on your total number of points, if we are both at D, but I have 1300 pts and you have 1200 pts, I am higher than you on the ladder obv.). You could set the ranking wherever you want. 80% of the ladder could have been B if they were good enough and won enough points. Or you could create an infinite amount of rankings so that each person occupied their own slot. Its irrelevant. The point is you knew were you stood and you moved up the ladder by accumulating points, not some fluxuating MMR system you can't see. This idea that the ladder measures skill is also kinda laughable. There are so many variables that go into getting wins against someone, the map you played them on, the build order, were they on tilt, were you playing exceptionally well, were they playing exceptionally bad? Did they get cheesed the game before so now they are looking out for cheese? Blind build order luck........ sure if we all play enough games things like that even out......but I dont believe your system is calling for that. Calling someone the best or something like skill is purely arbitrary anyway. What exactly is skill? Say i'm a guy who is a steady player and beats everyone on the ladder except 1 person 99% of the time. That 1 person has my number and just kicks my ass and I never beat him, but he only beats the rest of the ladder 75% of the time. Who is the better player? What does that even mean?
What is your point? Who said that everyone is already slotted into a specific place and can't move?
The post I made addressing iEchoic's concerns with TrueSkill is for TrueSkill, not the SC2 ladder system. The complaint that you can't move in the ladder because of your previous history doesn't exist in the SC2 ladder system because sigma increases when the outcome of a game is surprising, unlike TrueSkill where it decreases. So this point that you've made is irrelevant to SC2.
So ICCUP has both points and a grade like D? If this is true, the argument that 80% of the players that are in D are ranked the same, is no longer valid. But the argument that it's dumb that 80% of players are in D or C or whatever is still valid. Why is it suboptimal for it to be evenly spread like in SC2 (approximately 20/20/20/20/20 of active player)? In addition, you have not addressed the other complaints I've made against the ICCUP system.
Really, it's laughable that the ladder measures skill? I've already addressed this point in the OP:
There are also claims that skill simply cannot be measured accurately with MMR and that there’s no way to account for uncertainty about MMR, so that global ladders are meaningless. But this is just completely wrong. The uncertainty about MMR, is already measured by the system, and I’ve suggested that it should be explicitly included into points by using MMR-1.96*sigma. And even if sigma isn't directly used to calculate points it still tends to reduce as games are played. Skill can be measured with good accuracy, as shown by the near 50-50 matchmaking SC2 achieves using MMR, and the empirical evidence from similar skill rating system such as TrueSkill which have remarkable success.
You base the fact that skill can't be measured accurately because of all the vast number of factors that go into winning a game. That's like saying, the trajectory of a rocket cannot be calculated correctly due to the trillions of particle interactions that constitutes the motion of a rocket and its surroundings. The only thing that matters for skill is whether you're able to win. If you can beat 95% of players with nothing but a cannon rush, and that is the only trick you can pull, then guess what? How well you macro doesn't matter. The depth of your game knowledge doesn't matter. You're better than 95% of all players and deserve a percentile of 95, simply because you can beat 95% of players with a cannon rush.
And even if it were true that we can't measure skill correctly, then what? So we shouldn't try? We should knowingly and deliberately fudge the best measure of skill that we have because you find it incredulous that the MMR could possibly be correct?
You talk about a situation where A > B > C > A. Does it make sense to rank these players and say that A is better than B or C? Well if you account for how these players match up against other players instead of just each other, then you can rank them. And that's what MMR does.
And then you argue that competitive players don't care about portraits. So what? What's your point? That competitive players will be unwilling to play the ladder unless they are rewarded with ladder points beyond what they deserve and beyond what is mathematically correct? And you base this on what? If they don't play then their ranks will fall. If they're competitive then they should play to prevent that. And you're solution to this non-problem is that we should knowingly screw up the ranks just to get these people to try the MotW. Really.
Again, I'm highly confused by what your point is. Replace the SC2 ladder system with the ICCUP system? That it's desirable to reward people for playing by giving them ladder points with no regard for how that affects the accuracy of ranks?
|
United States12235 Posts
The SC2 ladder is more fluid than TrueSkill in that your sigma never shrinks so much that you get locked somewhere. Lose 100 games and you'll find yourself a few leagues lower, win 100 games and you'll move up higher.
The "80% of ICCup is D" simply follows the logic that everyone has to start somewhere. In every system, the starting value is going to be the most common just because people will create an account and not do anything with it, or they'll play one game or two games and then quit. The spread from the lowest player to the highest depends on how frequently people are playing games, and the higher your rating, the higher your win percentage has to be to maintain that rating gap over the next-highest player. I believe in ICCup there is a rule where you can't earn points from someone who is X grades below you, and that's not uncommon for most ranking systems. You're right that it's arbitrary, but then the entire grade system is arbitrary as well (weighting wins more heavily than losses in D than in C than in B than in A).
I think unranked ladder and race levels are just a couple of ways Blizzard is trying to separate the ladder into casual and hardcore in HotS.
You're right that players are prone to streaks, but yes, over enough games that balances out, and that's essentially the idea. Everybody is going to plateau somewhere, and realistically they're not going to fall very far below that either. Therefore their actual skill is somewhere in the middle. MMR isn't there to say "your skill is 1738, period.", it changes with every game and maybe you'll hover between 1725-1750 or 1700-1800 or 1600-1900.
|
Thanks for the detailed reply about my TrueSkill question, was enlightening. I've taken a full read through the OP with your points in mind and agree with everything you've said.
I do think there's value in creating discrete skill rankings (such as in ICCUP's A/B/C/D system), but think there should be higher granularity. I like TrueSkill's 1-50 ranking system, and think it's more enjoyable to play when your rating is bucketized into 50 displayed values, simply because it fosters easier discussion with people and is easier to conceptualize ("How good are you? I'm a 42 TS").
My ideal rating system would essentially be OP's, but with MMR bucketized into ~50 rankings, similar to TrueSkill. This has more to do with branding and psychological response more than anything functional, though.
|
United States12235 Posts
On January 02 2013 17:54 iEchoic wrote:
Thanks for the detailed reply about my TrueSkill question, was enlightening. I've taken a full read through the OP with your points in mind and agree with everything you've said.
I do think there's value in creating discrete skill rankings (such as in ICCUP's A/B/C/D system), but think there should be higher granularity. I like TrueSkill's 1-50 ranking system, and think it's more enjoyable to play when your rating is bucketized into 50 displayed values, simply because it fosters easier discussion with people and is easier to conceptualize ("How good are you? I'm a 42 TS").
My ideal rating system would essentially be OP's, but with MMR bucketized into ~50 rankings, similar to TrueSkill. This has more to do with branding and psychological response more than anything functional, though.
There's something to be said for granularity that would reduce some of the confusion and frustration present in the SC2 ladder system. Technically speaking, originally the ladder had 24 "buckets" via division tiers whereas now there are only 7, but the biggest problem with the tiers was they were completely opaque. Most commonly I've heard people complain that they're "high X/low Y" because they either actually teeter on the border between leagues or they at least think they do because of a larger matchmaking range. I remember Dustin Browder himself in an interview said he's "high Plat/low Diamond" which really defeats the purpose of the league system in my opinion because the Silver-through-Plat buckets are narrow enough that there's matchmaking overlap, but the Bronze and Diamond buckets are large enough that players can feel stuck. It's possible they changed divisions from fixed/tiered buckets to a more organic granular approach (where on the aggregate, top 3 in a division would equate to the top 3% of a league) but there's so much volatility from one division to the next that this doesn't neatly translate.
|
On January 02 2013 12:16 Sadist wrote:Going 30-0 and being on top the ladder SHOULD be discouraged. Yea that person may be the best player, but thats kinda dumb. They can just sit there forever at their 3000 ELO rating and never play a game again. This brings up another issue I have with ladder (It actually isn't related to ELO, but the current system), which is the whole placement thing. Before all my friends quit this game, a few were in Master league and all they did for most of the last year or two after achieving Master was do their placement game and then not play all season, then do the same thing the next season. My one friend did this for the better part of a year, staying in Master playing probably 10 1v1s in total. If you look at most divisions the bottom 10-20 spots are like this almost every time. It seems dumb to me that people can do this. If Blizzard insists on keeping this convoluted ladder system then they should have something in place to make sure people that are placed are actually playing so the bottom quarter of each league isn't filled with people who only played one game. It is pretty demoralizing trying to get into Master league and working my butt off when the bottom chunk of that league doesn't even play yet can maintain their spots by loading up the game once per season, getting matched once then either cheesing or F10-Ning then going on with their day.
I would rather have ELO personally as well. I like having a solid metric to tell where I am at rather than "Oh I sometimes play people in a higher league so I might be getting better but it might actually be that they are playing horribly and might be demoted soon but I don't know".
|
On December 22 2012 12:30 paralleluniverse wrote:Show nested quote +On December 22 2012 06:48 Apolo wrote:
Not only should we be allowed to hide our season history but also our current stats and rank. A part of ladder anxiety is the feeling of others judging our lack of skill. The bad feeling of being in bronze or silver etc is exacerbated by the fact that you wear it like a badge on top of your hat for everyone to see. It's like having a banner saying "I suck".
Not only that, but you play not knowing when, if you will be demoted. Each loss can mean demotion. Actually with this kind of system it would be surprising if there wasn't ladder anxiety. I completely disagree with hiding any current season stats. I agree that previous season history should be hidable because it's the only way to permanently stuff up your account, and then it can never be changed. However, current season stats cannot be screwed up because MMR is self-correcting, so why hide it? Previous season history is old information, so it's not so important. The most fundamental purpose of a ladder system is to rank correctly, not to hide stats because of the hurt feelings of those players with ladder anxiety. These suggested changes and unranked play should already solve a lot of ladder anxiety. You're absolutely right about Blizzard's league system stigmatizing bad players. I find it puzzling when people defend the current system as good for casuals even though Blizzard brands a mediocrity badge called “Bronze League” on 42.1% of players forehead. One of the advantages of a global ladder is that you could scrap the league system or make promotion and demotion criteria clearly stated in terms of percentiles. While in a global ladder, a person with a 0-20 percentile would be equivalent to Bronze, removing the league system and confining the percentile to the ladder summary page prevents these players from being force to wear an "I suck" on their heads, and it removes the anxiety of possibly getting demoted. No leagues system means no demotion. It's also deprives them of a derogatory label. tI might not make much of a difference, but it definitely wouldn't make things worst than they already are.
How would being ranked 999,999 out of 1,200,000 people make your more likely to ladder than being ranked 50th in bronze? It actually gives them a more derogatory label of being worse than 1million others rather than how ever many people are in their division. When you suck, there is no ranking that will make you feel better about it. If you are diamond you might prefer to say i'm ranked 7,000th in the world rather than rank 1 diamond, but then you are in the top 3% of players and while you still suck ass compared to pro's, compared to the other 97% you rock.
There is no way to make a bronze player feel better about being bronze, there simply isn't. You can rank them however you want, they are still ranked and still have that stigma attached to them. The moment you rank someone, in any way, they will care about that ranking and won't want to ruin it. I personally don't care whether i'm rank 1 diamond or rank 50 master, I play because I enjoy the competition and losing my current spot cause I go on a loss streak doesn't bother me.
I've experienced a global ladder and watched as I climbed those thousands of ranks, it was fun for me, not because of anything other than I like the competition. If you don't like to see how you rank against others, then you aren't going to like it. Doesn't matter how its presented. I played LotR: Battle for Middle Earth and at one point was ranked in the top 200 in the world.... didn't matter in the slightest to me, I played cause I loved the competition. I didn't all of a sudden stop playing to keep my spot... because if i did that, I'd lose it anyways lol. Thats the nature of competition.
Stop trying to coddle newbies, give them unranked play and leave them to it.... there is nothing you can do that will make them feel better about being ranked, they will always have "ladder anxiety" because they care more about epeen than competition.
|
lol
LOL (League of Legends) has "copied" SC2's ladder system. Details here: http://na.leagueoflegends.com/news/new-league-system-coming-soon-ranked
I've never played LOL. SC2's ladder system has ample flaws as documented in the OP. Yet, despite this, LOL managed to vastly improve on SC2's ladder system. However, this new ladder system has major blunders of its own.
The article starts with the classical strawman argument against a global ladder:
We decided to move to the new league system for a few reasons. For starters, having a single ladder with all ranked players doesn't provide a lot of incentive for advancement. When you’re ranked 290,000 and have 289,999 opponents left to pass on the way up, that process can seem meaningless and interminable. Tiers and divisions also provide milestones and manageable goals you can strive to achieve at your skill level. Through leagues we can move away from focusing on a single number as the core indicator of a player’s skill, and instead move toward something more compelling: competition on a small ladder with a relatable number of opponents.
It fails to mention that a percentile can simply be used to overcome this problem, as I've explained in the OP:
A percentile should be used. For example, 84.2 instead of 3756 out of 20000. This number is very easy to understand, 84.2 simply means that you’re better than 84.2% of all players that are ranked (this is exactly how university admission ranks work in Australia).
So this isn't even a problem with a global ladder, it's an issue with how your place in the global ladder is displayed. In fact, if they're so determined to stop using numbers, they can simply rename 90 percentile as Diamond/Tier 3/50 or 39 percentile as Silver/Tier 5/75. Oh wait, their new ladder system uses these labels. But then, why not use them on the Elo percentile? That is, why not have a global ladder or call it that?
Now, this new ladder system is essentially a global ladder. Everyone is ranked comparably on a line, unlike the SC2 league system. Their ladder is broken into 6 leagues (named Bronze to Diamond, with Masters called Challenger instead). Within each league (except Challenger), there are 5 division tiers, where each division tier is higher skilled than the previous division tier. So in effect there's really 26 SC2-style leagues, and it is intended that we can infer that Tier 1 is better than Tier 2, etc. Within each division, players are ranked by points from 0 to 100, essentially a percentile. It's unclear whether division points are effectively a percentile for the division or for all divisions in the division tier. If it's the former, then division points are as meaningless as they are in SC2 (as explained in the OP). If it's the latter, then that's good and this is basically a rebranded global ladder. And the following argument I've used against the SC2's ladder system cannot be applied to LOL.
Leagues do not solve the problem of division ranks being meaningless and there being no way to get a reasonable measure of your skill relative to all active players. The 5 leagues other than Masters and GM cover an approximately 20% skill range, in the sense that Platinum league contains players in the top 20%-40%. This is a very large skill gap.
In LOL, a 3.8% skill range is small enough, and you can use division points to calculate a finer global percentile.
Also, there's a clearly articulated promotion criteria, unlike the SC2 ladder system. To get promoted in LOL, you just need to get 100 division points (i.e. get to the top of your division), and then win a best of 3. This is an imperfect promotion criteria. What's the purpose of requiring the player to win a best of 3 when the system can already determined the player's skill through probably hundreds of games in the history? What is the mathematical rationale that these 3 games should be given greater weight? What statistical evidence is there to suggest that this approach will model player skill more accurately? Indeed, singling out 3 particular games is completely arbitrary. They probably chose this just because it sounds like something cool to do, despite there being no mathematical rationale behind it. In fact, this is a distortionary factor. What if you win against 1 higher skilled team and lost against 2 even higher skilled teams, so that on net, you gain rank? Like the SC2 ladder system, the goal should be to promote when the system is reasonably sure of the players skill as measured by uncertainty about Elo (or MMR), and this should just be baked into the calculation of division points. But wait, Elo has no uncertainty. This is one of the reasons why Elo is flawed. Taking a Bayesian approach, like the SC2 ladder system, TrueSkill, or Glicko would be far better than Elo. But despite these imperfections, at least LOL has an explicit and transparent promotion criteria.
Now, this is just crazy:
Losing a ranked game in the league system will cost some of your League Points. If you’re already at the bottom of your division, this may mean falling back to the previous division . Once you’ve earned a skill tier, however, you can never be demoted to the previous tier unless you stop playing for a prolonged period of time.
In LOL, you can't be demoted to a lower league, only to a lower division. Why not? This will be a problem when the system has erroneously placed a new player into the wrong league, or if a player just manages to scrape into a league and becomes relatively worse afterwards. This is a needless, arbitrary, and artificial gimmick that prevents the ladder from self-correcting when player skill changes or when the ladder has made a mistake. It distorts the ranks, messes up the ladder, and for absolutely no reason at all. This is just baffling. What if a player, after being promoted to Diamond, decides to smurf by losing 50 games? He can smurf Bronze players while remaining in Diamond. This is absurd. At least LOL avoids the problem in SC2 whereby players can play 1 game at the start of the season to get put into Diamond or Masters and camp on that league regardless of inactivity.
And finally, there's the fact that Elo will now be hidden:
Our matchmaking system still matches you by skill level, but this “rating” is no longer visible and does not have any bearing on your seasonal rewards or ladder standing. Your standing in your league is now determined by your tier, division and League Points, not your matchmaking rating.
Why? To measure skill, why should the matchmaking system use Elo and the ranking system use leagues and divisions instead? Obviously, one of these is a better measure of skill. So why not use that one everywhere? Well, the answer to this question in SC2 is that using different skill measure for ranking and matchmaking allows for uncertainty about skill to be taken into account (although points and bonus pool do a really bad job as explained in the OP), to allow points to inflate via the bonus pool (this could be done far more optimally as explained in the OP), or to allow points to reset in a season reset without effecting MMR.
But none of these reasons apply to LOL. Elo can't take into account the uncertainty about skill because it sucks. Points don't inflate in LOL because Riot is smart enough to not copy the egregiously terrible SC2 bonus pool system that most of the OP is devoted to demolishing. And since LOL's league and divisions break players into 3.8% skill range, a season reset will have virtually no effect.
So there appears to be no reason why Riot couldn't have, as I've explained above, simply relabeled an Elo percentile of 90 as Diamond/Tier 3/50 or a percentile of 39 as Silver/Tier 5/75. Then there would be no reason to hide Elo, as the players league, division tier and division points would directly correspond to Elo. Problems of flip-flopping between leagues could be solved by the suboptimal, but workable, idea of having a best of 3 when the player hits 100 division points. Nothing more needs to be done.
In conclusion, in a hilarious and unexpected move LOL has copied SC2's league system. It has disowned it's previous global ladder system, despite it's current ladder system being effectively a rebranded global ladder system. Riot has avoided the flaws of the current SC2 ladder system, while still offering a global ladder, clear promotion criteria, and not implementing the bonus pool system. Yet, the LOL league system makes several major mistakes. Some design choices seemingly have no defensible rationale. A lot of these problems with the SC2 and LOL ladder system could have been avoided by asking 3 questions:
1. What is the purpose of <obfuscating factor X>?
2. Is the above purpose legitimate and desirable?
3. If so, is there a better way to achieve this without introducing artificial factors that distort ranks?
And that's what I've set out to do in the OP.
On balance, the new LOL ladder system (at least according to the description given), if it stopped using Elo, would probably be better than SC2's current ladder system. But it makes mistakes that Blizzard should avoid. In my opinion, the ladder system I've outlined in the OP is still preferable to both.
|
|
|
|
|
|
EPT
