|
On May 19 2013 21:49 KalWarkov wrote:Show nested quote +On May 19 2013 21:41 a_flayer wrote: Right now the system also creates unfair situations when giving games of western tournaments the same value as korean tournaments. When two koreans go vs each other on some of the western team leagues, and the game is played on the US, Koreans living in Europe/US have a significant advantage. Think of things like lag or being used to lag and possibly even motivation based on the fact that the korean living in EU/US doesn't play in GSL/PL/GSTL and will thus give more in these insignificant online things). I believe this may result in things like ForGG and Polt being ranked higher than they would be, thus making Aligulac show inaccurate standings in terms of player skill.
ForGG is not a very good player. Why does Lucifron say that ForGG is the best player in europe right now, taking MVP and MC into account as well?
You're trying to use logic to argue with an irrational anti-fan. :/
|
On May 19 2013 21:46 Abominous wrote: To prevent farming maybe you could implement something like elo that once the rating difference is too high, player simply does not gain any rating from winning that match. I suppose that would put Forgg way aside, since he's farmed insignificant Europeans for quite a while!
Also some kind of weight factor for the matches/tournaments might help as well, as suggested before. The amount of points a player gains from a victory does take the rating of the other player into account.
Stephano's 2-1 vs Thorzain still meant Stephano lost a few points because he was so much higher rated vT than T'zains vZ.
|
Maybe we should make a FAQ of all the ideas to improve the rating, and why they wouldn't work. It's not like the suggestions made in this thread are bad. They're not bad at all. It's just that we (for the most part) thought of them already, and it's usually not quite as simple as it first sounds.
For instance, tying the rating to a prize pool is an entirely reasonable idea. The higher the prize pool, the better the players, and the more serious they will be about winning, right? Well, yes. Kinda. But then you have to take into account the existence of qualifiers, which have no prize pool at all. Or the concept of tournaments that offer only a small prize pool plus the participation in a larger tournament (Code A). Or the fact that WCS Korea will be weighted exactly like WCS NA, which surely isn't right? And suddenly the idea seems a whole lot more complicated than you initially thought.
|
On May 19 2013 21:54 Conti wrote: Maybe we should make a FAQ of all the ideas to improve the rating, and why they wouldn't work. It's not like the suggestions made in this thread are bad. They're not bad at all. It's just that we (for the most part) thought of them already, and it's usually not quite as simple as it first sounds.
I have a feeling that a more comprehensive FAQ would solve nothing...
|
On May 19 2013 21:32 MasterOfPuppets wrote:Show nested quote +On May 19 2013 21:21 figq wrote:On May 19 2013 20:49 v_lm wrote: I agree. Or at least implementing a "at stake" factor. Indeed, it would be nice if each match/game had a weight, related to the amount of money on the line for it. Sure, sometimes it's pride and glory, rather than money, but still money is a good measurable factor. It's really not. Here's the thing, if you introduce a weighing system, it will be arbitrary at least to some extent, and it will produce "unfair" situations. For example, if you weigh by prize pool, do you seriously think a LAN like Assembly or The Gathering or ONOG or even some DreamHacks and IEMs are as hard overall as ProLeague or the Code A Qualifiers? BB has long been considering a weighing system, the only thing is that no matter how fair it may seem, none is completely fair. To me, the best idea would be to weigh the tournament based on the average rating of the participating players. But even then, what if you end up with a very stacked Code S where upsets happen and many favourites fall to objectively lesser players, and the champion has an absurdly easy path to victory? (July 2011 comes to mind ) That would inflate his rating more than a bit... So it's tricky... Proleague and Code A introduce two important problems with calculating "money on the line" (which is the real problem):
1. Proleague (and other Team Leagues): the players may be motivated by team rewards that aren't public. So there's still probably some value for "money on the line", but we don't have access to it. 2. Code A (and other initial stages of tournaments) may not have much "money on the line" for the match itself, but winning it unlocks potential access to a lot more "money on the line". What factor to put in front of such possibility-for-money is questionable. So again, there is some monetary reason behind the importance of the match, but we can't calculate it with objective precision.
|
What about an "at stake" factor proportional/correlated to the offline/online part AND the mean ELO at a certain round ? That way winning in "stacked" tournaments counts more than winning in random tournaments. EDIT : Ok it has already been said, however I don't think it would be an issue
ex : July 2011 AMONG Nestea's results the GSL Ro8 counts more than other results HE might have shown (however I dont know if he participated in a lot of other tournament at that time) but this inflates not that much his ELO cause it remains results against (mostly) non top tier player. Byun however who 2-0ed Polt the former GSL champion gains a lot more points for his win in the Ro8 than Nestea.
Other example : Let's say Dimaga wins vs Happy in WCS EU. The level is quite stacked but Happy is quite an easy opponent + Show Spoiler +(at least aligulac easy ) compared to, say, Lucifron and ForGG. Dimaga doesn t get more points by winning Happy because the global level is high, he gets the same "raw" amount of points (based on Happy's ELO) but multiplied by an amount correlated to the mean ELO of all the player of the Ro8 while his loss vs the same Happy in Russia vs Ukraine is multiplied by an amount correlated to the mean ELO of the russian and ukrainian players.
|
On May 19 2013 22:25 v_lm wrote:What about an "at stake" factor proportional/correlated to the offline/online part AND the mean ELO at a certain round ? That way winning in "stacked" tournaments counts more than winning in random tournaments. EDIT : Ok it has already been said, however I don't think it would be an issue ex : July 2011 AMONG Nestea's results the GSL Ro8 counts more than other results HE might have shown (however I dont know if he participated in a lot of other tournament at that time) but this inflates not that much his ELO cause it remains results against (mostly) non top tier player. Byun however who 2-0ed Polt the former GSL champion gains a lot more points for his win in the Ro8 than Nestea. Other example : Let's say Dimaga wins vs Happy in WCS EU. The level is quite stacked but Happy is quite an easy opponent + Show Spoiler +(at least aligulac easy ) compared to, say, Lucifron and ForGG. Dimaga doesn t get more points by winning Happy because the global level is high, he gets the same "raw" amount of points (based on Happy's ELO) but multiplied by an amount correlated to the mean ELO of all the player of the Ro8 while his loss vs the same Happy in Russia vs Ukraine is multiplied by an amount correlated to the mean ELO of the russian and ukrainian players. Your idea would mean that Happy would lose more points for losing to DIMAGA in WCS EU also. So did Happy play worse in WCS EU because the overall mean skill was higher? Or did he play better but still lost and shouldn't be punished so hard?
People forget, or doesn't know, that it goes both ways. Losing in GSL Code S isn't really that bad because the competition is so fierce. So we want to reward the winner a lot, but also not punish the loser too much?
|
You can argue a lot about data analysis and the like (this goes for everyone, not just the creators of the ranking system), but at the end of the day I think the long discontinued TL power rank was by far the most accurate method of all. Sure, it was 100% subjective, but I was hard pressed to disagree with any of the rankings from 2008 to 2011.
|
On May 19 2013 23:53 shadymmj wrote: You can argue a lot about data analysis and the like (this goes for everyone, not just the creators of the ranking system), but at the end of the day I think the long discontinued TL power rank was by far the most accurate method of all. Sure, it was 100% subjective, but I was hard pressed to disagree with any of the rankings from 2008 to 2011. In BW there were much fewer tournaments and games (at least that is my understanding of the scene). Which made it a bit easier to follow and rank players. Everything was Kespa, and you only needed to follow one scene (The Korean) to have an idea of the skill-levels of the players. Not that I disagree though. But the scene has changed a lot.
|
East Gorteau22261 Posts
On May 19 2013 20:16 Arceus wrote: a bit off topic but why TL writers stop doing the Power Ranking, now that we have regular SPL and GSL going on. There is too much to do and not enough people to do it right now, even with 3 or 4 additional dedicated writers. I wasn't around when Power Ranks stopped, but I'm guessing lack of resources was the main reason.
|
Any power rank that has player like ForGG as second best player in the world doesn't do much to convience me about it's credibility. I mean, just look at this history, against any decent player, he loses.
|
On May 20 2013 00:51 Odoakar wrote: Any power rank that has player like ForGG as second best player in the world doesn't do much to convience me about it's credibility. I mean, just look at this history, against any decent player, he loses. This is not a power rank.
|
On May 19 2013 23:12 Grovbolle wrote:+ Show Spoiler +Your idea would mean that Happy would lose more points for losing to DIMAGA in WCS EU also. So did Happy play worse in WCS EU because the overall mean skill was higher? Or did he play better but still lost and shouldn't be punished so hard?
People forget, or doesn't know, that it goes both ways. Losing in GSL Code S isn't really that bad because the competition is so fierce. So we want to reward the winner a lot, but also not punish the loser too much? Yeah, that's my point, and it's not about rewarding anybody, it is about to give more credit to performance done in big tournaments. I'm OK with Happy losing points overall if he's lets say 3-3 with Dimaga but lost 2/1 a series from WCS EU (lets say its a BO3 for the sake of argument) and won 2/1 in UKR vs RUS showmatch. You aren't ? Thank you for answering and discussing my feedback by the way.
|
|
On May 20 2013 00:38 Zealously wrote:Show nested quote +On May 19 2013 20:16 Arceus wrote: a bit off topic but why TL writers stop doing the Power Ranking, now that we have regular SPL and GSL going on. There is too much to do and not enough people to do it right now, even with 3 or 4 additional dedicated writers. I wasn't around when Power Ranks stopped, but I'm guessing lack of resources was the main reason.
There is no point in doing it either because..
On May 20 2013 00:23 Grovbolle wrote:Show nested quote +On May 19 2013 23:53 shadymmj wrote: You can argue a lot about data analysis and the like (this goes for everyone, not just the creators of the ranking system), but at the end of the day I think the long discontinued TL power rank was by far the most accurate method of all. Sure, it was 100% subjective, but I was hard pressed to disagree with any of the rankings from 2008 to 2011. In BW there were much fewer tournaments and games (at least that is my understanding of the scene). Which made it a bit easier to follow and rank players. Everything was Kespa, and you only needed to follow one scene (The Korean) to have an idea of the skill-levels of the players. Not that I disagree though. But the scene has changed a lot.
Still a lot of us would pick and prod at DJEtter's picks when it got released and anyone else who did it for that matter. There was still a lot of debate even with so few tournaments like OSL, MSL, Survivor/Challenge qualifiers, PL, GOMClassic, etc.
On May 20 2013 00:56 Grovbolle wrote:Show nested quote +On May 20 2013 00:51 Odoakar wrote: Any power rank that has player like ForGG as second best player in the world doesn't do much to convience me about it's credibility. I mean, just look at this history, against any decent player, he loses. This is not a power rank.
Precisely. It shouldn't be treated as such.
|
I never quite understand these, as ForGG has essentially never accomplished anything in SC2 (that I am aware of O.O). He is always so highly regarded as the favorite then goes out in such early/mid rounds.
|
On May 20 2013 13:57 v_lm wrote:Show nested quote +On May 19 2013 23:12 Grovbolle wrote:+ Show Spoiler +Your idea would mean that Happy would lose more points for losing to DIMAGA in WCS EU also. So did Happy play worse in WCS EU because the overall mean skill was higher? Or did he play better but still lost and shouldn't be punished so hard?
People forget, or doesn't know, that it goes both ways. Losing in GSL Code S isn't really that bad because the competition is so fierce. So we want to reward the winner a lot, but also not punish the loser too much? Yeah, that's my point, and it's not about rewarding anybody, it is about to give more credit to performance done in big tournaments. I'm OK with Happy losing points overall if he's lets say 3-3 with Dimaga but lost 2/1 a series from WCS EU (lets say its a BO3 for the sake of argument) and won 2/1 in UKR vs RUS showmatch. You aren't ? Thank you for answering and discussing my feedback by the way. Not sure if I am. On one side, winning in a high pressure tournament is harder (at least we assume so). But if winning is harder, losing is also more "forgiveable", because it was a harder tournament. I do understand you idea though. And of course we wan to discuss feedback, we are always trying to make the rating better
TheBB wrote a new part in the FAQ for this
Games in regular online tournaments shouldn't count as much as Code S. Well, first we have to realize that games are weighted, in a sense, by opponent skill. You get more points for beating a higher rated opponent than a lower, and you lose more points by losing to a lower rated opponent than a higher. In addition it is worth considering that simply weighing games higher will not automatically increase the rating of those playing. The winners will gain more points, true, but the losers will also lose more. The mean rating of the players playing will not change.
So aside from this, how should this weighing work?
Even stronger weighing by opponent? The "weighing" is a result of a Bayesian inversion formula depending on the underlying probability model chosen. It's not something that can just be changed, that is, there's no parameter encoding this. It's a much deeper mathematical concept.
Weighing by mean rating of opponent in a round? Well, why should this be any better than weighing by the actual opponents faced, which is what we already do?
Weighing by prize pool? The theory goes that strong players are likely to "try harder" if the prize is higher. There is some merit to this idea, but there are also problems. Some tournaments offer prizes in equipment, and not money. Some offer qualification to a higher tier. For example, there is no monetary prize in the GSL Up and Down groups, but nobody would question the incentive to win there. In addition, there are internal team incentives which are not generally public knowledge. And, additionally, if a player knowingly plays weaker in some games, should that not be reflected in the ratings?
Weighing by tournament? These arguments usually involve some classification of events into tiers of importance with coefficients associated with each level. This approach runs into the complexity problem. With five levels (say), the model becomes far more complicated for what is not shown (yet, anyway) to be reasonable benefit.
Weighing by online and offline? Yes, this is a legitimate idea and probably the one closest to being implemented. We have working experimental code with this feature already.
|
On May 20 2013 14:51 UberNuB wrote: I never quite understand these, as ForGG has essentially never accomplished anything in SC2 (that I am aware of O.O).
http://wiki.teamliquid.net/starcraft2/ASUS_ROG_Summer_2012 http://wiki.teamliquid.net/starcraft2/2012_DreamHack_Open/Valencia
Not including more minor LANs or online tournaments, which he crushes on a fairly regular basis.
On May 20 2013 14:51 UberNuB wrote: He is always so highly regarded as the favorite then goes out in such early/mid rounds.
You mean just like Bomber? (and MarineKing, as of late)
On May 19 2013 21:46 Abominous wrote: To prevent farming maybe you could implement something like elo that once the rating difference is too high, player simply does not gain any rating from winning that match.
What about ProLeague?
There are a lot of players in there with a fairly low rating (think 900-1200 range) that are still subjectively considered much better than foreigners, say, in the 1200-1500 range. And yet ProLeague is arguably the hardest or 2nd hardest tournament in the world, depending on your perspective. All ratings aside, it's fairly obvious that said A/B-teamers would make short work of most foreigners if pitted against them, just like top foreigners make short work of the mid and lower tier foreigners. But instead they get to face FlaSh/Bogus/HerO/sHy/Soulkey, and not low/mid-tier foreigners, which is why this misleading discrepancy exists. Do you think FlaSh beating 3 of these aforementioned Kespa B-teamers is worth less points than, for example's sake, Happy beating 3 mid-tier foreigners who are in the 1200-1500 rating range?
And weighing matches is a much more cumbersome solution than if we were to have more tournaments, and more cross-region tournaments. >_<
|
Weighing by online and offline? Yes, this is a legitimate idea and probably the one closest to being implemented. We have working experimental code with this feature already Glad to hear that !
|
United Kingdom14103 Posts
|
|
|
|