EPTThe subjectivity KILLS me here, but alas, it still provides an interesting subject.
Introducing the SC2 World Ranking - Page 7
| Forum Index > SC2 General |
|
FabledIntegral
United States9232 Posts
The subjectivity KILLS me here, but alas, it still provides an interesting subject. | ||
|
archonOOid
1983 Posts
| ||
|
catplanetcatplanet
3834 Posts
| ||
|
Skytt
Scotland333 Posts
On August 08 2012 05:15 NOOBALOPSE wrote: GSL should be excluded if you are trying to factor in only skill or be worth less. It depends a lot on preparation. There is a reason TaeJa didn't win the last one... Thats like saying San is a better protoss than Seed is despite Seeds GSL Championship and his win today over Coca just because San beat Coca in a Bo3 in the OSL Prelims MVP absolutely dominated the MLG he won and is the most sucessful player in the GSL. MC, DRG and MVP have all been in the finals of the GSL this year and can dominate foreign 3 day tournaments on a good day. If the GSL isn't included or is worth less, MVP wouldn't be in the top ten and MC/DRG would be lower down than they deserve. With some more tweaks to the points I think this can be a very accurate depiction of who's on top of SC2. Especially with the volatility that ELO has in SC2. In BW they had the Kespa rankings, which while still subjective provided another view on the top players (Bisu was often ranked lower than his ELO because Kespa weighed Starleague performance higher than Proleague, where Bisu would just faceroll nerds every week). The same could be said for DRG back in 2011 before he got into GSL, where he was the team league monster but couldn't make it into the individual leagues. We don't have a foreign scene and it would be a good incentive for our community to create a ranking system that has more accuracy and longevity than ELO does and combines the Koreans and foreigners together, especially when we are hoping that SC2 becomes more of a global game than BW was | ||
|
Metalteeth
United States115 Posts
On August 08 2012 05:23 FabledIntegral wrote: Hmm... I'm not sure doing something like having 2499 vs 2500 points matters whatsoever in GSL second place. I understand what you were trying to do with making a first better than two seconds, but relatively when including other points, it can significantly alter things. For example, is first in dreamhack and fourth in MLG really better than 2nd in GSL? And if so, is it better by such a small margin? The subjectivity KILLS me here, but alas, it still provides an interesting subject. 2499 is exactly what GSL uses. The GSL points are exactly the points that are given by GSL. Nothing else is above 2000, winning MLG is 2000. Of course it is subjective, but I've adjusted the points based on feedback to get it in line to what the community think are important. But it's hard to make changes when people don't give any decent feedback, just saying "the points system sucks!", and that's it. Considering the amount of quality players at the foreign grand slams, I would think that winning a Dreamhack or MLG is pretty equal to a SF result at GSL. If winning a single Dreamhack makes ThorZain a top foreign player, then it is clear that Dreamhack/MLG/etc are pretty worth it. As a fun experiment, I'm going to use the rankings as they are to try and predict the WCS South Korea Winner's Bracket results. When two players match up, I'll take the player with the higher ranking. If a KeSPA player shows up (since they only have the Proleague, which hasn't been put in yet), I'll just take the non-KeSPA player. Sorry KeSPA player fans  + Show Spoiler + Remaining in R1 2 DRG > 74 Creator 11 HerO (Liquid) > KeSPA EffOrt KeSPA HerO > 39 Puzzle 78 YuGiHo > KeSPA Reality 261 dreamertt < 22 Genius 19 Squirtle > 111 Bboong KeSPA RorO < 137 Hack 29 Leenock < 1 MC Round 2 244 Miya < 65 Curious 25 PartinG > 26 aLive 13 Polt > 224 TAiLS 8 Seed > 44 Gumiho 2 DRG > 11 HerO 39 Puzzle > 78 YuGiHo 22 Genius < 19 Squirtle 137 Hack < 1 MC Round 3 65 Curious < 25 PartinG 13 Polt < 8 Seed 2 DRG > 39 Puzzle 19 Squirtle < 1 MC Round 4 25 PartinG < 8 Seed 2 DRG < 1 MC Obviously this doesn't take into account history between the 2 players (look at DRG vs HerO for example), or the player's specific ability in the specific matchups, it simply takes the player with a higher ranking according to my math to win. Just felt like making the prediction  | ||
|
Bastosai
France23 Posts
But we should keep only results on latest season counts no? Per year basis, or the latest championship to be taken into account (for exemple NASL4 in place of NASL3, ie erase all of NASL3 result and only counts NASL4). | ||
|
Skytt
Scotland333 Posts
However this would be very time consuming and would be something that you would want to automate. Especially when you have to deal with 200+ player open tournaments at MLG or the ro128 in Dreamhack. | ||
Darkhorse
United States23455 Posts
| ||
|
Shiori
3815 Posts
| ||
|
Metalteeth
United States115 Posts
On August 08 2012 06:28 Darkhoarse wrote: Just spotted one strange thing quickly and I'm not sure if there are other errors, but for example SeleCt got 2nd in an MLG. The scale says that 2nd in an MLG is worth 1200, but his total rating is only in the 700's. But definitely a cool concept. The points degrade over time. So a 1200 point second gets smaller over time. | ||
|
Markwerf
Netherlands3728 Posts
- no correcting for attendence - difficulty of assessing points for tournament These flaws are too big to ever consider such a system over elo or glicko whatever. The traditional problems of ELO like point sitting and opponent selecting are not even present so you should really just make a world version of that if you want a.proper rating system. Since the korean scene and foreigner scene actually mix up a bit more now then before it wouldn't even be too bad. Fine tuning compared to TLPD could definitely be done like giving bigger matches a higher K value or choosing a different underlying assumption for distribution of skill. Either way this system is a completely dead end but you seem to have a misguided hope that number tweaking can actually fix it, it won't. | ||
|
Cascade
Australia5405 Posts
 I would still prefer to keep 100% from a tournament until that tournament is played again, but this is also fine. Some comments on this vs ELO: 1) ELO tries to estimate the skill of players right now. This measures recent achievements. These two are quite different, and you cannot really say that one is better than the other, as they measure different things. Taeja is a good example. He tears up skilled people right and left in smaller tournaments, but has not really placed very high in many big individual tournaments. Thus ELO ranks him high, this ranking not as much. On the other hand, a player like MC really knows how to put himself together and win (or place high in) big events, but can drop games to lower ranked players in less important occasions. Thus MC gets a very high score for recent achievements, but maybe not as high in ELO. 2) Korea vs foreigners. The two communities are a bit separated, so it is hard to compare between the two. ELO solves this by not comparing at all. Thus two separate rankings. This point ranking tries to compare them, but runs into the problem of balancing the points of GSL vs other tournaments, and the problem that some players participate in more tournaments than others. Compare to tennis where all top players play in all big tournaments. None of the two solutions are very impressive. 3) The two systems have different flaws, so it is useful to look at both. ELO is very good because it has little subjective bias. A win is a win, no matter the context. Some bias in which games you include, but probably not a huge deal. This point system, as has been pointed out a lot of times, is very subjective in terms of how the points are distributed. The OP has done a good job of trying to find a distribution that most people can accept, but even with the full support of the community, you cannot get away from the fact that the points are distributed in a subjective manner. The "a win is a win" in ELO is also one of it's main problems. beating MC in the GSL finals is worth as much as beating him in the round of 8 in a weekly tournament. Many would argue that winning in the GSL finals is a much larger indicator of skill than beating them in a weekly tournament. The argument would be that people prepare for the GSL, and will always bring his very best game, while in a weekly, people may not play at their very best in every game, due to various circumstances. This is better taken into account in a point system, where big tournaments count more. So putting them together, ELO can be seen more as a "current raw skill" in some sense, while the point system is more "recent achievements". Also note that ELO tries to measure skill right now, while the point system is counting over the last 6 months, and will lag behind a bit. So a new player coming out of nowhere will jump up in ELO very fast, but will not reach his peak in the point system until after about half a year after his first win. Thus the point system measures consistency a bit as well. Best picture can be seen by looking at both. High ELO but low point ranking means either a rising top player, or a skilled player that is not very clutch in big tournaments. Low ELO but high point ranking is either a declining top player, or a good "tournament player". | ||
|
dgwow
Canada1024 Posts
So you need to separate between America, Europe, Asia etc | ||
|
FabledIntegral
United States9232 Posts
On August 08 2012 06:02 Metalteeth wrote: 2499 is exactly what GSL uses. The GSL points are exactly the points that are given by GSL. Nothing else is above 2000, winning MLG is 2000. Of course it is subjective, but I've adjusted the points based on feedback to get it in line to what the community think are important. But it's hard to make changes when people don't give any decent feedback, just saying "the points system sucks!", and that's it. Considering the amount of quality players at the foreign grand slams, I would think that winning a Dreamhack or MLG is pretty equal to a SF result at GSL. If winning a single Dreamhack makes ThorZain a top foreign player, then it is clear that Dreamhack/MLG/etc are pretty worth it. As a fun experiment, I'm going to use the rankings as they are to try and predict the WCS South Korea Winner's Bracket results. When two players match up, I'll take the player with the higher ranking. If a KeSPA player shows up (since they only have the Proleague, which hasn't been put in yet), I'll just take the non-KeSPA player. Sorry KeSPA player fans + Show Spoiler + Remaining in R1 2 DRG > 74 Creator 11 HerO (Liquid) > KeSPA EffOrt KeSPA HerO > 39 Puzzle 78 YuGiHo > KeSPA Reality 261 dreamertt < 22 Genius 19 Squirtle > 111 Bboong KeSPA RorO < 137 Hack 29 Leenock < 1 MC Round 2 244 Miya < 65 Curious 25 PartinG > 26 aLive 13 Polt > 224 TAiLS 8 Seed > 44 Gumiho 2 DRG > 11 HerO 39 Puzzle > 78 YuGiHo 22 Genius < 19 Squirtle 137 Hack < 1 MC Round 3 65 Curious < 25 PartinG 13 Polt < 8 Seed 2 DRG > 39 Puzzle 19 Squirtle < 1 MC Round 4 25 PartinG < 8 Seed 2 DRG < 1 MC Obviously this doesn't take into account history between the 2 players (look at DRG vs HerO for example), or the player's specific ability in the specific matchups, it simply takes the player with a higher ranking according to my math to win. Just felt like making the prediction What the GSL uses itself is internally. I already made it very clear that it works with an internal system, where you can weight a single first place finish more than two second place finishes. I said where it fails to work is when you extrapolate it to be used with other point systems as well. That's the issue, where you're using whole number typically divisible by 100 (or at least 10), the second place becomes exponentially less valuable due to missing a *single* point. For example, if the second place was worth 2501 as opposed to 2499, it would have a huge impact on the rankings for those who got second place one would think, when in reality 2 points should be of absolute negligible value to anyone in the top 50 or so. | ||
|
Skytt
Scotland333 Posts
On August 08 2012 09:12 dgwow wrote: This kind of tournament system doesn't work because of the disparity between korean and foreigner skill, because koreans usually dominate foreign tournaments but many of them don't get to go to them, and likewise many foreigners don't get to go to korean tournaments. So you need to separate between America, Europe, Asia etc Building further upon the idea of weighting tournaments by the average elo of the players within it, you can create a factor that affects the ELO between the two and this is only needed between Korea and the foreigner ELO tables, you can work this out pretty easily too, MCs current foreign ELO is around 10% higher than his korean ELO. If you multiply naniwas korean ELO by 10% he gets placed just below Stephano and Nerchio to be the 3rd best foreigner, which I would consider pretty accurate. If you apply it to Taeja his foreign ELO is over 2.5k so this method would have to be refined, by maybe taking the average factor of which Korean ELO translates to Worldwide ELO. You can then work out from that how the average ELO of a nation/region is which can be used to further weigh the value of regionals like SA and SEA qualis where the player pool is of a lower average skill when compared to NA/EU/KR. | ||
|
IcedBacon
Canada906 Posts
| ||
|
Metalteeth
United States115 Posts
As for the idea of using ELO to rate tournaments, that is certainly a good idea. I would have to think about it and how to make ti work before implementing it (and reweighing and rewriting all the data I've created over 2 days -_-). If I do do that, I'm thinking that incorporating both the mean and the median would help to deal with very low or very high ELOs. | ||
|
minilance
Canada500 Posts
On August 08 2012 06:23 Skytt wrote: One possible way to improve weighting of tournaments would be to influence the weighting by the average ELO across the competitors. In theory this could allow tournaments with a stacked player pool to become worth more than an 'easy tournament' where the player pool is of a lower quality. This would also help deal with when there are multiple tournaments on in one day and a powerhouse player goes to the tournment with the weaker player pool and sweeps it for the easy money. Extending that could be that you ignore the ELO of the winner, so that if a player like Stephano with his 2.4k ELO wins a tournament where the average ELO is only 1970, the Average wouldn't be inflated artificially by his huge ELO However this would be very time consuming and would be something that you would want to automate. Especially when you have to deal with 200+ player open tournaments at MLG or the ro128 in Dreamhack. i think this would be thé best method | ||
|
Dubsy
Canada186 Posts
Kind of a waste of time as is. You have the infrastructure, you should really go back and try something else. Maybe get someone with some statistics background to give you a hand because you've done most of the legwork already. | ||
|
Exia0276
Hong Kong62 Posts
If you are aiming for a good skill based ranking, you can debate the particulars but you can't avoid the general ideas behind ELO. I guess the disadvantage of ELO is the calculation might seem mysterious to most people so one may wonder if they did their math right. Whereas a weighted participation-based system is easier for everyone to understand and see what's going on. | ||
| ||
