I like to think of Starcraft 2 as similar to another 1v1 sport, Tennis. Both have big tournaments that run a weekend most often (although tennis majors go longer). One thing I feel that really helps Tennis is a unified ranking system over all tournaments, the ATP. Every tournament is ranked, so a single list is a quick way of ranking all the players. So I decided to make one!
I love stats, and it was fun to make. Plus, you see every tournament seeding players, but by only that one tournaments past results. Why should someone be punished for not playing an MLG, even if they are the best in the world?
Now note, I didn't include ALL events. I used Liquipedia, both the Premier and Major events list. Every tournament that has finished since today (August 7th) last year. So one year of results. I skipped over Team Events because I don't know how to incorporate that into an individual list. I also skipped qualifiers (minus WCS National events), because I felt that it overlay reward qualified players. Some people may disagree with my decision to skip Code A, and I can understand that, but since Code S players go through Code A every season, I didn't want to think about how to work that out.
This is a regression system. Events degrade each day according to the equation 0.9999792^(#ofDays^2), where #ofDays is the number of days since the events FINALS occured.
I'll go through the player list later and clean it up, introduce race and tlpd links, etc.
If you want the Excel Sheet I was using to calculate the point totals, see here: + Show Spoiler +
If you want to look through and make sure that everything is good and correct, I would appreciate that!
Tournament Weightings (Warning: Wall of numbers): + Show Spoiler +
EDIT: ATTENTION. These are the point values for tournaments BEFORE the weekend of August 10th, 2012. All tournaments beyond that weekend will use the formula: [(Mean points of players in event)+(Median points of players in event)+(Prize for winner in US Dollars)]/10
Events degrade each day according to the equation 0.9999792^(#ofDays^2), where #ofDays is the number of days since the events FINALS occured.
So essentially, I wanted GSL to be the top event, since it is most prestigious of all events. Then the top events (MLG, Dreamhack, NASL, etc). Events listed as "Major" on Liquipedia are significantly weaker in point values.
interesting post! i disagree with some of the weightings (e.g. IPL - are they all worth 2k? they different greatly in terms of skill/difficult). IEMs are weighted far too strongly imo but otherwise nice :D
On August 07 2012 14:13 opterown wrote: interesting post! i disagree with some of the weightings (e.g. IPL - are they all worth 2k? they different greatly in terms of skill/difficult). IEMs are weighted far too strongly imo but otherwise nice :D
Well, I think he started from LA Battenet invitational, which was around late summer 2011. So this should mean that only IPL 3 and 4 were included (not 100% sure).
On August 07 2012 14:13 opterown wrote: interesting post! i disagree with some of the weightings (e.g. IPL - are they all worth 2k? they different greatly in terms of skill/difficult). IEMs are weighted far too strongly imo but otherwise nice :D
Thanks! The weightings are still open to change if people think they are skewed oddly. I do think that IPL3 and 4 are worth 2000, since I feel that an individual IPL is equal to a MLG Season Championship. IEM is still pretty important, but it seemed kinda odd to weight a "qualifier" event equal to the final world championship. Perhaps make an IEM stop 1500 (equal to an MLG arena) and the IEM WC 2000?
Honestly, it's far too subjective for me imo, since it's really just your opinion. I think I'll stick with TLPD, but who knows, maybe your way will catch on!
On August 07 2012 14:17 FabledIntegral wrote: Honestly, it's far too subjective for me imo, since it's really just your opinion. I think I'll stick with TLPD, but who knows, maybe your way will catch on!
Well, the only thing that is my opinion is how each tournament is weighed. Like I said, if people think that I have a tournament weighed strangely, I can easily go change it to what the community thinks is better. TLPD is good too, but since it is still separated between Korean and International, it's hard to get a sense of the two scenes together.
GSL should definitely be worth more than it is IMO. Winning a GSL is a MUCH bigger accomplishment than something like an MLG or IPL which are all relatively close in points.
You can just make a point system like that, it has to also deduct points from the loses you have. If you want a proper list, like as you said in Tennis, individual matches has to count, points added, points deducted etc.
I love the idea, and always thought someone should do it, but just smacking some points in them will not give a proper result. If you're willing to completely rework this, it would be great imo, but needs a lot of work.
On August 07 2012 14:21 Jaeger wrote: Interesting if arbitrary. Since there is no way to lose points seems like it heavily favors those who travel a lot.
It's actually weighted in a way that it doesn't require travelling. MVP is in the top 5, despite almost never leaving Korea. Winning a single event is better than getting mediocre results at 10 MLG events, so it is quality over quantity. The point scaling is based heavily on the scale for Tennis, which rewards winning or consistency. There isn't really a way to do it where those that go to more tournaments aren't rewarded for it.
As for losing points...I don't really know of a way to do incorporate that in a balanced maner.
This seems like a great way to complement ELO ratings.
As the sc2 scene is moving a lot faster than the tennis scene, I think having flat weights over a full year is a bit too long. We can see a few players quite high up that haven't done anything noteworthy in a long (by sc2 standards) time. You could make the points decay a bit faster as you suggest, but it will make it less transparent. You will have situation where players will slide away from their first place on a random day between events when their points decayed to much.
Then maybe better to remove points from a tournament when the same tournament runs again. So you lose your GSL points at the next GSL, etc. It makes much more sense in term of defending a championship, and it will make sure positions stay the same between championships. It will still coincide with tennis, as all tennis championships are run annually (right?).
just my personal rankings; i realise some are different, but mainly due to player pools etc. haven't looked at it very carefully yet, pretty quick random lookover
Is there any sort of built in decay for the world Tennis rankings like they have added for GSL this year? (I don't remember the exact percent but I believe it's 10% decay each season) It just seems a little weird that someone like Huk would still be in the top 10 at this time but I guess it could just feel like an overexaggeration of his recent dropoff. Do the tennis rankings reset after a period of time or does it just continually add until there is some change?
EDIT: I like the idea. It's always interesting to see other people's takes on power rankings and all the different ranking methods. I just had some questions about how it worked since I don't really follow tennis outside of the major events.
On August 07 2012 14:24 Cascade wrote: This seems like a great way to complement ELO ratings.
As the sc2 scene is moving a lot faster than the tennis scene, I think having flat weights over a full year is a bit too long. We can see a few players quite high up that haven't done anything noteworthy in a long (by sc2 standards) time. You could make the points decay a bit faster as you suggest, but it will make it less transparent. You will have situation where players will slide away from their first place on a random day between events when their points decayed to much.
Then maybe better to remove points from a tournament when the same tournament runs again. So you lose your GSL points at the next GSL, etc. It makes much more sense in term of defending a championship, and it will make sure positions stay the same between championships. It will still coincide with tennis, as all tennis championships are run annually (right?).
I'm starting to think regression would work, but I don't think removing points when a tournament runs again would work. It becomes too dependent on when the tournament runs, or even if it runs. A time deduction makes the most sense, but like you said, it makes things a bit random. But it works better than anything else, over all events. Maybe do something like past 3 months (still long, but not super long in SC2 terms) is 100%, 4-6 months is 75%, 7-9 months is 50%, and 10-12 months is 25%. It regresses, but I feel like it's too simple in a way...
If you want something like ATP its not just weights, there is season it starts in beginning of each year, you start with same number of points but you need to defend titles, depending on that is how much you win/lose, if this system was any good MMA and for example HuK wouldnt be even close to top 10.
EDIT: Then dont do tournament depended but for example every quater of year, you recheck tournaments and update ladders ( if you get what i mean ), maybe this year there are more premium tournaments in one quater, so numbers will change more rapidly.
This is a cool idea, and I love anything that gives me a ton of numbers to sort through, but I think you definitely need to make sure you include a regression system. Otherwise this will really lose meaning over time...
For balancing the points for each tournament, maybe easier to do like in tennis, and group them up? Just as example:
1. Grand slams: GSL, world championship(?) 10000 points pool
2. majors: MLG, IEM, dreamhack, NASL, ... 4000 point pool
3. minor: MLG arena, ... 2000 point pool
4. regional: ESL weekly, ... 500 point pool
Maybe that will be a bit less confusing than having each event have a unique amount of points. So whenever you watch a NASL, MLG or whatever, you know that you are watching a major, and all have the same value.
I think this is a good start, but the system could be a lot more refined. I would love to see you incorporate extra points for defeating other highly ranked players like they do in the ATP, but I imagine that it would be pretty hard to do. I also think you could come up with a better way to assign point values to certain tournaments, rather than choosing an arbitrary number based on your opinion. For example, you could use the size of a tournament, whether it's "open" or not, and the overall prize pool as variables for determining the total "points" awarded for the tournament. The WSOP (professional poker) does something similar to this.
In regards to losing points, I don't think this is necessary. If you attend more tournaments, you have the potential to gain more points...but that's how it is with almost any other sport. Tennis, golf, poker...If you're trying to determine an individual world #1, this is a fair way to do it. No need to deduct points for losses. Why should someone get punished for travelling to an MLG and losing in the 1st round vs. someone who just sat at home and did nothing? Doesn't seem very fair...
On August 07 2012 14:31 SRBNikola wrote: If you want something like ATP its not just weights, there is season it starts in beginning of each year, you start with same number of points but you need to defend titles, depending on that is how much you win/lose, if this system was any good MMA and for example HuK wouldnt be even close to top 10.
EDIT: Then dont do tournament depended but for example every quater of year, you recheck tournaments and update ladders ( if you get what i mean ), maybe this year there are more premium tournaments in one quater, so numbers will change more rapidly.
Thing is, ATP has no regression as far as I can tell. It is simply the past 52 weeks of Tennis, all added up. Granted, they have required tournaments, and if you don't compete (without an injury) you are deducted points.
On August 07 2012 14:36 mordk wrote: Aren't open seasons counted or something? NesTea has 3 GSLs, that alone should give 9000 pts right?
From the first post: This initial ranking goes from the 2011 Latin American Battle.net Invitational, through to WCS Mexico and ASUS ROG. So here you go, the very first SC2WR Top 10.
NesTea hasn't actually.....done much in tournaments in the past year. His success is over a year ago, so it didn't count.
On August 07 2012 14:33 Cascade wrote: For balancing the points for each tournament, maybe easier to do like in tennis, and group them up? Just as example:
1. Grand slams: GSL, world championship(?) 10000 points pool
2. majors: MLG, IEM, dreamhack, NASL, ... 4000 point pool
3. minor: MLG arena, ... 2000 point pool
4. regional: ESL weekly, ... 500 point pool
Maybe that will be a bit less confusing than having each event have a unique amount of points. So whenever you watch a NASL, MLG or whatever, you know that you are watching a major, and all have the same value.
The point totals are different, but that is pretty much exactly what I did. GSL at 3000, the other grand slam events at 2000, WCS Continentals at 1000, WCS Nationals/most larger one off tournaments at 500, and small one off tournaments at 250.
However, it sounds like a revised ranking might go something like: GSL: 5000 MLG, Dreamhack, IPL: 3000 ASUS, HSC, IEM WC, NASL - 2500 IEM Tour Stops, MLG Arena - 2000 WCS Continentals - 1000 Large Non-GrandSlam Tournaments (see Red Bull, Arena of Legends, etc) - 750 WCS Nationals - 500 Small Tournaments - 250
Weeklies would have to be seriously nerfed down to low amounts to not allow a mass buildup of winning easier weeklies to equal a large event.
Winning a GSL should be worth as much as every other tournament combined. Reaching the quarters is about as impressive as winning any other tournament.
On August 07 2012 14:33 Cascade wrote: For balancing the points for each tournament, maybe easier to do like in tennis, and group them up? Just as example:
1. Grand slams: GSL, world championship(?) 10000 points pool
2. majors: MLG, IEM, dreamhack, NASL, ... 4000 point pool
3. minor: MLG arena, ... 2000 point pool
4. regional: ESL weekly, ... 500 point pool
Maybe that will be a bit less confusing than having each event have a unique amount of points. So whenever you watch a NASL, MLG or whatever, you know that you are watching a major, and all have the same value.
I like this idea. OSL should be grand slam as well :D
On August 07 2012 14:43 Entirety wrote: Create a formula for exponential decay so that it reaches 50% in one year.
Here, I'll even do it for you:
Points = (Original Point Value) * 0.99810276865159^(# of days that have passed)
I agree with this post that some sort of decay should be added in. Isn't that what everyone was complaining about with the MLG rankings from last year? These rankings go back a full year and it looks like if you won 2 or 3 tournaments early on a year ago you are still near or in the top 10 even though you may not be as relevant anymore
On August 07 2012 14:43 Entirety wrote: Create a formula for exponential decay so that it reaches 50% in one year.
Here, I'll even do it for you:
Points = (Original Point Value) * 0.99810276865159^(# of days that have passed)
For example, NesTea's Open Season GSL victory was 632 days ago. (http://www.convertunits.com/dates/from)
Plug it into the formula... His original victory (3,000 points) is now worth 903.4 points. I'm sure someone can come up with a more elegant regression formula
On August 07 2012 14:43 Entirety wrote: Create a formula for exponential decay so that it reaches 50% in one year.
Here, I'll even do it for you:
Points = (Original Point Value) * 0.99810276865159^(# of days that have passed)
Maybe better with a gaussian? Exponentials start to fall of at a quite steep rate from day one, which doesn't feel really fair. Let the player keep the full points for at least while before you start taking it back. A gaussian starts flat but falls of faster after a long time.
And I'd make it reach 50% well before 12 months. is 2 NASL wins 10 and 14 months ago really worth as much as a NASL win today? Maybe 50% after 6 months is a better number. Which means only 6% left after a year, but 84% left after 90 days.
That would be points = (original point value) * 0.9999792^((#days)^2) or equivalently (original point value) * e^((#days/219)^2)
Gotta include ALL the results to have an accurate measurement. Team leagues, smaller events ... everything. and you must have some creative liscence as well. To not have Taja on the list after the month he's had is ridiculous.
Older events must lose their weighting over time. An event eighteen months ago cannot have as much merrit as an event last week.
You have a good foundation, but it needs improvements to be legit.
And no Nestea???
And there's no need to give points for anything outside third place for an event with the exception of GSL. Nobody cares who came 10th at MLG.
The rankings correlate almost exactly with the people with the most money to travel to the most events having the most points.
Those who compete more, have more points. Not saying DRG or MC don't deserve to be so high up, they definitely do, but this list just has an objective flaw in it in that it rewards participation more than skill. For example Huk is at 9 above players like Taeja, Nestea, Leenock, Genius, aLive, and Symbol.
There's no objective reason for this ranking aside from the fact that he's had more opportunity to compete at foreigner events where the competition was less than the GSL and he was able to place high enough to accumulate points for this ranking system while players in Code A couldn't.
On August 07 2012 15:14 Vindicare605 wrote: I don't like this ranking system at all.
The rankings correlate almost exactly with the people with the most money to travel to the most events having the most points.
Those who compete more, have more points. Not saying DRG or MC don't deserve to be so high up, they definitely do, but this list just has an objective flaw in it in that it rewards participation more than skill. For example Huk is at 9 above players like Taeja, Nestea, Leenock, Genius, aLive, and Symbol.
There's no objective reason for this ranking aside from the fact that he's had more opportunity to compete at foreigner events where the competition was less than the GSL and he was able to place high enough to accumulate points for this ranking system while players in Code A couldn't.
That's my critique.
Yes, this is definitely a big problem with this approach, that doesn't really exist in tennis where all the players have money to travel as much as they want. ELO is better in that way, but the problem with ELO is that it rank a win in a GSL finals as high as a win in a ESV weekly round of 16 (see for example uproar about taeja being ELO rank 1 when liquid got him). While we find a better ranking I think it could be useful to have both this and ELO. As their drawbacks are quite different, I think the two complete each other well. Possibly you could argue that this approach is very similar to the price money ranking, and a bit redundant for that reason.
I love the idea and the iniciative! Thanks for sharing your effort. Its not flawless, but its a great start! Just a few minor improvements and should be even better! Its hard to rate GSL, because YES it weights more than the rest of the tournaments, but it puts all the Koreans so much ahead of the rest, just for being in GSL for a few seasons. So its hard to find the right balance. I think thats why the TLPD is separated, because the Korean scene is just different. To put it in another way, you think players like Supernova and Keen and Curious and Genius are great players! And they are! But in a tournament outside Korea, you wouldnt consider them favorites..? Or even in past experiences, they dont even get to the finals, or even semifinals! Or drop way down the road.. So that means there are foreigners who really can put a fight and beat them, but just because they are not in GSL they arent considered top players. I hope I made my point clear. Longstory short! Loved it(: but its gonna be hard to find the right balance!
I know some ranking systems will take the 10 best results over a certain period. This way players that participate in many tournaments will have many opportunities to raise their ranking, but they'll only do so by going deeper than they did in their 10th best result.
another alternative to ELO is http://www.sc2charts.net/en/edb/ranking/players the system they use is a lot slower to move than ELO (i think ELO moves a bit too fast and sc2charts a bit too slow o.O)
Tournament Weightings should be based on the overall level of competition. Good way to do this is to look at Ro32 player list of any given tournament. It tells you how "hard" it would be to win that tournament.
And if you are going to skip Code A, then you'd also need to do the same for MLG arena, non-WC IEM, WCS nationals, etc.
IMO, x2 GSL points, add points for Code A, x1/2 for everything else but MLG.
The point you give each league are completely random, If you were able to get the stream number from each event ( just avg stream numbers ) and than award every single event points based on stream number ( maybe prize pool as well ) than that would be something. But unless you want to do that please retain from starting random bull shit storms like this just to get people arguing over the fact that they would have awarded those random numbers in a different way.
Ok, so after a night sleep, some thought, and reading all the replies, I have some things to add here.
1) If you are going to criticize this project, please read all the material in the OP. Asking "Why isn't Jinro in the top 10?", when he has barely done anything in a year, is a waste of time.
2) It seems like a lot of people want a regression, I'll work on getting that in. Cascade, with the Gaussian he gave, thank you. I'll try adding that in! :D
3) To people complaining about the tournament ratings, some of it is valid, some of it is useless: "I don't like the point values!" Well that is nice, what would you change? It seems most people think GSL should be worth more. But as someone said in here, if it gets too high, the system will overvalue mid-level Koreans who do well in GSL and nothing else. But too low, and GSL success isn't worth the difficulty that is has. What I'm thinking is bump GSL up to 4000 points for winning it. Keep MLG/IPL/Dreamhack/ASUS/IEM WC/etc (the non-GSL Grand Slams) at 2000. Drop IEM tour stops to 1500. Keep everything else the same, for now.
4) Team Leagues. The problem with team leagues is the concept of regression to the mean. In Individual tournaments, over a year, enough random draws will happen such that a players "true" skill should be determined. But team leagues are a lot less random, due to team lineups. If we reward bonus points for success in Team Leagues, how should we do it? Points for All Kills? That hurts players on good teams. For example, MC will probably never get an All Kill in GSTL, because the team he is on is so good, and MC isn't the lead. Points for winning an overall match? But that rewards bad players. Should the lower members of Liquid get bonus points for simply being on the same team as Taeja? Points for winning single games within a team match? This works best, but honestly, the amount of work I would have to put in, tabulating EVERY match from EVERY team league....that is insane.
5) "You have to include every tournament." Thing is, that seems virtually impossibly. The place where tournaments are the most congregated with easy to access results is Liquipedia, so that is what I'm using. And since it DOES include every Grand Slam event (which are worth much more than a minor event), it was just so much easier to use it instead of searching out there for every tournament. As for weekly/daily cups, the problem is point inflation. Points have to be big enough that winning a weekly cup is worth something, but not enough that winning a string of cups is equal to a major tournament.
6) "You reward players who travel." Yes....that is how competitive SC2 works....There is a reason players like MC, MVP, and DRG are at the top. They don't travel to every event (well, except MC), but they get top finishes pretty much every time. The system doesn't reward going to many events, just look at how fast the points drop the further down you finish. You have to finish well at every event to get points that matter.
Things to do today: Revise the points structure (see above in this post, revised a bit, GSL is better). Add in point regression. Hopefully I should have a revised standings tonight.
GSTL is a massive indicator of who is good. This is where we saw MMA and DRG on the rise long before their code S and blizzard cup victories. I don't think you can have an accurate representation without some influence of the GSTL.
I know its tough to add, but it's too big of an indicator of skill to overlook.
On August 07 2012 23:05 Metalteeth wrote: 3) To people complaining about the tournament ratings, some of it is valid, some of it is useless: "I don't like the point values!" Well that is nice, what would you change? It seems most people think GSL should be worth more. But as someone said in here, if it gets too high, the system will overvalue mid-level Koreans who do well in GSL and nothing else. But too low, and GSL success isn't worth the difficulty that is has. What I'm thinking is bump GSL up to 4000 points for winning it. Keep MLG/IPL/Dreamhack/ASUS/IEM WC/etc (the non-GSL Grand Slams) at 2000. Drop IEM tour stops to 1500. Increase last year's Dreamhack 8 person invites to 1500. Keep everything else the same, for now.
"Overvaluing" mid-level Koreans isn't possible - if you're in Code S, you're essentially guaranteed to be better than all but one of the players not in the GSL. It's these very same "mid-level Koreans" that dominate foreign events when the top ones don't go. Violet and Puma in the top 10 rather prove that.
On August 07 2012 23:46 NightOfTheDead wrote: GSL should bear double if not triple weight, no offense but Killer(Chille) was never better than Squirtle in a year.
That was before regression. Killer (Chile) had last years Latin American BattleNet Invite, some IEM success, then he won WCS Chile and WCS South America. Those last two were 1500 points.
gsl needs like 1.5x the points lol, squirtle and parting aint even in the top 30, and creator isnt even in top 100 lol. also there is absolutely no way in hell does stephano seem better than MKP, hero, taeja etc. Flaws are kinda apparent with huk and violet in the top 10 rofl. Good ranking system tho, this has the potential to be better alot tho.
OK, made some changed. Added a regression (see a previous comment for the formula, go Gaussian statistics!), and changed around some of the tournament values. GSL got a boost, IEM was dropped a little. Please see OP for full everything, but Here is a revised Top 10.
I think you'd almost have to separate the korean standings from the foreign standings. There are so few koreans who can afford to travel to a bunch of tournaments, and the reverse is also true: there's no more than a handful of foreigners who are able to go to korea and compete in gsl/gstl and such. You'd have to come up with some points system that includes gstl and korean weeklies and such, and then you could use the rankings you have now minus gsl as the foreigner ranking.
I just feel like the ranking loses any meaning past the top handful of players that can play in GSL and foreign tournaments. Mana is ranked higher than Nestea, even though Mana hasn't competed in gsl and nestea rarely leaves korea. The purpose is to determine who is playing objectively better at that moment, but these come down to your subjective relative weighting.
That said, I think a ranking system like this is awesome to have. It'd be really cool to have a #1 ranking that players strive for that actually means something, and tournaments could start using it for a more fair seeding system. It would be cool to see who's really the top foreigner according to the stats, and there could be interesting storylines like Roger Federer at Wimbledon a few weeks ago reclaiming his world #1 ranking. Imagine Mvp winning another GSL to get back on top after a year of struggle with his wrists. Imagine the excitement when some no-name upsets the #1 ranked player in straight sets (a la Bisu over sAviOr).
I don't think the system could ever work because so many players don't even try so many leagues. It's not fair to a ton of non-Korean players that the GSL has any weight at all because it's not a viable tournament to play without taking up residence near its location. The GSL isn't hard just because it has good players. It's hard because you have to move to a foreign country that speaks virtually no English just to participate. And then you've got to pay more and travel longer to continue to participate in NA/EU tournaments.
I understand rankings in other sports work similarly in the sense that a player can forgo practice or rest to participate in more competitions in an attempt to bolster ranking. But the situation for SC2 is way more extreme and I believe impossible to solve.
On August 08 2012 00:17 Metalteeth wrote: Ok, made some changed. Added a regression (see a previous comment for the formula, go Gaussian statistics!), and changed around some of the tournament values. GSL got a boost, IEM was dropped a little. Please see OP for full everything, but here is a revised Top 10.
There has to be a more precise/mathematical/logical basis to the way tournaments are given points.
The different tournament formats should also make a difference in the way points are allocated. For example, knockouts are usually thought to be more difficult than group stages, while the presence of a loser's bracket allows second chances (leniency). Going undefeated in a 64-man seeded knockout tournament is harder than a 2-groupstage then knockout tournament, which I think is again more difficult than a tournament with loser's brackets. Of course this is again opinion; there must be a way to determine how these tournament formats make a difference to difficulty.
Anyway, as for a better way to determine points...
These things should matter: Difficulty of player pool (while this is subjective, giving a difficulty range should be enough to mitigate most opinion) Size of player pool Size of prize pool Estimated viewership numbers (again in a range rather than specific) Subjective Prestige (hard not to consider this) Invite only or Qualifiers (invite only should matter less, imo) Regional Difficulty Quotient (sorry, not all regions are equal)
There are probably more. I am typing this in the middle of a hurricane so I am too bothered to try to make it more detailed T_T
What's weird about this is that different people have different "styles" of tournaments. Players like MVP and NesTea are amazing at GSL style (knowing your opponent much in advance) so they can prepare for them. But then if you look at their success in team leagues, they crumble. They aren't nearly as good playing without tons of practice vs that one person. If there was some way to represent that, then this might work.
As many people have noted, the subjectivity in the assignment of tournament weights is the problem. As far as I can see, the system chess uses to rate tournaments would be the most naturally applicable. That is, calculate the average ELO for an event, and then assign a "category" value to that tournament. For example, the very strongest tournaments are category 21 events, where the average ELO ranges from 2751-2775.
This makes much more sense than the blanket assignment of weights to a tournament, especially when one considers that the same tournament could differ greatly in the strength of its participants over several seasons.
A salient point here is that, if one trusts the ELO enough to assign the tournament value based off of it, then why not simply use the ELO in the first place to determine rankings? I would argue that we should stick to the ELO system, although I believe we should amalgamate the "international" and "korea" list to have one, unambiguous reference for who is the current best player.
As a last remark, it could be argued that one should have different ELO "types" based on the type of tournament held. For instance, chess has different rankings for different time controls. The classical time control ELO differs from the rapid and blitz lists - although the intersection is clearly non-zero. The SC2 equivalent of classical time controls might be something like the GSL, where one has no less than one week to prepare for each match. The equivalent of rapid time controls could be something like an MLG, where one has hours (or indeed even minutes) to prepare.
tl;dr - The Chess system is a better model for SC2 rankings than the Tennis model.
On August 08 2012 00:26 Liquid`NonY wrote: I don't think the system could ever work because so many players don't even try so many leagues. It's not fair to a ton of non-Korean players that the GSL has any weight at all because it's not a viable tournament to play without taking up residence near its location. The GSL isn't hard just because it has good players. It's hard because you have to move to a foreign country that speaks virtually no English just to participate. And then you've got to pay more and travel longer to continue to participate in NA/EU tournaments.
I understand rankings in other sports work similarly in the sense that a player can forgo practice or rest to participate in more competitions in an attempt to bolster ranking. But the situation for SC2 is way more extreme and I believe impossible to solve.
Ofc Korea has some unique features, but is not that different from amateurs that cannot afford to travel to X country just because they are amateur so they have not much money to go.
But yes, the gap is too large even for accomplished players to go and play GSL even if they can afford it. It's very different from tennis.
On August 08 2012 00:24 Kovaz wrote: That said, I think a ranking system like this is awesome to have. It'd be really cool to have a #1 ranking that players strive for that actually means something, and tournaments could start using it for a more fair seeding system. It would be cool to see who's really the top foreigner according to the stats, and there could be interesting storylines like Roger Federer at Wimbledon a few weeks ago reclaiming his world #1 ranking. Imagine Mvp winning another GSL to get back on top after a year of struggle with his wrists. Imagine the excitement when some no-name upsets the #1 ranked player in straight sets (a la Bisu over sAviOr).
That right there is exactly why I did it. Having a unified ranking system helps to create a narrative (something Day9 absolutely loves!) over multiple tournaments. I am always frustrated that GSL ranks by GSL points, MLG ranks by MLG points, NASL ranks by NASL results. Having a unified ranking (compared to TLPD, which, while very helpful, has some issues I feel) is a great thing for the scene.
Ridicolous. The problem is obviously that, compared to korea, everybody sucks at SCII. If it was possible only matches involving koreans should count. Its nice if Goody beats Cloud but it still does not mean shit.
...and this is why the ranking completely blows. End of story.
Nerchio's score is heavily boosted by his recent win at HSC5, and two QFs at Dreamhack. Naniwa and PuMa have not had top victories in the recent events. And their results in GSL, while good, are not enough to make up a Grand Slam win from Nerchio.
On August 08 2012 00:42 TiTanIum_ wrote: It looked fine, but Potiguar being better than Bomber, seems silly.
Bomber has been not doing too well since Red Bull Battlegrounds. As for Potiguar, a problem with the system is that WCS South America has already ran (along with some VERY easy WCS Nationals) boosting the South American players that did well. That is why Killer (Chile), Fenix, and Potiguar are all in the Top 100. As Europe, NA, and Korea have their WCS events, I expect the South American players to start dropping.
On August 07 2012 14:17 FabledIntegral wrote: Honestly, it's far too subjective for me imo, since it's really just your opinion. I think I'll stick with TLPD, but who knows, maybe your way will catch on!
Well, the only thing that is my opinion is how each tournament is weighed. Like I said, if people think that I have a tournament weighed strangely, I can easily go change it to what the community thinks is better. TLPD is good too, but since it is still separated between Korean and International, it's hard to get a sense of the two scenes together.
TLPD avoids to compare the Korean and International community. Yours does but in a completely unfair way since most foreigners can't compete in GSL and few koreans travel to many tournaments. It's better not to compare than to compare in a terrible way. This method you devised is even unfair for comparing foreigners amongst eachother or koreans amongst eachother because there is no punishment or correction for difference in tournament attendence between players.
ELO based systems are simply superior for trying to be get an objective as possible ranking for a game like starcraft. A point system like the ATP works in tennis because there are rules that pretty much force equal attendence, the top players are forced to attend all grand slams, 8 ATP 1000 tournaments and only a limited number of points from ATP 500 tournaments are counted. That is simply not the case in many other sports including starcraft. Starcraft unlike tennis also has many different tournament formats including round robin systems, double elimination etc which make it a bit unelegant to use a point system. For example an ELO system would differ results between dominating a MLG by 5-0'ing your group and smashing all through the winners bracket compared to getting last in group and still winning from the 'losers' bracket, a point system like this would not.
Point systems are elegant in their simplicity which make it easy to make statements like: "if MC beats DRG here and goes on to win the GSL he is the new #1!!" but the conditions for it to work properly are just not in place for starcraft. You would have to contrive difficult attendence penalties etc. to make it work. The system is already a bit ugly now by implementing a regression based decay for the points which is imo unneccesary difficult since you would have to update your rankings each day then if you plan to keep this up. Just using the tennis system where results count equally till they are removed after a year is far easier as you only need to update after tournaments for the players involved that year or the previous. If neccesary you can make a two stage drop, ie points are halfed after half a year or something but regression based is just not elegant. Remember that elegancy is the primary reason to choose a point system over an ELO based system in the first place..
Finally if you implement this you would be best to do a sensitivity analysis. Rank your tournaments in order of importance which is much easier to get an agreement about, for example (1 GSL 2 MLG, IPL, WCS, Dreamhack 3 others) or just by prize money and than use some software to calculate your top 10 or so with different weights. You can see then how much the weighting really matters for the relative ranking of your top players.
edit: by the way Wimbledon is also entitled to change seedings based on grass results only. So in a way they are not only seeding by overall ranking but also by a different ranking similar to how tournaments like GSL just use their own results.
On August 08 2012 00:17 Metalteeth wrote: Ok, made some changed. Added a regression (see a previous comment for the formula, go Gaussian statistics!), and changed around some of the tournament values. GSL got a boost, IEM was dropped a little. Please see OP for full everything, but here is a revised Top 10.
On August 08 2012 01:38 Markwerf wrote: TLPD avoids to compare the Korean and International community. Yours does but in a completely unfair way since most foreigners can't compete in GSL and few koreans travel to many tournaments. It's better not to compare than to compare in a terrible way. This method you devised is even unfair for comparing foreigners amongst eachother or koreans amongst eachother because there is no punishment or correction for difference in tournament attendence between players.
I've seen plenty of people complain that they want TLPD to be combined, so I thought about try to make something that is combined. Is it perfect? No. But I made it all in 12 hours, and already changed stuff around. As for being "compeltely unfair", I disagree. Of course a player who goes to less events is going to be ranked lower, that is obvious for pretty much any result. Besides, I would say MVP pretty much never goes to foreign events (and when he does, he tends to not do well). And yet he is top 5, because he is good in Korea. The rankings don't require travelling to every single event, and success in GSL is enough to make top 50.
On August 08 2012 01:38 Markwerf wrote:
ELO based systems are simply superior for trying to be get an objective as possible ranking for a game like starcraft. A point system like the ATP works in tennis because there are rules that pretty much force equal attendence, the top players are forced to attend all grand slams, 8 ATP 1000 tournaments and only a limited number of points from ATP 500 tournaments are counted. That is simply not the case in many other sports including starcraft. Starcraft unlike tennis also has many different tournament formats including round robin systems, double elimination etc which make it a bit unelegant to use a point system. For example an ELO system would differ results between dominating a MLG by 5-0'ing your group and smashing all through the winners bracket compared to getting last in group and still winning from the 'losers' bracket, a point system like this would not.
The thing is, tournaments do not differentiate between running through the losers and crushing your group. Her0 at the last Dreamhack lost 2 games the entire way, but only gets a QF result because of it. While something like Leenock's run in MLG Providence, gave him a victory. A single tournament could give some results, but I'm thinking that the randomness of a year's worth of tournaments will even out the differences in tournaments, producing a single number that represents the player's average ability.
On August 08 2012 01:38 Markwerf wrote:
Point systems are elegant in their simplicity which make it easy to make statements like: "if MC beats DRG here and goes on to win the GSL he is the new #1!!" but the conditions for it to work properly are just not in place for starcraft. You would have to contrive difficult attendence penalties etc. to make it work. The system is already a bit ugly now by implementing a regression based decay for the points which is imo unneccesary difficult since you would have to update your rankings each day then if you plan to keep this up. Just using the tennis system where results count equally till they are removed after a year is far easier as you only need to update after tournaments for the players involved that year or the previous. If neccesary you can make a two stage drop, ie points are halfed after half a year or something but regression based is just not elegant. Remember that elegancy is the primary reason to choose a point system over an ELO based system in the first place..
Everyone else was complaining that there WASN'T regression. This resulted in HuK and IdrA in the top 10, despite not doing too well recently. And as for updating the rankings each day, it takes all of 2 minutes. I open Excel, it auto executes the new rankings, and I sort by points. Done.
On August 08 2012 01:38 Markwerf wrote:
Finally if you implement this you would be best to do a sensitivity analysis. Rank your tournaments in order of importance which is much easier to get an agreement about, for example (1 GSL 2 MLG, IPL, WCS, Dreamhack 3 others) or just by prize money and than use some software to calculate your top 10 or so with different weights. You can see then how much the weighting really matters for the relative ranking of your top players.
edit: by the way Wimbledon is also entitled to change seedings based on grass results only. So in a way they are not only seeding by overall ranking but also by a different ranking similar to how tournaments like GSL just use their own results.
That is EXACTLY what I did, if you would look at the weightings. These are the values for winning the event GSL - 4000 MLG Championship, IPL, Dreamhack, etc (the non-GSL Grand Slams) - 2000 IEM Tour Stops, MLG Arenas - 1500 WCS Continentals - 1000 WCS Nationals, Large "Major" Tournaments (see Revenge of the Nerds, Red Bull Battlegrounds, etc) - 500 Small "Major" Tournaments - 250
As for GSL, I took a look at the official GSL Rankings, I think I will actually use those for GSL now, meaning winning GSL gives you 5000. And it will include Code A, which some people were saying needs to be in.
As for Team Leagues, I think I came up with a way for it to work. Because rewardings for only All Kills are too dependent on that player's team makeup, but rewarding a team victory favors poor players on good teams, I will give just bonus points for each win by that player in a team league. I'm thinking 20 points per win in GSTL, 10 points per win in IPL TAC/IPTL, 5 points per win for EGMC. Should I then include smaller team leagues like the ISTL or STL though?
I'm gonna hold off on adding Team Leagues for now, I want to see if people think those numbers need to be adjusted. I'm really just throwing them out there for now.
On August 08 2012 01:57 Greggle wrote: Stephano above MVP, MKP and MMA? I think it got worse =/
Stephano in the past 5 months has won NASL, Top 6 at 2 MLG Arenas, Top 6 at MLG Spring, Top 6 at IPL4, and SF at Dreamhack Summer. There is a reason that Stephano is as high as he is, it's because he does REALLY WELL at a lot of events. MVP hasn't done much since his last GSL win. MKP in the same time span won MLG Winter (which is decaying), 3rd at MLG Spring, but only Ro16 in GSL, and no BIG wins. And come on, I would think Stephano being that high is not a surprise.
On August 07 2012 14:17 FabledIntegral wrote: Honestly, it's far too subjective for me imo, since it's really just your opinion. I think I'll stick with TLPD, but who knows, maybe your way will catch on!
well taeja is no.1 on tlpd for months now and he has certainly not been no.1 for months (I still wouldn't even consider him to be top 4 until he makes at least a final in GSL because ro8 is not enough to call him no1) he has been number one for a long time now (he became number 1 shortly before joining liquid....) and he certainly couldn't be considered even top 10 at that time he just won online stuff in koreaand they ranked him number 1 so think tlpd is far more subjective than this eventhough this is also kinda messed up imo
The TLPD ELO scores might not represent the true top 5 players for korea and international, but its a nice way to see which players are hot right now. Players like Vortix and Titan are not the powerhouses you expect from Europe, but their recent results are pretty amazing.
I still don't accept Stephano as being top 5, I don't think you are valuing the GSL enough. To start off the RO8 is severely undervalued. If I look at any of the past 3 seasons RO8 players I can easily see any one of them taking an MLG if they were to attend.
Also, the GSL is a ~2 month tournament with the hardest qualifiers in the world, which aren't being counted at all. You can't compare that to a weekend tournament where you can plow through a pool of players which is 6x the size of the pool of players that can actually win it. If you look at a code S pool close to half of the players could have an argument made for them winning the tournament at the start of the season (and often times a player that seems hopeless to win such as Seed wins anyway). Every weekend tournament is padded with players who simply don't stand a chance, versus the GSL which has a constant pool of both champions and up and comers with few if any remaining fillers.
To account for the difference in average player skill, lower frequency of events and time investment the GSL needs to be worth at least 6000-8000 for first place, and with a slower degradation of points for lower finishes.
Thank you for making this. It was an interesting read.
I don't agree with the results, and strongly disagree with the relation you made to tennis. In tennis the best players all compete at the same 4 grand slam tournaments every year, where I believe the only grand slam tournament sc2 has where the best all come to is the GSL.
All of the European tournaments are heavily european player based and all of the american tournaments are heavily NA player based. The GSL is heavily korean based, but has been proven that only few foreigners who are good enough can come and compete at their level. This is why I think the GSL should be treated points-wise as a grand slam, compared to a smaller invitational tournament in tennis(ASUS, NASL, Dreamhack, IEM, etc.).
GSL should be highest in points by a lot. Many foreigners have come to Korea with very little success.
I'm in the the process of changing it again. GSL is now worth 5000 for winning (compared to the foreign grand slams, which are worth 2000). I'm adding in Code A now, following the GSL Ranking system, which look like this + Show Spoiler +
On August 07 2012 16:27 NHY wrote: Tournament Weightings should be based on the overall level of competition. Good way to do this is to look at Ro32 player list of any given tournament. It tells you how "hard" it would be to win that tournament.
And if you are going to skip Code A, then you'd also need to do the same for MLG arena, non-WC IEM, WCS nationals, etc.
IMO, x2 GSL points, add points for Code A, x1/2 for everything else but MLG.
Code A seems too weird to put in, its like an odd losers bracket of a different tournament. What points would you do for someone who gets knocked out RO32 Code S, and wins back into Code S, vs someone who gets RO16, but is then knocked out in their Code A games? Do up and downs matter? Seems too convoluted.
I am not MvP fan but to have any ranking that shows that he is not the most accomplished player/top of the ranking is silly. MC did achieve a lot but MvP should still be 1st. GSL should be worth 2x as much due to the fact that it gives you so much time you prepare against known opponent so it shows how great you are when both sides can prepare their best.
nice idea, but has a few flaws. Doesnt take GSTL results into effect (and maybe it shouldnt) but seeing the likes of gumiho so far down when he destroys in the team leagues is odd.
having Ret and Mana who have never won anything? ahead of 2 time MLG winnner HuK who also has homestory and a DH under his belt? might be wrong about that :p
still good job, you obviously put some time into this
wrong violet more focus on gsl plz be more precise - you missed some tournaments. some weighting is very questionable.
also, some promotion for my ranking if i may do so (also, i think this should be a blog @ OP. although its obviously way harder to get ppl to see it, rankings like this belong to "blogs" imo)
I like the idea quite a bit, but I feel like the best way to do this would be a collaboration of many people and their ideas for how to weight certain tournaments and so forth. Great idea though.
On August 08 2012 03:58 ThatGuy89 wrote: nice idea, but has a few flaws. Doesnt take GSTL results into effect (and maybe it shouldnt) but seeing the likes of gumiho so far down when he destroys in the team leagues is odd.
having Ret and Mana who have never won anything? ahead of 2 time MLG winnner HuK who also has homestory and a DH under his belt? might be wrong about that :p
still good job, you obviously put some time into this
Well, the problem with team leagues has been talked about before.
As for your second paragraph, the results decay as time passes. So Ret (Who got second at WCS Netherlands, a SF at NASL S3 recently) compared to HuK (QF at NASL, 10th at WCS Canada, only some moderate MLG success), shows that lately, Ret has been the better player. This isn't an all time ranking, more recent results matter more.
ALSO!
I have changed things round again. GSL is worth more (5000 points for winning), and Code A is added. This has made it so that GSL success is more important. So here is the top 10:
As you can see, the added importance in GSL means that Seed is now in the Top 10, and we see Nerchio drop down to 12, since his Dreamhack is less important compared to GSL now. Stephano is Stephano still though.
seems like a good idea, just wondering weather you should not give points for at least the OSL as its fully SC2 now too and it is (money and fame wise) the one of the, if not THE biggest Esports League.
On August 08 2012 04:08 SoniC_eu wrote: How are the points weighted? U just made up numbers for each tourney win?
Well, GSL is now based on the ranking points GSL themselves gives out. The others, well yes I "just made up numbers", but the numbers are based off of prestige of the various events. And I've made changes based on feedback from people in this thread. Technically, every single rating style similar to this "just makes up numbers" for tournaments, because there is no easy way to just make a number for the strength of the tournament.
On August 08 2012 04:16 gCgCrypto wrote: seems like a good idea, just wondering weather you should not give points for at least the OSL as its fully SC2 now too and it is (money and fame wise) the one of the, if not THE biggest Esports League.
OSL isn't finished yet. As I said in the OP, I list tournaments from the date they FINISH. I will add OSL in once it is finished. If you mean the qualifiers, the only "qualifiers" I have in the rankings are Code A and the WCS Nationals. Adding the OSL qualifiers would be like adding the Code A Prelims.
It would be arbitrary to define the "weightings" for all of these things. And it would be arbitrary overall to claim that this system of measurement actually works in any way and offers any kind of improved or refined way of deducing what this ultimately tries to.
On August 08 2012 04:08 SoniC_eu wrote: How are the points weighted? U just made up numbers for each tourney win?
Well, GSL is now based on the ranking points GSL themselves gives out. The others, well yes I "just made up numbers", but the numbers are based off of prestige of the various events. And I've made changes based on feedback from people in this thread. Technically, every single rating style similar to this "just makes up numbers" for tournaments, because there is no easy way to just make a number for the strength of the tournament.
On August 08 2012 04:16 gCgCrypto wrote: seems like a good idea, just wondering weather you should not give points for at least the OSL as its fully SC2 now too and it is (money and fame wise) the one of the, if not THE biggest Esports League.
OSL isn't finished yet. As I said in the OP, I list tournaments from the date they FINISH. I will add OSL in once it is finished. If you mean the qualifiers, the only "qualifiers" I have in the rankings are Code A and the WCS Nationals. Adding the OSL qualifiers would be like adding the Code A Prelims.
I was more talking ablut OSL dual tournament, wich finished just today and would be at least equivalent to code a. However i agree to the OSL qualifiers not being listed etc.
Really great work man I appreciate this, although your margin of error is significant in terms of skill and rank. But for a lifetime of sc2 and success the is sweet!
GSL should be excluded if you are trying to factor in only skill or be worth less. It depends a lot on preparation. There is a reason TaeJa didn't win the last one...
Hmm... I'm not sure doing something like having 2499 vs 2500 points matters whatsoever in GSL second place. I understand what you were trying to do with making a first better than two seconds, but relatively when including other points, it can significantly alter things. For example, is first in dreamhack and fourth in MLG really better than 2nd in GSL? And if so, is it better by such a small margin?
The subjectivity KILLS me here, but alas, it still provides an interesting subject.
It's a nice idea, but without sufficient rationale for the weighings and without online tournaments and team leagues it's not very useful. The regression is nice.
On August 08 2012 05:15 NOOBALOPSE wrote: GSL should be excluded if you are trying to factor in only skill or be worth less. It depends a lot on preparation. There is a reason TaeJa didn't win the last one...
Thats like saying San is a better protoss than Seed is despite Seeds GSL Championship and his win today over Coca just because San beat Coca in a Bo3 in the OSL Prelims
MVP absolutely dominated the MLG he won and is the most sucessful player in the GSL.
MC, DRG and MVP have all been in the finals of the GSL this year and can dominate foreign 3 day tournaments on a good day. If the GSL isn't included or is worth less, MVP wouldn't be in the top ten and MC/DRG would be lower down than they deserve.
With some more tweaks to the points I think this can be a very accurate depiction of who's on top of SC2. Especially with the volatility that ELO has in SC2. In BW they had the Kespa rankings, which while still subjective provided another view on the top players (Bisu was often ranked lower than his ELO because Kespa weighed Starleague performance higher than Proleague, where Bisu would just faceroll nerds every week). The same could be said for DRG back in 2011 before he got into GSL, where he was the team league monster but couldn't make it into the individual leagues.
We don't have a foreign scene and it would be a good incentive for our community to create a ranking system that has more accuracy and longevity than ELO does and combines the Koreans and foreigners together, especially when we are hoping that SC2 becomes more of a global game than BW was
On August 08 2012 05:23 FabledIntegral wrote: Hmm... I'm not sure doing something like having 2499 vs 2500 points matters whatsoever in GSL second place. I understand what you were trying to do with making a first better than two seconds, but relatively when including other points, it can significantly alter things. For example, is first in dreamhack and fourth in MLG really better than 2nd in GSL? And if so, is it better by such a small margin?
The subjectivity KILLS me here, but alas, it still provides an interesting subject.
2499 is exactly what GSL uses. The GSL points are exactly the points that are given by GSL. Nothing else is above 2000, winning MLG is 2000. Of course it is subjective, but I've adjusted the points based on feedback to get it in line to what the community think are important. But it's hard to make changes when people don't give any decent feedback, just saying "the points system sucks!", and that's it. Considering the amount of quality players at the foreign grand slams, I would think that winning a Dreamhack or MLG is pretty equal to a SF result at GSL. If winning a single Dreamhack makes ThorZain a top foreign player, then it is clear that Dreamhack/MLG/etc are pretty worth it.
As a fun experiment, I'm going to use the rankings as they are to try and predict the WCS South Korea Winner's Bracket results. When two players match up, I'll take the player with the higher ranking. If a KeSPA player shows up (since they only have the Proleague, which hasn't been put in yet), I'll just take the non-KeSPA player. Sorry KeSPA player fans + Show Spoiler +
Obviously this doesn't take into account history between the 2 players (look at DRG vs HerO for example), or the player's specific ability in the specific matchups, it simply takes the player with a higher ranking according to my math to win. Just felt like making the prediction
Very interesting and I've always wondered why we didn't refer to Tennis more than that. But we should keep only results on latest season counts no? Per year basis, or the latest championship to be taken into account (for exemple NASL4 in place of NASL3, ie erase all of NASL3 result and only counts NASL4).
One possible way to improve weighting of tournaments would be to influence the weighting by the average ELO across the competitors. In theory this could allow tournaments with a stacked player pool to become worth more than an 'easy tournament' where the player pool is of a lower quality. This would also help deal with when there are multiple tournaments on in one day and a powerhouse player goes to the tournment with the weaker player pool and sweeps it for the easy money. Extending that could be that you ignore the ELO of the winner, so that if a player like Stephano with his 2.4k ELO wins a tournament where the average ELO is only 1970, the Average wouldn't be inflated artificially by his huge ELO
However this would be very time consuming and would be something that you would want to automate. Especially when you have to deal with 200+ player open tournaments at MLG or the ro128 in Dreamhack.
Just spotted one strange thing quickly and I'm not sure if there are other errors, but for example SeleCt got 2nd in an MLG. The scale says that 2nd in an MLG is worth 1200, but his total rating is only in the 700's. But definitely a cool concept.
I really wouldn't say that GSL SF appearance is on par with an MLG or DH win. GSL is higher just because every single person you play is Code S. While foreign tournaments attract a lot of top talent, players usually don't have to smash their way through 5+ Code S players to get first. They usually can win their groups against a lot of foreigners and a decent Korean, then play through the winner's bracket.
On August 08 2012 06:28 Darkhoarse wrote: Just spotted one strange thing quickly and I'm not sure if there are other errors, but for example SeleCt got 2nd in an MLG. The scale says that 2nd in an MLG is worth 1200, but his total rating is only in the 700's. But definitely a cool concept.
The points degrade over time. So a 1200 point second gets smaller over time.
You're still avoiding the option of just making an overall ELO or even glicko/trueskill rating which would just be far superior systems. Whatever you tweak your system has two flaws that can't be fixed by tweaking the in your eyes "only subjective part": - no correcting for attendence - difficulty of assessing points for tournament These flaws are too big to ever consider such a system over elo or glicko whatever. The traditional problems of ELO like point sitting and opponent selecting are not even present so you should really just make a world version of that if you want a.proper rating system. Since the korean scene and foreigner scene actually mix up a bit more now then before it wouldn't even be too bad. Fine tuning compared to TLPD could definitely be done like giving bigger matches a higher K value or choosing a different underlying assumption for distribution of skill.
Either way this system is a completely dead end but you seem to have a misguided hope that number tweaking can actually fix it, it won't.
I would still prefer to keep 100% from a tournament until that tournament is played again, but this is also fine.
Some comments on this vs ELO: 1) ELO tries to estimate the skill of players right now. This measures recent achievements. These two are quite different, and you cannot really say that one is better than the other, as they measure different things. Taeja is a good example. He tears up skilled people right and left in smaller tournaments, but has not really placed very high in many big individual tournaments. Thus ELO ranks him high, this ranking not as much. On the other hand, a player like MC really knows how to put himself together and win (or place high in) big events, but can drop games to lower ranked players in less important occasions. Thus MC gets a very high score for recent achievements, but maybe not as high in ELO.
2) Korea vs foreigners. The two communities are a bit separated, so it is hard to compare between the two. ELO solves this by not comparing at all. Thus two separate rankings. This point ranking tries to compare them, but runs into the problem of balancing the points of GSL vs other tournaments, and the problem that some players participate in more tournaments than others. Compare to tennis where all top players play in all big tournaments. None of the two solutions are very impressive.
3) The two systems have different flaws, so it is useful to look at both. ELO is very good because it has little subjective bias. A win is a win, no matter the context. Some bias in which games you include, but probably not a huge deal. This point system, as has been pointed out a lot of times, is very subjective in terms of how the points are distributed. The OP has done a good job of trying to find a distribution that most people can accept, but even with the full support of the community, you cannot get away from the fact that the points are distributed in a subjective manner.
The "a win is a win" in ELO is also one of it's main problems. beating MC in the GSL finals is worth as much as beating him in the round of 8 in a weekly tournament. Many would argue that winning in the GSL finals is a much larger indicator of skill than beating them in a weekly tournament. The argument would be that people prepare for the GSL, and will always bring his very best game, while in a weekly, people may not play at their very best in every game, due to various circumstances. This is better taken into account in a point system, where big tournaments count more.
So putting them together, ELO can be seen more as a "current raw skill" in some sense, while the point system is more "recent achievements". Also note that ELO tries to measure skill right now, while the point system is counting over the last 6 months, and will lag behind a bit. So a new player coming out of nowhere will jump up in ELO very fast, but will not reach his peak in the point system until after about half a year after his first win. Thus the point system measures consistency a bit as well.
Best picture can be seen by looking at both. High ELO but low point ranking means either a rising top player, or a skilled player that is not very clutch in big tournaments. Low ELO but high point ranking is either a declining top player, or a good "tournament player".
This kind of tournament system doesn't work because of the disparity between korean and foreigner skill, because koreans usually dominate foreign tournaments but many of them don't get to go to them, and likewise many foreigners don't get to go to korean tournaments.
So you need to separate between America, Europe, Asia etc
On August 08 2012 05:23 FabledIntegral wrote: Hmm... I'm not sure doing something like having 2499 vs 2500 points matters whatsoever in GSL second place. I understand what you were trying to do with making a first better than two seconds, but relatively when including other points, it can significantly alter things. For example, is first in dreamhack and fourth in MLG really better than 2nd in GSL? And if so, is it better by such a small margin?
The subjectivity KILLS me here, but alas, it still provides an interesting subject.
2499 is exactly what GSL uses. The GSL points are exactly the points that are given by GSL. Nothing else is above 2000, winning MLG is 2000. Of course it is subjective, but I've adjusted the points based on feedback to get it in line to what the community think are important. But it's hard to make changes when people don't give any decent feedback, just saying "the points system sucks!", and that's it. Considering the amount of quality players at the foreign grand slams, I would think that winning a Dreamhack or MLG is pretty equal to a SF result at GSL. If winning a single Dreamhack makes ThorZain a top foreign player, then it is clear that Dreamhack/MLG/etc are pretty worth it.
As a fun experiment, I'm going to use the rankings as they are to try and predict the WCS South Korea Winner's Bracket results. When two players match up, I'll take the player with the higher ranking. If a KeSPA player shows up (since they only have the Proleague, which hasn't been put in yet), I'll just take the non-KeSPA player. Sorry KeSPA player fans + Show Spoiler +
Obviously this doesn't take into account history between the 2 players (look at DRG vs HerO for example), or the player's specific ability in the specific matchups, it simply takes the player with a higher ranking according to my math to win. Just felt like making the prediction
What the GSL uses itself is internally. I already made it very clear that it works with an internal system, where you can weight a single first place finish more than two second place finishes. I said where it fails to work is when you extrapolate it to be used with other point systems as well.
That's the issue, where you're using whole number typically divisible by 100 (or at least 10), the second place becomes exponentially less valuable due to missing a *single* point. For example, if the second place was worth 2501 as opposed to 2499, it would have a huge impact on the rankings for those who got second place one would think, when in reality 2 points should be of absolute negligible value to anyone in the top 50 or so.
On August 08 2012 09:12 dgwow wrote: This kind of tournament system doesn't work because of the disparity between korean and foreigner skill, because koreans usually dominate foreign tournaments but many of them don't get to go to them, and likewise many foreigners don't get to go to korean tournaments.
So you need to separate between America, Europe, Asia etc
Building further upon the idea of weighting tournaments by the average elo of the players within it, you can create a factor that affects the ELO between the two and this is only needed between Korea and the foreigner ELO tables, you can work this out pretty easily too, MCs current foreign ELO is around 10% higher than his korean ELO. If you multiply naniwas korean ELO by 10% he gets placed just below Stephano and Nerchio to be the 3rd best foreigner, which I would consider pretty accurate. If you apply it to Taeja his foreign ELO is over 2.5k so this method would have to be refined, by maybe taking the average factor of which Korean ELO translates to Worldwide ELO.
You can then work out from that how the average ELO of a nation/region is which can be used to further weigh the value of regionals like SA and SEA qualis where the player pool is of a lower average skill when compared to NA/EU/KR.
People, when it comes to the weightings of individual tournaments, OF COURSE it is subjective. Any way of doing it is subjective. If you think there should be a specific change, please say it. But if you are just saying "They are subjective, this is bad", you are not helping whatsoever. Other people have gone over the problems with ELO and the TLPD. This isn't a perfect system meant to replace ELO, but it can compliment it.
As for the idea of using ELO to rate tournaments, that is certainly a good idea. I would have to think about it and how to make ti work before implementing it (and reweighing and rewriting all the data I've created over 2 days -_-). If I do do that, I'm thinking that incorporating both the mean and the median would help to deal with very low or very high ELOs.
On August 08 2012 06:23 Skytt wrote: One possible way to improve weighting of tournaments would be to influence the weighting by the average ELO across the competitors. In theory this could allow tournaments with a stacked player pool to become worth more than an 'easy tournament' where the player pool is of a lower quality. This would also help deal with when there are multiple tournaments on in one day and a powerhouse player goes to the tournment with the weaker player pool and sweeps it for the easy money. Extending that could be that you ignore the ELO of the winner, so that if a player like Stephano with his 2.4k ELO wins a tournament where the average ELO is only 1970, the Average wouldn't be inflated artificially by his huge ELO
However this would be very time consuming and would be something that you would want to automate. Especially when you have to deal with 200+ player open tournaments at MLG or the ro128 in Dreamhack.
Why is there no negative values? Is this supposed to be purely a measure of volume?
Kind of a waste of time as is. You have the infrastructure, you should really go back and try something else. Maybe get someone with some statistics background to give you a hand because you've done most of the legwork already.
I appreciate the effort but this is a tournament participation ranking. If that's what you are aiming for that's fine, but sc2 earnings gives me a good enough guide for tourament participation. Earnings also naturally factor in tournament weighting somewhat, as a tournament with more prize money will have more 'weight'.
If you are aiming for a good skill based ranking, you can debate the particulars but you can't avoid the general ideas behind ELO.
I guess the disadvantage of ELO is the calculation might seem mysterious to most people so one may wonder if they did their math right. Whereas a weighted participation-based system is easier for everyone to understand and see what's going on.
On August 07 2012 14:28 goswser wrote: HAHAHA Mana is higher than nestea.
cool points eh? : P I hear Nestea hasn't won much ^^ Just a GSL or two lol
WAIT EDIT!
IF GSL is 5000, I don't get it... MVP won 4 GSL's meaning 20,000 pts no? that goes for Nestea for 3 and MC for 2 right?
Nestea hasn't done well in the recent past, which is what this measures more. People wanted a regression (until 1 year past the event, when the event no longer adds in), so events further in the past mean less. Same with MVP (except for his latest one).
I think that wins in a team league environmen should be saved for a team ranking. This is really cool and I like this more than the TLPD ELO style. It changes so much compared to this. Plus the other top 10's, like FlameWheel's are completely subjective.
The enormous favor given to GSLs is pretty disgusting considering that this system gives no consideration to tournament format (some players like Supernova are plain bad at competing in compressed tournaments, but they automatically get more points just by maintaining a constant GSL presence). I don't understand the point system either, it seems rather arbitrary.
On August 08 2012 12:43 CosmicSpiral wrote: The enormous favor given to GSLs is pretty disgusting considering that this system gives no consideration to tournament format (are preparation-based tournaments necessarily worth more than compressed ones?). The point system should also be explained, as the spread seems incredibly random and pointless right now.
What do you mean explained? Do you mean the exact reasoning behind the points, or the points themselves. If the second, check the OP. If the first, well it is subjective. I had an initial spread, but changed things around due to feedback. A lot of people think that GSL should be more important than the other events, so I made it so. Any points ranking system will be subjective. I adapted Tennis style points, added in GSL Rankings, and used the base to make points for other tournaments. I know it won't please everyone, but I feel a majority wants GSL ranked higher than the others, seemed to be a pretty clear majority.
It is an interesting system and could work if worked on more, but it is just plain hard to group together foreigners and koreans ... in tennis you have pretty universal representation at all the majors ... the top players will be there sans an injury ... but with a lot of korean competitions there is hardly any foreigner presence and with travelling to foreign competitions.. a lot of koreans do not get to do this much or do not wish to...
I think your top 10 has decent weight but extending it out to a top 100 is too much of a stretch .. there just isn't enough cohesion among the two scenes for there to be a truely "good" method to rank foreigners and koreans together in my opinion
On August 08 2012 12:43 CosmicSpiral wrote: The enormous favor given to GSLs is pretty disgusting considering that this system gives no consideration to tournament format (are preparation-based tournaments necessarily worth more than compressed ones?). The point system should also be explained, as the spread seems incredibly random and pointless right now.
What do you mean explained? Do you mean the exact reasoning behind the points, or the points themselves. If the second, check the OP. If the first, well it is subjective. I had an initial spread, but changed things around due to feedback. A lot of people think that GSL should be more important than the other events, so I made it so. Any points ranking system will be subjective. I adapted Tennis style points, added in GSL Rankings, and used the base to make points for other tournaments. I know it won't please everyone, but I feel a majority wants GSL ranked higher than the others, seemed to be a pretty clear majority.
"I wanted GSL to be the top event, since it is most prestigious of all events" is not sufficient reasoning to rate it higher than other events. Premier events are considered more prestigious than majors, monthlies, weeklies, etc. because it is implied that the best competition attends those events even when it is not the case (IEM San Paulo anyone?). Ideally the system would show that GSL has the best competition in the world and not assume it.
I think you should just start from scratch, figure out what you want your system to do, and build it around those goals.
Feedback needs to be considered, not necessarily accepted.
On August 08 2012 13:18 CosmicSpiral wrote: I think you should just start from scratch, figure out what you want your system to do, and build it around those goals.
Feedback needs to be considered, not necessarily accepted.
Listen, you can think that it needs a change, but don't put words in my mouth. I wanted to make a ranking system based on Tennis, with fixed points for the events based on prestige, and that is what I did. "figure out what I want my system to do?" I can see myself in the short future trying a different variety using the TLPD ELO ranks to determine strength for tournaments, but don't say that I failed what I wanted to do.
On August 08 2012 13:18 CosmicSpiral wrote: I think you should just start from scratch, figure out what you want your system to do, and build it around those goals.
Feedback needs to be considered, not necessarily accepted.
Listen, you can think that it needs a change, but don't put words in my mouth. I wanted to make a ranking system based on Tennis, with fixed points for the events based on prestige, and that is what I did. "figure out what I want my system to do?" I can see myself in the short future trying a different variety using the TLPD ELO ranks to determine strength for tournaments, but don't say that I failed what I wanted to do.
You are making a ranking system for SC2. Why don't you make a ranking system specifically designed for SC2? The tennis scene is very different so the same principles cannot be applied. It's the same reason why a certain chess-based ranking system has failed to reliably rank SC2 players.
I bring up the question because there are so many scenarios and problems that are simply not addressed.
- What about showmatches? - What about weekly/monthly tournaments? - What about invitationals that are not premier events? - What about qualifiers? - Should team leagues have a different ranking? Should they be included at all? - How do you account for close set/serieslosses? - How do you account for individual runs against top competition that does not result in a high tournament placing? - How do you account for the overall player strength of an individual tournament? - How do you account for sickness, fatigue, injuries, etc. that result in poor placements? - How do you account for matchup strength? - How do you account for monetary issues that limit attendance at premier tournaments? - How do you account for deviations that the system will interpret incorrectly? - What are the obvious problems that the system will create, and how should they be dealt with?
On August 08 2012 13:18 CosmicSpiral wrote: I think you should just start from scratch, figure out what you want your system to do, and build it around those goals.
Feedback needs to be considered, not necessarily accepted.
Listen, you can think that it needs a change, but don't put words in my mouth. I wanted to make a ranking system based on Tennis, with fixed points for the events based on prestige, and that is what I did. "figure out what I want my system to do?" I can see myself in the short future trying a different variety using the TLPD ELO ranks to determine strength for tournaments, but don't say that I failed what I wanted to do.
Mate, he's got a point. In "making a ranking system based on Tennis," you haven't necessarily made a meaningful ranking system for SC2. At best, you can only call it "my list of the best players right now."
On August 08 2012 13:50 CosmicSpiral wrote: You are making a ranking system for SC2. Why don't you make a ranking system specifically designed for SC2? The tennis scene is very different so the same principles cannot be applied. It's the same reason why a certain chess-based ranking system has failed to reliably rank SC2 players.
I bring up the question because there are so many scenarios and problems that are simply not addressed.
- What about showmatches? - What about weekly/monthly tournaments? - What about invitationals that are not premier events? - What about qualifiers? - How do you account for individual runs against top competition that does not result in a high tournament placing? - How do you account for the overall player strength of an individual tournament? - How do you account for sickness, fatigue, injuries, etc. that results in poor placements? - How do you account for matchup strength? - How do you account for monetary issues that limit attendance at premier tournaments? - How do you account for deviations that the system will interpret incorrectly?
Before you ask questions, please actually read the OP and the thead, most of these questions are answered already.
On August 08 2012 13:50 CosmicSpiral wrote: You are making a ranking system for SC2. Why don't you make a ranking system specifically designed for SC2? The tennis scene is very different so the same principles cannot be applied. It's the same reason why a certain chess-based ranking system has failed to reliably rank SC2 players.
I bring up the question because there are so many scenarios and problems that are simply not addressed.
- What about showmatches? - What about weekly/monthly tournaments? - What about invitationals that are not premier events? - What about qualifiers? - How do you account for individual runs against top competition that does not result in a high tournament placing? - How do you account for the overall player strength of an individual tournament? - How do you account for sickness, fatigue, injuries, etc. that results in poor placements? - How do you account for matchup strength? - How do you account for monetary issues that limit attendance at premier tournaments? - How do you account for deviations that the system will interpret incorrectly?
Before you ask questions, please actually read the OP and the thead, most of these questions are answered already.
You haven't addressed them. You've dismissed them because you have not figured out how to incorporate them into your system. You even ignore Code A results because "I didn't want to think about how to work that out." Huge essential parts of the SC2 tournament scene have no meaning in your system.
On August 08 2012 13:18 CosmicSpiral wrote: I think you should just start from scratch, figure out what you want your system to do, and build it around those goals.
Feedback needs to be considered, not necessarily accepted.
Listen, you can think that it needs a change, but don't put words in my mouth. I wanted to make a ranking system based on Tennis, with fixed points for the events based on prestige, and that is what I did. "figure out what I want my system to do?" I can see myself in the short future trying a different variety using the TLPD ELO ranks to determine strength for tournaments, but don't say that I failed what I wanted to do.
You are making a ranking system for SC2. Why don't you make a ranking system specifically designed for SC2? The tennis scene is very different so the same principles cannot be applied. It's the same reason why a certain chess-based ranking system has failed to reliably rank SC2 players.
I bring up the question because there are so many scenarios and problems that are simply not addressed.
- What about showmatches? - What about weekly/monthly tournaments? - What about invitationals that are not premier events? - What about qualifiers? - Should team leagues have a different ranking? Should they be included at all? - How do you account for close set/serieslosses? - How do you account for individual runs against top competition that does not result in a high tournament placing? - How do you account for the overall player strength of an individual tournament? - How do you account for sickness, fatigue, injuries, etc. that result in poor placements? - How do you account for matchup strength? - How do you account for monetary issues that limit attendance at premier tournaments? - How do you account for deviations that the system will interpret incorrectly? - What are the obvious problems that the system will create, and how should they be dealt with?
Hmm, I agree with you in general, that the point system has some big flaws and that it probably will not get used widely.
But I think you are overdoing it a bit when it comes to shoot him down. He has made a fair try to convert the tennis point system to starcraft, I think that deserves more than this kind of replies.
Of course a completely new system for sc2, that perfectly addresses every little detail you can think of would be really sweet, but it doesn't seem to be that easy to design. Which is why we have many different rankings in parallel, like the ELO from TLPD, ladder rankings, sc2earnings, power rankings etc. Unless you have some great idea to solve this problem, I don't feel it is fair to be too aggressive to people that works on new ways to approach the problem.
For the list, you are just doing an as long list as possible for the sake of it... Do you for example really need to take teh argument "you need to decide how many, if any, points to give to all the smaller events." and plit it up into the first four points? Why would you treat a close loss any different? Why would you treat an injured player any different? Why would you treat different matchup differently (do you treat left handed players differently in tennis?)? Anyway, I just feel that you add things to the list to make the list long rather than actually trying to say something.
But again, I agree with your general standpoint, and imo, the one big insurmountable problem is your third last point (that you list together with all the random ones...). ie that all the best players don't play in all the top tournaments. Which is the one reason that this point system isn't really going to work that well.
The second largest issue is that people would have to agree on how to distribute the points among the tournaments, but I think that could be done decently if it were not for the main problem. (Which would be your point 1 to 4 more or less. )
The rest of the details you bring up I think could be sorted out, or ignored. If the main problem was not there.
On August 08 2012 13:50 CosmicSpiral wrote: You are making a ranking system for SC2. Why don't you make a ranking system specifically designed for SC2? The tennis scene is very different so the same principles cannot be applied. It's the same reason why a certain chess-based ranking system has failed to reliably rank SC2 players.
I bring up the question because there are so many scenarios and problems that are simply not addressed.
- What about showmatches? - What about weekly/monthly tournaments? - What about invitationals that are not premier events? - What about qualifiers? - How do you account for individual runs against top competition that does not result in a high tournament placing? - How do you account for the overall player strength of an individual tournament? - How do you account for sickness, fatigue, injuries, etc. that results in poor placements? - How do you account for matchup strength? - How do you account for monetary issues that limit attendance at premier tournaments? - How do you account for deviations that the system will interpret incorrectly?
Before you ask questions, please actually read the OP and the thead, most of these questions are answered already.
You haven't addressed them. You've dismissed them because you have not figured out how to incorporate them into your system. You even ignore Code A results because "I didn't want to think about how to work that out." Huge essential parts of the SC2 tournament scene have no meaning in your system.
And if you had read the thread, or even gone through the list of points in the OP, you would know that he now does include code A. (But didn't update the OP properly...) 8 pages isn't too much to read if you are going to criticize this hard imo. You really seem to be here just to shoot him down, rather than objectively trying to evaluate the virtues and problems of his calculations.
Again, I agree with your general standpoint though.
The ATP is not the ranking system of tennis. It is the association of tennis professionals that essentially organizes tournaments and the yearly calender for events.
This system won't work for several important reasons: -There is no organized SC2 system (ATP) -There is no official ruling body for SC2 (ITF, currently, things are handled by negative backlash where tournaments would single-handedly offer up their own punishment and teams and managers would add to that. There needs to be an official UNIFIED ruling body to do this sort of thing, as well as make a standardized set of rules of conduct for both players and tournaments.) -There is no organized calender for events. This is by far the most important factor. Events often overlap with each other or take place one after another. This makes "dominating" the scene on an international level insanely hard, seeing as if you win an event, you have about a day or two to fly over to another part of the world and immediately jump into it. Events are literally spammed because the different groups of interest (IEM, MLG, and GSL) are all fighting for the audience as well as the attendance of professionals.
The fact is, if you want a legitimate ranking system, you'd need to get EVERYONE together and have them hammer things out (which I don't ever see happening, seeing how the GSL-MLG partnership really doesn't mean jack squat at this point and IEM isn't even included in it). OR, you can just go with the current system and have the "international" rankings (derived from all non-Korean events) and the "Korean" rankings (based solely on the few Korean events going on).
Although, I do have suggestions for how to almost completely clean up SC2, and I've had them for a good while now... (Hell, the ATP and ITF have more or less had it hammered out for the past 40 years.)
1) KeSpa, GOM-TV, MLG, IEM, and ALL OTHER organizations interested in holding PROFESSIONAL SC2 tournaments as well as the professional players themselves (or at least representatives of the players) need to get together and decide a tournament schedule that would work for everyone involved.
Some key points would be to REDUCE TRAVEL TIMES for the players (so that travel problems would be mitigated).
January-March: US Circuit (MLG). Tournaments will be held around the US, trying to keep to a West to East setup. (So we'd start off with MLG Anahiem then end off with an MLG on the East coast, say, like in New York? Maybe ending with the NASL in NY with top seeds decided by the performance of the players in MLG, with remaining spots given out based on an open tournament.) MLG might have to reorganize themselves for this. Obviously, MLG can't cram all of it's tournaments of numerous different games into a 3 months span. So perhaps they will create some SC2-only events to fit them in, with their events for the remainder of the year being void of Tier 1 SC2 events and holding Tier 2 SC2 events instead (more on that later). April-August: IPL, IEM, Dreamhack, an ESWC I know IPL is in the US, but I see NASL, IPL, and GSL being the biggest events, and having 2 massive events in one section of the year with another section being completely void of one, I though IPL should just be moved back a bit or be moved to take place in Europe. Of course, we can just fly back to the US for IPL, then fly to Korea from there. September-November: KeSpa tournaments followed by an OSL, then the "official world championship of SC2" being the GSL (hell, it's named the GLOBAL SC2 league, let's have GOM take responsibility for that bit and live up to it since we all DO look to the GSL for the best SC2 in the world). After that, we can have an exhibition-type tournament (like Blizzard Cup) with no (or minimal) point rankings due to the small player pool (or we can treat it like the ATP's Year Ending Championships and give it a large pool of ranking points). December: THE PLAYERS NEED A DAMN BREAK! I'd extend it to 2 months or a month and a half if I could, but with KeSpa coming in and more tournaments just popping up, it's hard to really fit everything in. Players SHOULD be allowed time to spend with their families or at the very least have private, uninterrupted practice where they can attempt to make improvements on their play.
I also feel that tournaments should be split up into 4 Tiers.
Tier 3: Online tournaments, low level, low points gain, frequent (can be daily or weekly), low prize pool. 32 Draw, 64 Draw, 128 Draw, or 256 Draw.
Tier 2: Mix of offline/online tournaments (probably primarily online), medium level (up-and-coming players), slightly more points up for grabs, frequent (daily or weekly), slightly larger prize pool. 32 Draw, 64 Draw (max for offline), or 128 Draw
Tier 1: Offline (LAN) tournaments, high level (MLG, IEM, KeSpa), large points gain, bi-weekly? weekly? (that's for the Players and tournaments to agree upon), high prize pool. 32 Draw or 64 Draw (excluding qualifiers)
God Tier: NASL, IPL, OSL, GSL. Offline, highest level, highest points gain (even amonst the first 3 with GSL having a little bit more), preferably once a season (but like I said, kind of hard to fit it that way), mouth-watering prize pool. 32 Draw, 64 Draw, or 128 Draw (excluding qualifiers; 32 recommended for double elimination, 64 for single elimination)
Tournaments of different tiers MAY overlap with each other EXCEPT for Tier 1 and God Tier. This means you can participate in some Tier 2 events as you participate in Tier 1 events if you don't see yourself going far enough in Tier 1 events or don't plan to participate in God Tier events. Most Tier 2 and Tier 3 events won't be heavily streamed and they are mostly played online so it won't matter too much if they overlap, since viewership and travel distance won't be an issue. Blizzard's WCS can be treated like the Olympics of SC2, held once every 4 years, and tournaments can reschedule themselves slightly to make space for the WCS.
Tier 3: 10 Points to 1st, 7 points to finalist (2nd), 5 points to Ro4, 3 points to Ro8, 1 point to Ro16 (if 64 Draw or higher) Tier 2: 100 Points to 1st, 70 points to finalist, 50 points to Ro4, 30 points to Ro8, 10 points to Ro16, 5 points to Ro32, 1 point to Ro64 Tier 1: 500 Points to 1st, 350 points to finalist, 250 points to Ro4, 150 points to Ro8, 50 points to Ro16, 25 points to Ro32, 5 points to Ro64 God Tier: 1000 points to 1st, 700 points to finalist, 500 points to Ro4, 300 points to Ro8, 100 points to Ro16, 50 points to Ro32, 10 points to Ro64 Obviously, these numbers are subject to change to suit the needs of the StarCraft 2 World Tour (or whatever they decide to call themselves).
2) The players (and everyone involved in the teams in general - Managers, coaches, etc) need to get together and decide on rules of conduct during games and at live events (like to what limit is BM actually punishable, the boring, yet somehow necessary video/discussion on why cheating/match-fixing is wrong and should never be committed, etc).
Then the TOURNAMENTS need to get together and do the same, except for themselves. This will likely deal with rules of operation and dates of operation as opposed to what tournaments shouldn't do and what is punishable.
3) Two organizations need to be made (or more, depending on how you want the governing body of SC2 to be structured). One will manage and control the tournaments, and the other will look to protect player rights and try (as well as punish) any sort of player or tournament misconduct. Basically a financial/governing body and a judicial body.
The reason for this is that there have been reports of both players and tournaments who don't pay up.
Points from a tournament will last until the that same tournament from the year after is completed. Meaning points gained from GSL 2013 will last until the COMPLETION of GSL 2014. If a player becomes ill or injured, their points upon report of injury shall be recorded, to be used as a "Provisional Ranking" for admission to tournaments (with minimal rank requirements, such as a Tier 1 or God Tier tournament) upon their return to the tour. This "Provisional Ranking", may be used to entry to tournaments equal to once per month missed (meaning missing 1 month allows you to use your Provisional Ranking on your next tournament, and being gone 3 months allows you to use it for the next 3 tournaments; with a maximum of 6). However, standard point loss shall be applied. (So your ranking will reflect your absence on tour, but will not prevent you from playing tournaments you've worked hard to gain access to.)
It'd be nice for something like this to be used (the ATP and ITF made it work), but the Koreans currently hold a monopoly on both the best players and most prestigious tournaments (the GSL and soon the OSL). Why should they give that up? For the players? For the viewers? Eff that!
Also, since tournaments should ideally be finished in 1 week for Tier 1 events and a max of 2 weeks for God Tier events, it's sort of hard to cram large draw tournaments into that time span (especially for double elimination) while still not overplaying the players. Normally we see a total of 4 matches a day from the GSL. With 6 or 8 casters (or Tastetosis and 2-3 Wolfs), you can fit 3-4 groups of 4 each day. In a 128 Draw SE (single elimination), you can finish the Ro128 and Ro64 in the first 6 days, get Sunday off, then do the Ro16 on Monday, Ro8 on Wednesday, Ro4 on Friday, and Finals on Sunday. Then, you have the predicted "popular" matches (or at least a majority of them) on the free stream, with the rest on pay streams that viewers can sort of tab through to either find their favorite players or favorite matchups. You can have Tastetosis stay on the free streams (so that people are likely to tune in) or torture the freeloaders (myself included) with bad casters (to attempt to give us an incentive to go to the pay streams; though bad casting can ruin a good match from great players). Even doing the group format that we have (hell, the entire current format), we can probably get through things pretty quickly like that. But I would suggest moving Code A BEFORE Code S. That way, we can root for our players as they qualify for the upcoming GSL Code S that will happen immediately after.
There are probably some other things I'm missing, but eh... I don't really care all that much anyway. Realistically, like I said, I don't believe any of this will ever be considered by any of the SC2 tournaments or organizations currently at work. I just felt like venting a bit, throwing out a few ideas. Someone would have to take a stand, and that someone needs to be the players. But they can't really afford to give up what they have right now in hopes of something better. We need another BoxeR to step in and change everything. We don't need to get eSports out to the world anymore. We just need to clean it up so that it runs smoothly and efficiently for everyone involved.
Of course, copying the ATP's format would be a bit boring. So if anyone else could come up with another, nice, make-believe schedule it'd be cool to look at. Obviously you would want to take into account the convenience of the players as much as possible, because it is them and their skills we are trying to market here.
And yeah, it's a messy read, but there's a TON of things that would need to be done before legitimate Global rankings can even be thought of.
On August 08 2012 13:18 CosmicSpiral wrote: I think you should just start from scratch, figure out what you want your system to do, and build it around those goals.
Feedback needs to be considered, not necessarily accepted.
Listen, you can think that it needs a change, but don't put words in my mouth. I wanted to make a ranking system based on Tennis, with fixed points for the events based on prestige, and that is what I did. "figure out what I want my system to do?" I can see myself in the short future trying a different variety using the TLPD ELO ranks to determine strength for tournaments, but don't say that I failed what I wanted to do.
You are making a ranking system for SC2. Why don't you make a ranking system specifically designed for SC2? The tennis scene is very different so the same principles cannot be applied. It's the same reason why a certain chess-based ranking system has failed to reliably rank SC2 players.
I bring up the question because there are so many scenarios and problems that are simply not addressed.
- What about showmatches? - What about weekly/monthly tournaments? - What about invitationals that are not premier events? - What about qualifiers? - Should team leagues have a different ranking? Should they be included at all? - How do you account for close set/serieslosses? - How do you account for individual runs against top competition that does not result in a high tournament placing? - How do you account for the overall player strength of an individual tournament? - How do you account for sickness, fatigue, injuries, etc. that result in poor placements? - How do you account for matchup strength? - How do you account for monetary issues that limit attendance at premier tournaments? - How do you account for deviations that the system will interpret incorrectly? - What are the obvious problems that the system will create, and how should they be dealt with?
All very easy to solve...
1) They're exhibitions. Who gives a damn? They're called SHOWmatches for a reason! They play for a bit of money and have some fun. 2 players did a Bo30 showmatch. Can you REALLY say they were being 100% serious? There were some SERIOUS troll builds in that showmatch and it was purely for entertainment purposes. Whether you're serious or trolling, show matches will be entertaining as long as both players are on the same page.
2) Drop them to a lower tier (less points gained). You can easily participate in these tournaments as they are both frequent and online. As such, they have VERY low prestige and priority ratings, and such shall be reflected by lower point gains.
3) Treat them as exhibitions. Honestly, if there are invitationals without a fair and unbiased entry system, then it would basically be like giving free points to inflate the rankings of undeserved players. If it became that invitations would only be given out to the top 8 ranked (or performing) players that accept the invitation, THEN you can allow those tournaments to give out ranking points. However, by doing that, their status changes from "not premier events" to "premier events". However, they can still be JFF (just for fun) tournaments, and if such designation is given, then they will not be allowed to give out ranking points.
4) Qualifier points should be based on the upper threshold of point gain and the level of tournament they grant access to. For example, if the highest possible amount of points you can gain in a tournament (winning the highest points yielding tournament) is 100 points, and the lowest possible gain in that tournament is 5 points, then qualifiers should not give out points except maybe 1-3 points to the top 8 or top 4. If it was instead 10,000 points and the lowest possible gain was 100 points (just by attending), you could probably give out points to most of the top 32 or even top 64. Obviously, those who qualify cannot "double dip" by gaining both points from the main event and qualifiers. For the most part though, I don't think qualifiers should give points unless the points rating system is extremely high or you give out .1 or .01 points for qualifier events. This is probably why the ATP raised it's points threshold from 1000 to 2000 (to make smaller events more distinguishable in terms of points)
5) Team leagues should not be included. This is how the ITF does it. However, a separate ranking can be made for it, though it would be near 100% pointless since there are not enough team events out there yet to really say much about it. I mean, you can basically do it yourself, assigning point values to each result, giving all tournaments the same maximum point values then write your rankings based on that.
6) A loss is a loss. Is this even a serious question? Double elimination formats even give you a second chance against this, which as a tennis player, I am slightly against. Though as a fan that doesn't want my players to be eliminated, I do support it from a viewer's perspective. Results are what matters. Whoever can consistently bring out the best results on the biggest stages cannot be disputed as the #1 player, regardless of what plays he used to get there (cheese or macro), how close his matches were (be it comebacks of his own merit or throws from his opponents), or how many times he BARELY beat players who are arguable "infinitely better players". That person will simply have had a better past 12 months. He posted consistent results and his peers didn't. You can say his peers are better, but under-performing. That happens all the times. But until they bring results and start winning those close matches, they CANNOT be considered #1 players and should not gain bonus points just for having a close match. That's beyond retarded. How can you objectively state that two players had a close match? What defines a close match? What may seem like a close match to viewers can be a blowout to experts. What seems like a blowout to viewers may have been a close match to experts. It's best to just leave things to the results. If you won, you get the points. If you lost, you don't. Do we give extra prize money to 2nd place because he had a close match? Do we give a SPECIAL Silver medal to 2nd place runners or swimmers in the Olympics because it was a CLOSE race? Because they lost by a tenth of a second? A hundredth of a second? NO. We give them the Silver medal, pat them on the back, and say "better luck next time. Keep training hard and you'll surely win next time."
7) The top competition lost potential points for not performing (or in the double elimination setup, loses one of his two lives). How is this even remotely an issue? The top player wasn't playing at his best. Discussion over.
8) You account for the overall strength of a player in the individual tournament by giving him the appropriate points for whatever place he finished? Like... Seriously bro... The farther I go into this, the more it's like you're just throwing words together to pray for an argument. That or you want some random guy who deserves to be in the top 10, yet not the #1 position to somehow be shoved into the #1 position with a shitty biased ranking system that is nothing more than a popularity contest. There will always be people who will oppose the #1 player in the world being in that position, but more often than not, they deserve that position.
9) They played poorly, someone else played better. Gee... I wonder who's the better player that day... Hmmm... The ranking system is to show who's the best player in the past 365 days. Not who is the best player is they miraculously played at 100% for all 365 days. That's not possible. If that were true in tennis, Federer would not be #1. It would not be Nadal or Djokovic either. It'd be some random player who beats the crap out of the ball and since he's playing at 100% all the time, by sheer luck all his shots land in for winners and he smokes the crap out of every player in the world. Hell, if he did that one day, and we went by your logic, this guy would undisputedly be the #1 player in the world. But... He's not. Why? He doesn't post consistent results. You know who does? Federer does (by some miracle...). Even as a Federer fan, I question his position at #1 in the world. But he's posting more consistent results somehow than the Nadal and Djokovic. So he IS the #1 player in the world right now, and I can't dispute it. I mean, he originally fell out of the #1 position because he had mono. He still stayed at #2 player in the world with it, but he clearly wasn't the best player in the world anymore. Many tennis fans can easily say that if he never got mono, he would've been #1 that entire year as well. But you have to take an objective view on the situation and say that during that time, he was not the #1 player in the world, Nadal was.
10) You can have a stat sheet for that if you want. The best player should be the most well-rounded and consistent player of the tour. It doesn't matter if someone never loses a PvZ. It won't help them unless they get a crazy lucky draw. And that luck won't last the entire year. The guy will have a few good results, and his ranking will reflect it. His ranking won't go up drastically unless he gets better at the other matchups.
11) If you post results at smaller, easier to participate events, and constantly work hard, you WILL get noticed and you WILL get a sponsorship. Player's I've never heard of are getting sponsorships and team support. If you work hard and can get results, you WILL go to premier tournaments eventually.
12) There will NEVER be deviations that the system interprets incorrectly. As long as you create a fair and unbiased system, that can NEVER happen. EVER. WORST CASE SCENARIO: a total unknown comes out and wins the GSL, then never posts any other notable result as he constantly loses in the first round. His ranking will spike up for 1 year, be around mid-level, then drop back down. We get that all the time in tennis. Players randomly get a hot-streak in a major, ride the wave for about a year then fall back off. It is fair to say they don't deserve the ranking they achieved? Not really... They had a good run. If you want to account for these random wins and hot streaks, then look at a player's average ranking over the course of several years. Players like this will never get to he #1 spot anyway. They might break into the top 10 for a while, but what really matters is the top 5, the top 3, the top 1.
13) Well, if there were obvious problems with the system, I'm pretty sure you would've thought of them by now right? I mean... past #5, your concerns really have been nothing but utter bullshit. If you've thrown out 7 non-existent problems, what other "obvious problems" are there? #1-5 were legitimate, though simple to solve concerns. The only real problem with implementing it now is the poor international tournament organization. You can't really account for the fact that if you want to play in the GSL, you really can't spend your time doing much else for the next several months. Fix the current tournament calender setup, and a ranking system will not have ANY problems.
Edit: Actually... The biggest (and only) problem will be if tournaments aren't properly scored, meaning a large tournament isn't assigned enough points and is given the point ranking of a lower level tournament. Also, scaling the point values of different tier tournaments to each other (as well as different results such as Ro64 vs Ro32 vs Ro16 vs Ro8 vs Ro4 vs 2nd vs 1st) can only really be done through trial and error to get the proper scaling. Of course, you can just assign points and just say "well, this guy deserves to be #13 because these were the unbiased values we gave to each tournament and each placing of each tournament".
On August 08 2012 17:07 figq wrote: Include team leagues. They do matter. DongRaeGu was first a team league celebrity.
No way a valid current top 100 doesn't include Coca.
No, they don't. I mean, they do, but it would be unfair to other players on the team who simply don't have the opportunity to play team league matches. DRG is ALWAYS going to play in clutch positions. But whoever always gets chosen to play the first position has the best opportunity to get free points. As a result, his rating is inflated. So they should be eliminated as they bring an unfair way to obtain points to players. DRG started off as a team league celebrity, but he's also now a individual league celebrity as well. The fact is, before he broke out, he doesn't really deserve to be in the top 10 of the individual league rankings with his performance. Sure, the skill was there, but he still needed work. Eventually he DID break out and exploded to the top. I feel that excluding team leagues from the rankings would not hurt him as he still would be considered a top player. It would be unfair to consider someone with no achievements in the individual league a top player when someone like Mvp has been crushing face since before DRG became a team league celebrity and until DRG became an individual league champion. If we do that, the rankings will mean nothing and will be worth as much as biased speculation on who the best player is.
And CoCa doesn't have to be outside the top 100 if the current international tournament system was fixed. All of this ranking business is pretty irrelevant until that is fixed unless you want to split things up into Korean and International (non-Korean) ratings, which is not much different from the Korean and International ELO ratings that we currently have.
On August 08 2012 17:07 figq wrote: Include team leagues. They do matter. DongRaeGu was first a team league celebrity.
No way a valid current top 100 doesn't include Coca.
No, they don't. I mean, they do, but it would be unfair to other players on the team who simply don't have the opportunity to play team league matches
1. Their teams send them to play, because they are good enough to play, so they deserve the recognition for their skill. 2. Same argument applies to all individual tournaments - they only include players who are sent there by their teams, players who are seeded or invited etc.
On August 08 2012 17:07 figq wrote: Include team leagues. They do matter. DongRaeGu was first a team league celebrity.
No way a valid current top 100 doesn't include Coca.
No, they don't. I mean, they do, but it would be unfair to other players on the team who simply don't have the opportunity to play team league matches
1. Their teams send them to play, because they are good enough to play, so they deserve the recognition for their skill. 2. Same argument applies to all individual tournaments - they only include players who are sent there by their teams, players who are seeded or invited etc.
I think he means that in an all-kill format, if you are on a strong team, your team mates may win before you get a chance to be sent out. Or a medium strong player may not be sent out at all. Likewise in a not all-kill format, you may not have to play your game if you are towards the end and your team already won/lost. Thus you will get more or less opportunities to gain points depending on which team you are on, which is not fair when you try to compare individuals.
On August 08 2012 17:07 figq wrote: Include team leagues. They do matter. DongRaeGu was first a team league celebrity.
No way a valid current top 100 doesn't include Coca.
No, they don't. I mean, they do, but it would be unfair to other players on the team who simply don't have the opportunity to play team league matches
1. Their teams send them to play, because they are good enough to play, so they deserve the recognition for their skill. 2. Same argument applies to all individual tournaments - they only include players who are sent there by their teams, players who are seeded or invited etc.
I think he means that in an all-kill format, if you are on a strong team, your team mates may win before you get a chance to be sent out. Or a medium strong player may not be sent out at all. Likewise in a not all-kill format, you may not have to play your game if you are towards the end and your team already won/lost. Thus you will get more or less opportunities to gain points depending on which team you are on, which is not fair when you try to compare individuals.
The main conclusion from this is that making such ranking is futile, because there could be a top 10 world class player who doesn't make it into top 100.
On August 08 2012 17:07 figq wrote: Include team leagues. They do matter. DongRaeGu was first a team league celebrity.
No way a valid current top 100 doesn't include Coca.
No, they don't. I mean, they do, but it would be unfair to other players on the team who simply don't have the opportunity to play team league matches
1. Their teams send them to play, because they are good enough to play, so they deserve the recognition for their skill. 2. Same argument applies to all individual tournaments - they only include players who are sent there by their teams, players who are seeded or invited etc.
I think he means that in an all-kill format, if you are on a strong team, your team mates may win before you get a chance to be sent out. Or a medium strong player may not be sent out at all. Likewise in a not all-kill format, you may not have to play your game if you are towards the end and your team already won/lost. Thus you will get more or less opportunities to gain points depending on which team you are on, which is not fair when you try to compare individuals.
The main conclusion from this is that making such ranking is futile, because there could be a top 10 world class player who doesn't make it into top 100.
The conclusion from this is that if you were to do such a ranking, it would not include team leagues.
The reason making this ranking is futile is completely unrelated to the team leagues.
8 person Dreamhack Invites, AoL.The Gathering, RB Battlegrounds: 1st: 500
Is the only one i think is a bit out. I mean some of those 8 men invitee DH tournaments were stacked. 2011 ones, Valencia and Stockholm had MC IdrA WhiteRa HuK DRG BoxeR Naniwa Hero Thorzain. Deserves a bit more credibility from 500 points xD maybe 1000
And RyLai, no matter how awesome it would be to reorganise everything as you suggested, and even assuming that all the parts involved would manage to agree on something like that:
I don't think there is enough money in sc2 to have essentially every progamer (and their coaches) travelling around to every tier 1 and tier 0 event, even if you organise them in best possible order. Just imagine the number of people (hundreds?) and all the flight tickets around the world, and hotel bills. One can dream though.
On August 08 2012 17:07 figq wrote: Include team leagues. They do matter. DongRaeGu was first a team league celebrity.
No way a valid current top 100 doesn't include Coca.
No, they don't. I mean, they do, but it would be unfair to other players on the team who simply don't have the opportunity to play team league matches
1. Their teams send them to play, because they are good enough to play, so they deserve the recognition for their skill. 2. Same argument applies to all individual tournaments - they only include players who are sent there by their teams, players who are seeded or invited etc.
1) Their skill is recognized with the team league trophy. 2) That's a reason why things need to be changed in the pro scene before we even begin arguments on something as trivial as a ranking system.
On August 08 2012 17:33 figq wrote: The main conclusion from this is that making such ranking is futile, because there could be a top 10 world class player who doesn't make it into top 100.
Yes, but this can be fixed. Not easily. But it can be fixed. The problem is that there is probably nobody willing to do it. You would need well-performing, popular players like Mvp, Flash, MKP, DRG, and Taeja to take a stand and change the current system. We would also need more sponsors to send more players around the world... And that is much more difficult... But it could happen with time and hard work. BoxeR pushed eSports forward, but there are still changes that need to be done.
And including team league results would not all of a sudden put a top 10 world class player into the top 100. And if things change and that happens to still be the case, it's the result of the player simply not being given the chance to perform. If he gets sent to a few events (which if he is THAT good, people WILL notice him and WILL sponsor him), then he WILL quickly rise up in the rankings.
And under the current ELO system by TLPD, CoCa comes in at #12. I don't see what the problem is. DRG is #13. He's in the top 20.
On August 08 2012 17:39 Cascade wrote: And RyLai, no matter how awesome it would be to reorganise everything as you suggested, and even assuming that all the parts involved would manage to agree on something like that:
I don't think there is enough money in sc2 to have essentially every progamer (and their coaches) travelling around to every tier 1 and tier 0 event, even if you organise them in best possible order. Just imagine the number of people (hundreds?) and all the flight tickets around the world, and hotel bills. One can dream though.
Yes, money is the other big issue. But at the same time, international SC2 players are doing this all the time anyway. It only makes a big difference to Koreans.
And I wouldn't fly the coaches. Maybe the managers. We can't baby our players and give them EVERYTHING you know. ^^
Obviously, we can't make large steps such as these over a matter of months, or even years. Decades, maybe. The most we can hope for is over the next few years is that we will be able to do something like this with 16-32 players at a time for Tier 1 and Tier S events (there, I think that name works ^^).
Professional Tennis started off really small. There is no reason Starcraft can't do the same. The Open era started 40-50 years ago, and during that time tournaments got bigger and bigger.
I mean, another solution would be to lengthen each series of events. For example, the US section could be held over the course of 1-2 years, then the European section for the next 1-2 years, then finally the Asian section for the final 1-2 years. It would cut down international travel costs. But the fact is, if we want to move forward, we need to reorganize professional SC2 as we know it. You can't really dump money into a failing system and hope it improves. You need to have a working system, then put money into that. Then the money is spent well, investors get what they want, growth occurs, and everyone is happy. Granted, I don't work with ATP tennis so I don't know how sponsorships and endorsements for tournaments work, but I do know that you get your company's name/logo on the back drop and constantly have your name pop up on the ads that go on during changeovers and breaks. ^^
I honestly don't believe any of this will happen (maybe because of my pessimistic nature), but I do honestly believe these ideas could work if the right people took an interest and tried to apply the concepts behind them. I mean, who WOULDN'T want to be able to objectively compare Koreans and Internationals? The only chances we get are few and far in between. At the very least, I just want professional SC2 to be united. I mean, KeSpa and GOM shouldn't have even had to fight when KeSpa tried to get into the SC2 scene. There should've already been a system where KeSpa could assimilate into. Granted, we did start small and tournaments were slowly popping up, but I think it's time for us to start organizing ourselves. I mean, I've sort of lost interest because things keep changing, new tournaments keep popping up, and because of all the new tournaments there's like a damned new tournament every day and I can't keep track of it all. I don't even bother watching most of the time anymore and just focus on the GSL whenever I happen to see it on the Upcoming Events list. I mean, even the old tournaments are throwing in new crap just to fill the time so they can have viewers. I mean, IPL fight club? NASL KotH? Surely, we can spend our time better than this can't we? But, then again, for the organizations involved, I can't really blame them. They need a consistent source of income. They need to pay their employees and their employees need to pay their bills... Perhaps SC2 boomed faster than it should've? I don't even know...
Oh shit... I forgot to include TSL in my list of tournaments... That too should be considered a major event. But the thing I truly admire about the TSL is that it only comes once a year, around the same time each year, and that never changes. As a result, they can build hype, and everyone looks forward to it.
And for the record, GOAT talks are silly, but hell, if everything got cleaned up and achievements got standardized (so to say), we could start those obnoxious GOAT talks. Obviously Mvp would be a pick for this sort of "Pre-Open Era" with 4 GSLs. But, when he got 2 of those when GSLs happened every month (1 of which being a weird International GSL tournament), 1 before the GSL during the Open Era of GSL, and 1 when the GSL became truly hard to win and happens like once ever 2-3 months? It becomes hard to make a call. >.> Especially if someone a decade or two later wins 5 GSLs when they only come up once a year. Of course, Mvp will be the greatest of OUR time (until the Queen buff hit).
While I agree that GSL is the most prestigious tournament, I am not sure that it is worth two or three times as much as an MLG or Dreamhack championship. Ranking tournaments based on something other than perceived difficulty, is very important for the credibility of the system. Non-objective factors that I can think of right now include, size of the player pool, percentage of players seeded, prize pool, number of matches required to win. Unfortunately, I can't think of any way to rank tournaments based on the skill of the players in the tournaments without already having a system to rank the skill of players. (I guess something could be done with ELO)
One issue that is particularly striking to me is the low point values given for WCS nationals. GOMtv is running WCS Korea instead of GSL code S. Objectively, WCS Korea should be ranked like any other WCS National tournament, but it is basically Code S in a double elimination format.
I will think about this some more, leaving to watch WCS korea now.
On August 08 2012 17:39 Cascade wrote: And RyLai, no matter how awesome it would be to reorganise everything as you suggested, and even assuming that all the parts involved would manage to agree on something like that:
I don't think there is enough money in sc2 to have essentially every progamer (and their coaches) travelling around to every tier 1 and tier 0 event, even if you organise them in best possible order. Just imagine the number of people (hundreds?) and all the flight tickets around the world, and hotel bills. One can dream though.
I mean, another solution would be to lengthen each series of events. For example, the US section could be held over the course of 1-2 years, then the European section for the next 1-2 years, then finally the Asian section for the final 1-2 years. It would cut down international travel costs. But the fact is, if we want to move forward, we need to reorganize professional SC2 as we know it.
I honestly don't believe any of this will happen (maybe because of my pessimistic nature), but I do honestly believe these ideas could work if the right people took an interest and tried to apply the concepts behind them. I mean, who WOULDN'T want to be able to objectively compare Koreans and Internationals? The only chances we get are few and far in between. At the very least, I just want professional SC2 to be united. I mean, KeSpa and GOM shouldn't have even had to fight when KeSpa tried to get into the SC2 scene. There should've already been a system where KeSpa could assimilate into. Granted, we did start small and tournaments were slowly popping up, but I think it's time for us to start organizing ourselves.
I mean, I've sort of lost interest because things keep changing, new tournaments keep popping up, and because of all the new tournaments there's like a damned new tournament every day and I can't keep track of it all. I don't even bother watching most of the time anymore and just focus on the GSL whenever I happen to see it on the Upcoming Events list. I mean, even the old tournaments are throwing in new crap just to fill the time so they can have viewers. I mean, IPL fight club? NASL KotH? Surely, we can spend our time better than this can't we? But, then again, for the organizations involved, I can't really blame them. They need a consistent source of income. They need to pay their employees and their employees need to pay their bills... Perhaps SC2 boomed faster than it should've? I don't even know...
I really, really dislike this idea, this would basically mean the SC2 tournament scene goes through a 6 year cycle which is longer than a lot than most players careers last just now. I guess you intended it as an exaggerated concept compared to your previous idea.
I agree however that the big organisations in the scene need to come together to organise tournaments, formats, scheduling and such. KeSPA, GOM, MLG, IPL, NASL, IEM, Dreamhack and any other tournament that wants to run offline events with a significant prize pool ($5000 and above?) should have to agree upon dates, ensuring that rules are standardised so that no confusion arises at different tournaments (MLG Rules anyone?). They need to agree on a schedule so that no major tournaments are conflicting, this is good for teams, players, sponsors and for the events, who will be able to get the players they want and less division between the tournaments (It's pretty tiring getting up at 10ish for a european tournament, then staying up til 5am to watch MLG, then potentially korean leagues after that )
I don't have so much of an issue with online fluff tournaments, most are casted from replays anyway which are easier for the players to arrange. I feel MLG took a great initiative with restructuring their league system, especially with the funding of players to travel to the Arenas.
If GOM and Kespa lay down the foundations of the GSL/OSL that last 3 months, with a months gap in between. This will allow non-Korean tournaments to choose dates better suited to those competing in the Leagues to be able to fly out without comprimising their performance in the leagues. What you also have to consider is that the Proleague will most likely be restructured to incorporate the ESF teams, which will probably be more important to Korean teams than flying players to foreign tournaments. We could see B-teamers being given the opporunity to gain much needed experiance at MLGs and IPLs etc.
I think we should do it similar to Tennis, where we have 'majors' which are all ranked the same, and then other smaller events, with the largest of these smaller events being half the points score of a major. I think majors are GSL, Dreamhack, MLG, IPL, IEM finals and NASL.
I think the lack of foreigners in GSL makes up for lack of Koreans in NASL so its more or less fair.
On August 08 2012 21:09 Micket wrote: I think we should do it similar to Tennis, where we have 'majors' which are all ranked the same, and then other smaller events, with the largest of these smaller events being half the points score of a major. I think majors are GSL, Dreamhack, MLG, IPL, IEM finals and NASL.
I think the lack of foreigners in GSL makes up for lack of Koreans in NASL so its more or less fair.
except GSL is about 10 times as hard as any of the other tournaments mentioned with it so it'd be kind of unfair to put them together. Its nothing like tennis tbh.
On August 08 2012 21:09 Micket wrote: I think we should do it similar to Tennis, where we have 'majors' which are all ranked the same, and then other smaller events, with the largest of these smaller events being half the points score of a major. I think majors are GSL, Dreamhack, MLG, IPL, IEM finals and NASL.
I think the lack of foreigners in GSL makes up for lack of Koreans in NASL so its more or less fair.
except GSL is about 10 times as hard as any of the other tournaments mentioned with it so it'd be kind of unfair to put them together. Its nothing like tennis tbh.
And the lack of koreans in the NASL is because they don't like the organisation
On August 08 2012 17:39 Cascade wrote: And RyLai, no matter how awesome it would be to reorganise everything as you suggested, and even assuming that all the parts involved would manage to agree on something like that:
I don't think there is enough money in sc2 to have essentially every progamer (and their coaches) travelling around to every tier 1 and tier 0 event, even if you organise them in best possible order. Just imagine the number of people (hundreds?) and all the flight tickets around the world, and hotel bills. One can dream though.
I mean, another solution would be to lengthen each series of events. For example, the US section could be held over the course of 1-2 years, then the European section for the next 1-2 years, then finally the Asian section for the final 1-2 years. It would cut down international travel costs. But the fact is, if we want to move forward, we need to reorganize professional SC2 as we know it.
I honestly don't believe any of this will happen (maybe because of my pessimistic nature), but I do honestly believe these ideas could work if the right people took an interest and tried to apply the concepts behind them. I mean, who WOULDN'T want to be able to objectively compare Koreans and Internationals? The only chances we get are few and far in between. At the very least, I just want professional SC2 to be united. I mean, KeSpa and GOM shouldn't have even had to fight when KeSpa tried to get into the SC2 scene. There should've already been a system where KeSpa could assimilate into. Granted, we did start small and tournaments were slowly popping up, but I think it's time for us to start organizing ourselves.
I mean, I've sort of lost interest because things keep changing, new tournaments keep popping up, and because of all the new tournaments there's like a damned new tournament every day and I can't keep track of it all. I don't even bother watching most of the time anymore and just focus on the GSL whenever I happen to see it on the Upcoming Events list. I mean, even the old tournaments are throwing in new crap just to fill the time so they can have viewers. I mean, IPL fight club? NASL KotH? Surely, we can spend our time better than this can't we? But, then again, for the organizations involved, I can't really blame them. They need a consistent source of income. They need to pay their employees and their employees need to pay their bills... Perhaps SC2 boomed faster than it should've? I don't even know...
I really, really dislike this idea, this would basically mean the SC2 tournament scene goes through a 6 year cycle which is longer than a lot than most players careers last just now. I guess you intended it as an exaggerated concept compared to your previous idea.
I agree however that the big organisations in the scene need to come together to organise tournaments, formats, scheduling and such. KeSPA, GOM, MLG, IPL, NASL, IEM, Dreamhack and any other tournament that wants to run offline events with a significant prize pool ($5000 and above?) should have to agree upon dates, ensuring that rules are standardised so that no confusion arises at different tournaments (MLG Rules anyone?). They need to agree on a schedule so that no major tournaments are conflicting, this is good for teams, players, sponsors and for the events, who will be able to get the players they want and less division between the tournaments (It's pretty tiring getting up at 10ish for a european tournament, then staying up til 5am to watch MLG, then potentially korean leagues after that )
I don't have so much of an issue with online fluff tournaments, most are casted from replays anyway which are easier for the players to arrange. I feel MLG took a great initiative with restructuring their league system, especially with the funding of players to travel to the Arenas.
If GOM and Kespa lay down the foundations of the GSL/OSL that last 3 months, with a months gap in between. This will allow non-Korean tournaments to choose dates better suited to those competing in the Leagues to be able to fly out without comprimising their performance in the leagues. What you also have to consider is that the Proleague will most likely be restructured to incorporate the ESF teams, which will probably be more important to Korean teams than flying players to foreign tournaments. We could see B-teamers being given the opporunity to gain much needed experiance at MLGs and IPLs etc.
The long SC2 season was a possible solution to the money issue due to flight costs. If we reduce the amount of flights required, then we can reduce the costs to the players and teams, making a truly international SC2 World Tour schedule more viable. Sadly, I think that a 3-6 year season is bad as well... If any changes were to be made to the season, I'd think shortening it would be better, but that would make travel costs a bit more hectic, make traveling more exhausting, and completely squeeze out anything smaller than MLGs and IEMs (meaning only MLG, IEM, GSL, NASL, TSL, and IPL; I feel like I'm missing some major tournaments again). It's nice that MLG will pay for travel, but I doubt GOM will do the same. These are the guys who boycotted the NASL for both travel and hotel expenses. GOM however DOES have the GOM house, so living is taken care of. And I double the NASL will do it either since they wouldn't put out in order to keep the Koreans (probably more that they couldn't really afford it).
And I feel that spamming major tournaments sort of lessens the prestige of winning it. The Olympics are rather huge because you only get one chance every 4 years to win an Olympic gold in any event. The majors in tennis are huge because there are only 4 every year as opposed to ~10 Master's Series and like 15-20 ATP Tour events. I feel like there should be fewer of the larger tournaments, since these are what players should really be practicing and playing for, and the uncountable tournaments should be the lower level tournaments where players earn points, money, and set themselves up mentally for the major tournaments.
But again, the international tournament calender can be in any format so long as the players and teams agree to it and as a result every team sends their players to a majority of the events. What we should be really aiming for is to send the best of the world (Koreans AND Internationals) to a few tournaments every year, and for the rest of the year have them compete around the world as they see fit (also aiming to try and minimize fatigue from air travel). At the very least we should be aiming a few times a year to have everyone compete in one tournament so that we can see how everyone stacks up against one another and have an amazing tournament. I mean, when was the last time when we had something as crazy as the GSL Super Tournament? EVERYONE in Korea was in on it, and it was AMAZING to see everyone go at it. Sure, it ended in sort of a blowout, but that will happen sometimes.
At this point, the only place to watch the BEST SC2 is at the GSL (or the Korean WCS selection tournament). And that's kind of unfortunate. The NASL used to be sick when the Koreans participated (even without the absolute top players of that time). I mean, we say that SC2 has become international, but there's still the Korean TLPD and the International TLPD.
As for what should be majors, only the GSL, NASL, and IPL have prize pools of 100k or higher (though OSL comes close at 90k).
After that comes ESWC, Blizzcon, Dreamhack Winter, Blizzard Cup, IEM World Championship, and MLG Championships with over 50k in prize money.
Then there's everything else, that's 20k to about 30k. Homestory Cup, ASUS ROG, IEM, MLG Arena, Dreamhack (aside from Winter), an TSL.
Then we have a bunch of random online stuff.
I feel like ALL of the top players should attend the GSL, NASL, IPL, and OSL. After that, they can attend the other events. However, due to qualifying requirements, it'd probably be best to stick to one tournament chain (IEM or MLG), then supplement that with Dreamhack, Homestory Cup, or ASUS ROG. I feel that players should be able to choose their schedules outside of majors (which should be mandatory). Tennis forces players to play a minimum number of events, which is kind of sad.
On August 08 2012 21:09 Micket wrote: I think we should do it similar to Tennis, where we have 'majors' which are all ranked the same, and then other smaller events, with the largest of these smaller events being half the points score of a major. I think majors are GSL, Dreamhack, MLG, IPL, IEM finals and NASL.
I think the lack of foreigners in GSL makes up for lack of Koreans in NASL so its more or less fair.
We can't do it like tennis until we standardize the schedule and make it so that everyone has a near equal opportunity to play every tournament. As it is, the Koreans are sitting in Korea playing the GSL and nothing else. The lack of foreigners in the GSL is a result of the fact that it's not really worth the time and effort for most people to even attempt qualifying. This results in, as many have said before, the GSL being the most difficult tournament to win because it's the ONLY tournament that the top Koreans are really concerned about, raising its difficulty level and lowering the difficulty of others. If you want to debate that, look at how the top players of the international scene only go mid-way through the GSL. HuK, Taeja, HerO, PuMa - these are players who more often than not are basically owning the international scene and have all got wrecked in Korea. Yes, they did post up Ro8 results, which are damn good, but we haven't had a semifinal appearance since Jinro. Stephano looked like AMAZING last year, and looked ordinary against the Koreans in the Blizzard Cup.
If we can equalize the player base in every "major" tournament, then we can consider all "majors" to be equal, but as it is, foreigner events are much easier than the GSL, so the GSL deserves a lot more weight in any form of ranking system.
Just a question. You say that you did this because you love stats and did it for fun. So why did you post it on TL? Are you looking for feedback on the system and hoping it will become actually useful in the future? I think it's way too subjective to be of any use.
I agree with a lot of people here that you need to split it into KR and non-KR. Right now this is like opterown's World Champion thing, which is just for fun.
On August 08 2012 13:18 CosmicSpiral wrote: I think you should just start from scratch, figure out what you want your system to do, and build it around those goals.
Feedback needs to be considered, not necessarily accepted.
Listen, you can think that it needs a change, but don't put words in my mouth. I wanted to make a ranking system based on Tennis, with fixed points for the events based on prestige, and that is what I did. "figure out what I want my system to do?" I can see myself in the short future trying a different variety using the TLPD ELO ranks to determine strength for tournaments, but don't say that I failed what I wanted to do.
You are making a ranking system for SC2. Why don't you make a ranking system specifically designed for SC2? The tennis scene is very different so the same principles cannot be applied. It's the same reason why a certain chess-based ranking system has failed to reliably rank SC2 players.
I bring up the question because there are so many scenarios and problems that are simply not addressed.
- What about showmatches? - What about weekly/monthly tournaments? - What about invitationals that are not premier events? - What about qualifiers? - Should team leagues have a different ranking? Should they be included at all? - How do you account for close set/serieslosses? - How do you account for individual runs against top competition that does not result in a high tournament placing? - How do you account for the overall player strength of an individual tournament? - How do you account for sickness, fatigue, injuries, etc. that result in poor placements? - How do you account for matchup strength? - How do you account for monetary issues that limit attendance at premier tournaments? - How do you account for deviations that the system will interpret incorrectly? - What are the obvious problems that the system will create, and how should they be dealt with?
Hmm, I agree with you in general, that the point system has some big flaws and that it probably will not get used widely.
But I think you are overdoing it a bit when it comes to shoot him down. He has made a fair try to convert the tennis point system to starcraft, I think that deserves more than this kind of replies.
Of course a completely new system for sc2, that perfectly addresses every little detail you can think of would be really sweet, but it doesn't seem to be that easy to design. Which is why we have many different rankings in parallel, like the ELO from TLPD, ladder rankings, sc2earnings, power rankings etc. Unless you have some great idea to solve this problem, I don't feel it is fair to be too aggressive to people that works on new ways to approach the problem.
For the list, you are just doing an as long list as possible for the sake of it... Do you for example really need to take teh argument "you need to decide how many, if any, points to give to all the smaller events." and plit it up into the first four points? Why would you treat a close loss any different? Why would you treat an injured player any different? Why would you treat different matchup differently (do you treat left handed players differently in tennis?)? Anyway, I just feel that you add things to the list to make the list long rather than actually trying to say something.
But again, I agree with your general standpoint, and imo, the one big insurmountable problem is your third last point (that you list together with all the random ones...). ie that all the best players don't play in all the top tournaments. Which is the one reason that this point system isn't really going to work that well.
The second largest issue is that people would have to agree on how to distribute the points among the tournaments, but I think that could be done decently if it were not for the main problem. (Which would be your point 1 to 4 more or less. )
The rest of the details you bring up I think could be sorted out, or ignored. If the main problem was not there.
edit: oops, do --> don't >_>
If I thought Metalteeth was stupid I would simply dry witty insults like those stuck-up people on old British comedies. He has put in a lot of work into this project; he should be proud that he has created a working model in the first place. But if he is serious about creating an accurate model then there are a lot of obvious issues he has to address. The first and most important one is that he is trying to apply the ATP point system to a different scene. Several people before me have already mentioned why this will not work. If this is just for fun then it doesn't matter where the results agree with public perception.
That is why I suggested that he should start from nothing and build up. Many of the rankings you mentioned have certain problems that make them unreliable and they all stem from their origins. Elo was created for chess, specifically for the way the chess scene worked; when applied to SC2 it is not nearly as reliable, Hell it does not even work smoothly in competitive chess anymore (engine abuse is pretty common when Elo actually matters).
Statements 1-5 are a convoluted way of questioning the entire scope of the system. The intuitive way of judging skill is a holistic, coherent process. You look at the player's achievements. You learn what tournaments he has played, what tournaments he has not played, and what tournaments he cannot play. You look at periods of strength and weakness, how often they occur, and attempt to figure out patterns. You watch his games, you learn his approach, you make judgments on the strength of his playstyle. You look at the competition he faced along the way; you judge the competition by the same standards as mentioned. Obviously there is no solid foundation to start off from, it's all kind of circular.
But the point is you look at everything to make a proper judgment. Team leagues, qualifiers, weeklies and monthlies all provide valuable information. They are not necessarily prestigious or considered important, but that is irrelevant. A proper system that attempts to statistically analyze skill uses all available information at its disposal. So if a system only uses major and premier tournaments then it already missing crucial data, and so it cannot be taken seriously.
Statements 6-10 question the lack of consideration towards factors within the tournament. Tournament placings are a combination of skill and luck. A bad bracket can be the difference between 1st place and 4th place. Avoiding your weakest matchup can result in a tournament win (hello Jjakji!). Barely losing a series because of one mistake is important, especially when the loser was a heavy underdog; this is one reason why Elo can be somewhat accurate. I think it's obvious that if a certain player loses a match because he has to throw up from food poisoning (MLG Providence), his loss should hurt his ranking less than if he lost under normal circumstances. However if his condition is chronic (Mvp or sC) then it is accepted as an unfortunate part of his "skill set", so to speak.
Statements 11-13 are just general problems. Everyone cannot attend the same tournament every single time. No system can perfectly analyze every little thing. No system can perfectly rank skill within a player pool. But any well-designed system can overcome 11 as well as anticipate when situations 12 and 13 will occur. I don't mind Elo that much since I know when it will skew a player's ranking and I can just adjust it in my own head.
I don't like the ATP system so I think you can guess my stance on that. There must be some way to rank individual major/premier tournaments through a combination of general skill level and prestige. Prestige alone means nothing but how it affects a player's mindset (and subsequently his performance) cannot be denied.
Well the main problem will always be there. I don't see a way around it.
I think it's like baseball, you have BA/OBP/SLG, while each one doesn't give you a good idea what the player is about, the combination of the 3 tells a pretty good picture.
Maybe for SC2, we can do something like winrate/elo/earning
On August 08 2012 09:03 Cascade wrote: Nice to see this progress.
I would still prefer to keep 100% from a tournament until that tournament is played again, but this is also fine.
Some comments on this vs ELO: 1) ELO tries to estimate the skill of players right now. This measures recent achievements. These two are quite different, and you cannot really say that one is better than the other, as they measure different things. Taeja is a good example. He tears up skilled people right and left in smaller tournaments, but has not really placed very high in many big individual tournaments. Thus ELO ranks him high, this ranking not as much. On the other hand, a player like MC really knows how to put himself together and win (or place high in) big events, but can drop games to lower ranked players in less important occasions. Thus MC gets a very high score for recent achievements, but maybe not as high in ELO.
2) Korea vs foreigners. The two communities are a bit separated, so it is hard to compare between the two. ELO solves this by not comparing at all. Thus two separate rankings. This point ranking tries to compare them, but runs into the problem of balancing the points of GSL vs other tournaments, and the problem that some players participate in more tournaments than others. Compare to tennis where all top players play in all big tournaments. None of the two solutions are very impressive.
3) The two systems have different flaws, so it is useful to look at both. ELO is very good because it has little subjective bias. A win is a win, no matter the context. Some bias in which games you include, but probably not a huge deal. This point system, as has been pointed out a lot of times, is very subjective in terms of how the points are distributed. The OP has done a good job of trying to find a distribution that most people can accept, but even with the full support of the community, you cannot get away from the fact that the points are distributed in a subjective manner.
The "a win is a win" in ELO is also one of it's main problems. beating MC in the GSL finals is worth as much as beating him in the round of 8 in a weekly tournament. Many would argue that winning in the GSL finals is a much larger indicator of skill than beating them in a weekly tournament. The argument would be that people prepare for the GSL, and will always bring his very best game, while in a weekly, people may not play at their very best in every game, due to various circumstances. This is better taken into account in a point system, where big tournaments count more.
So putting them together, ELO can be seen more as a "current raw skill" in some sense, while the point system is more "recent achievements". Also note that ELO tries to measure skill right now, while the point system is counting over the last 6 months, and will lag behind a bit. So a new player coming out of nowhere will jump up in ELO very fast, but will not reach his peak in the point system until after about half a year after his first win. Thus the point system measures consistency a bit as well.
Best picture can be seen by looking at both. High ELO but low point ranking means either a rising top player, or a skilled player that is not very clutch in big tournaments. Low ELO but high point ranking is either a declining top player, or a good "tournament player".
you really have no clue what you're talking about..
both measure the same thing, current skill of the player.. making an ELO or similar system (glicko etc.) for world ranking would work fairly well now since there are enough matches between foreigners and koreans. The match is a match thing is simply not true either, ELO can easily be configured to have different K-values for different types of matches/tournaments...
Overall ELO is just FAR superior to the method given in this thread and combining results is not better.. it's just worse than using ELO alone. TLPD is just not the only way to make an ELO system, it could be vastly improved.. it is in the right direction though
On August 08 2012 09:03 Cascade wrote: Nice to see this progress.
I would still prefer to keep 100% from a tournament until that tournament is played again, but this is also fine.
Some comments on this vs ELO: 1) ELO tries to estimate the skill of players right now. This measures recent achievements. These two are quite different, and you cannot really say that one is better than the other, as they measure different things. Taeja is a good example. He tears up skilled people right and left in smaller tournaments, but has not really placed very high in many big individual tournaments. Thus ELO ranks him high, this ranking not as much. On the other hand, a player like MC really knows how to put himself together and win (or place high in) big events, but can drop games to lower ranked players in less important occasions. Thus MC gets a very high score for recent achievements, but maybe not as high in ELO.
2) Korea vs foreigners. The two communities are a bit separated, so it is hard to compare between the two. ELO solves this by not comparing at all. Thus two separate rankings. This point ranking tries to compare them, but runs into the problem of balancing the points of GSL vs other tournaments, and the problem that some players participate in more tournaments than others. Compare to tennis where all top players play in all big tournaments. None of the two solutions are very impressive.
3) The two systems have different flaws, so it is useful to look at both. ELO is very good because it has little subjective bias. A win is a win, no matter the context. Some bias in which games you include, but probably not a huge deal. This point system, as has been pointed out a lot of times, is very subjective in terms of how the points are distributed. The OP has done a good job of trying to find a distribution that most people can accept, but even with the full support of the community, you cannot get away from the fact that the points are distributed in a subjective manner.
The "a win is a win" in ELO is also one of it's main problems. beating MC in the GSL finals is worth as much as beating him in the round of 8 in a weekly tournament. Many would argue that winning in the GSL finals is a much larger indicator of skill than beating them in a weekly tournament. The argument would be that people prepare for the GSL, and will always bring his very best game, while in a weekly, people may not play at their very best in every game, due to various circumstances. This is better taken into account in a point system, where big tournaments count more.
So putting them together, ELO can be seen more as a "current raw skill" in some sense, while the point system is more "recent achievements". Also note that ELO tries to measure skill right now, while the point system is counting over the last 6 months, and will lag behind a bit. So a new player coming out of nowhere will jump up in ELO very fast, but will not reach his peak in the point system until after about half a year after his first win. Thus the point system measures consistency a bit as well.
Best picture can be seen by looking at both. High ELO but low point ranking means either a rising top player, or a skilled player that is not very clutch in big tournaments. Low ELO but high point ranking is either a declining top player, or a good "tournament player".
you really have no clue what you're talking about..
both measure the same thing, current skill of the player.. making an ELO or similar system (glicko etc.) for world ranking would work fairly well now since there are enough matches between foreigners and koreans. The match is a match thing is simply not true either, ELO can easily be configured to have different K-values for different types of matches/tournaments...
Overall ELO is just FAR superior to the method given in this thread and combining results is not better.. it's just worse than using ELO alone. TLPD is just not the only way to make an ELO system, it could be vastly improved.. it is in the right direction though
I like the idea of this, but there a few issues that would have to be addressed properly for the list to have any credibility.
Mainly, weightings. The weightings between events is something that no one person can determine without many others disagreeing. For between tournament weights to be credible, these would have to be determined in the very least by a committee of trusted authorities who can properly estimate just how more difficult certain tournaments are over others, quantitatively.
Simply weighting events from different leagues however is now good enough, because the same event in a series from the same league is likely harder at some times than at others (think early seasons of GSL or MLG to today). Therefore, every event would have to be weighted INDIVIDUALLY for the list to remain credible. To further complicate this same issue, the difficulty of a particular event is in large part determined by who else is competing in that same event. Therefore, the best way to determine the proper weights of finishing in different tournaments is going to have to rely on some metric of the skill of the players registered for that event. This is problematic since quantifying player skill is essentially the end goal of this exercise to begin with!
Lastly, weights for finishing in different places within a given tournament have to be given a similar treatment to the two points above. First place in the GSL Finals isn't universally difficult between seasons, and the difference between 1st and 2nd, or 2nd and 3rd is not universally equal either. What's more, you could argue that within a given season, the weighting of getting 1st place may change depending on who is left in the tournament at any given point in time. If by some "fluke" of chance most of your best players get eliminated early, the suddenly getting 1st place isn't as difficult an achievement.
I think these issues, each in their own way, would need to be addressed for a system of this nature to be a reliable indicator of true ranking and skill. At the end of the day, the truth of the matter is we all HATE the BCS system because it is convoluted and not obvious to the lay person watching games. The sort of measures that a system like this would require to be a reliable and valid system is plagued by that same fault. :\
So it seems that a lot of people think that the tournament weightings are subjective. Which is true. But people want a method of ranking the tournaments, not by a (seemingly) random weighting, but by measuring how strong the players there are. So I'm trying something for FUTURE tournaments. I will not be changing how past tournaments are weighted (part of the problem being how do you weigh players from before a tournament? TLPD is already updated, and if I start back rating tournaments, eventually I get no weight to determine the first tournament), only testing out rating future tournaments. I will definitely take ideas, but right now I'm testing out various ways of combining the mean and median of a player pool to determine a "weight" for the tournament.
Inside the spoiler tag, you will see data on various upcoming tournaments, and ways I'm trying to weight them.
Ok, first one I'm taking a look at is TSL4. 32 players. Using the current up to date data. These are the player points of each player in the tournament:
Giving a mean of 1268.36, and a median of 720.42. Now, the first thing I thought of was just adding together the mean and median. This ensures that a single high ranked player won't overpower a tournament. The problem though is that this way means that a 4 person invite with the top 4 players in the world for a small prize pool would be worth more than a GSL or MLG could possibly be. So there has to be some sort of equalizing factor. So then I thought....prize money. Take TSL4. 1st place gets $15,000. So a possibly way is to average the prize pool (in US dollars) with the mean+median of a tournament, then divide that average by a number, lets try 5. For TSL:, that means that winning TSL4 would net a player 3397.75 points, with dwindling results further down (which would be dependent on the structure of the tournament).
Ok, this seems decent, let's try another upcoming tournament. IEM has a tour stop soon, with a 1st place prize of $6,500. Many would consider this IEM stop a lower player pool, so with the same formula, what does winning IEM Cologne give? 1728.31. Ok, this seems like a decently weighted result.
Try this on another group, the next MLG. Because Open play is a strange beast (and we don't know who will be there fully), I will only use those from pool play. Results: 5878.43 for winning MLS Summer. Wow, that is quite a lot. A GSL would be a lot more though, thanks to the larger prize pool for winning a GSL. What do you guys think about this? I'm thinking the equation needs work, but it's a decent start.
To recap: Possible way to weight the WINNER of a future tournament: (Mean of players' ratings+Median of players' ratings+Prize pool in US Dollars)/5.
Alright, trying another one. Let's imagine that GSL S4 was about to start, with the same players as GSL S3, along with the same prize pool. Winner of that tournament would get: 9581.27. Ok, that is a massive point hall. If that sticks, then we will get massive point inflation as tournaments run. I want to balance it such that points won't substantially increase with time to a stupid amount.
A solution to this? Divide everything in half. Instead of dividing by 5, try 10. That makes TSL4 1698.88, IEM Cologne 864.155, MLG Summer 2939.22, and GSL 4790.64. Those are better numbers, much better. Hmm. Thinking about how this would work. A tournament with a large prize pool relative to the player quality (see some of the WCS Nationals without as much quality players, like New Zealand or Colombia) would be weighted too high, whereas a tournament with a good pool but little prize would underrate the tournament. However, this does provide a decent way of measuring the importance of the tournament. Plus, with the largest tournament (GSL) not super large, there shouldn't be point inflation, barring a massive influx of money into SC2. And if that happens, the formula can always be adjusted.
On August 09 2012 12:20 Metalteeth wrote: So it seems that a lot of people think that the tournament weightings are subjective. Which is true. But people want a method of ranking the tournaments, not by a (seemingly) random weighting, but by measuring how strong the players there are. So I'm trying something for FUTURE tournaments. I will not be changing how past tournaments are weighted (part of the problem being how do you weigh players from before a tournament? TLPD is already updated, and if I start back rating tournaments, eventually I get no weight to determine the first tournament), only testing out rating future tournaments. I will definitely take ideas, but right now I'm testing out various ways of combining the mean and median of a player pool to determine a "weight" for the tournament.
Inside the spoiler tag, you will see data on various upcoming tournaments, and ways I'm trying to weight them.
Ok, first one I'm taking a look at is TSL4. 32 players. Using the current up to date data. These are the player points of each player in the tournament:
Giving a mean of 1268.36, and a median of 720.42. Now, the first thing I thought of was just adding together the mean and median. This ensures that a single high ranked player won't overpower a tournament. The problem though is that this way means that a 4 person invite with the top 4 players in the world for a small prize pool would be worth more than a GSL or MLG could possibly be. So there has to be some sort of equalizing factor. So then I thought....prize money. Take TSL4. 1st place gets $15,000. So a possibly way is to average the prize pool (in US dollars) with the mean+median of a tournament, then divide that average by a number, lets try 5. For TSL:, that means that winning TSL4 would net a player 3397.75 points, with dwindling results further down (which would be dependent on the structure of the tournament).
Ok, this seems decent, let's try another upcoming tournament. IEM has a tour stop soon, with a 1st place prize of $6,500. Many would consider this IEM stop a lower player pool, so with the same formula, what does winning IEM Cologne give? 1728.31. Ok, this seems like a decently weighted result.
Try this on another group, the next MLG. Because Open play is a strange beast (and we don't know who will be there fully), I will only use those from pool play. Results: 5878.43 for winning MLS Summer. Wow, that is quite a lot. A GSL would be a lot more though, thanks to the larger prize pool for winning a GSL. What do you guys think about this? I'm thinking the equation needs work, but it's a decent start.
To recap: Possible way to weight the WINNER of a future tournament: (Mean of players' ratings+Median of players' ratings+Prize pool in US Dollars)/5.
Alright, trying another one. Let's imagine that GSL S4 was about to start, with the same players as GSL S3, along with the same prize pool. Winner of that tournament would get: 9581.27. Ok, that is a massive point hall. If that sticks, then we will get massive point inflation as tournaments run. I want to balance it such that points won't substantially increase with time to a stupid amount.
A solution to this? Divide everything in half. Instead of dividing by 5, try 10. That makes TSL4 1698.88, IEM Cologne 864.155, MLG Summer 2939.22, and GSL 4790.64. Those are better numbers, much better. Hmm. Thinking about how this would work. A tournament with a large prize pool relative to the player quality (see some of the WCS Nationals without as much quality players, like New Zealand or Colombia) would be weighted too high, whereas a tournament with a good pool but little prize would underrate the tournament. However, this does provide a decent way of measuring the importance of the tournament. Plus, with the largest tournament (GSL) not super large, there shouldn't be point inflation, barring a massive influx of money into SC2. And if that happens, the formula can always be adjusted.
ugh you're acting so stupid. You're basically trying to invent ELO again... You want tournaments that are automatically ranked by the quality of players that are attending?? Ie you want a player that beats difficult opponents on average to gain more points than beating weaker opponents? That is what ELO does... difference being that with ELO you can also lose points, which just happens to solve the problem of certain players (MC?) attending way more tournaments than others...
Stop trying to reinvent the wheel, points systems suck for any sport which doesn't have rigourous rules like tennis.. Even in tennis ELO like systems are used at lower level.. only the ATP pro's are ranked by the point system..
On August 09 2012 12:20 Metalteeth wrote: So it seems that a lot of people think that the tournament weightings are subjective. Which is true. But people want a method of ranking the tournaments, not by a (seemingly) random weighting, but by measuring how strong the players there are. So I'm trying something for FUTURE tournaments. I will not be changing how past tournaments are weighted (part of the problem being how do you weigh players from before a tournament? TLPD is already updated, and if I start back rating tournaments, eventually I get no weight to determine the first tournament), only testing out rating future tournaments. I will definitely take ideas, but right now I'm testing out various ways of combining the mean and median of a player pool to determine a "weight" for the tournament.
Inside the spoiler tag, you will see data on various upcoming tournaments, and ways I'm trying to weight them.
Ok, first one I'm taking a look at is TSL4. 32 players. Using the current up to date data. These are the player points of each player in the tournament:
Giving a mean of 1268.36, and a median of 720.42. Now, the first thing I thought of was just adding together the mean and median. This ensures that a single high ranked player won't overpower a tournament. The problem though is that this way means that a 4 person invite with the top 4 players in the world for a small prize pool would be worth more than a GSL or MLG could possibly be. So there has to be some sort of equalizing factor. So then I thought....prize money. Take TSL4. 1st place gets $15,000. So a possibly way is to average the prize pool (in US dollars) with the mean+median of a tournament, then divide that average by a number, lets try 5. For TSL:, that means that winning TSL4 would net a player 3397.75 points, with dwindling results further down (which would be dependent on the structure of the tournament).
Ok, this seems decent, let's try another upcoming tournament. IEM has a tour stop soon, with a 1st place prize of $6,500. Many would consider this IEM stop a lower player pool, so with the same formula, what does winning IEM Cologne give? 1728.31. Ok, this seems like a decently weighted result.
Try this on another group, the next MLG. Because Open play is a strange beast (and we don't know who will be there fully), I will only use those from pool play. Results: 5878.43 for winning MLS Summer. Wow, that is quite a lot. A GSL would be a lot more though, thanks to the larger prize pool for winning a GSL. What do you guys think about this? I'm thinking the equation needs work, but it's a decent start.
To recap: Possible way to weight the WINNER of a future tournament: (Mean of players' ratings+Median of players' ratings+Prize pool in US Dollars)/5.
Alright, trying another one. Let's imagine that GSL S4 was about to start, with the same players as GSL S3, along with the same prize pool. Winner of that tournament would get: 9581.27. Ok, that is a massive point hall. If that sticks, then we will get massive point inflation as tournaments run. I want to balance it such that points won't substantially increase with time to a stupid amount.
A solution to this? Divide everything in half. Instead of dividing by 5, try 10. That makes TSL4 1698.88, IEM Cologne 864.155, MLG Summer 2939.22, and GSL 4790.64. Those are better numbers, much better. Hmm. Thinking about how this would work. A tournament with a large prize pool relative to the player quality (see some of the WCS Nationals without as much quality players, like New Zealand or Colombia) would be weighted too high, whereas a tournament with a good pool but little prize would underrate the tournament. However, this does provide a decent way of measuring the importance of the tournament. Plus, with the largest tournament (GSL) not super large, there shouldn't be point inflation, barring a massive influx of money into SC2. And if that happens, the formula can always be adjusted.
I understand you are trying to find some way in which the prestige of a tournament enhances/lessens the performance of a player within it. However, the prize pool of a tournament is independent of a player's performance within said tournament.
I don't think you will find many people who believe IEM Cologne is half as prestigious as TSL4. Being live is important for a tournament's prestige.
Sticking with the equation to determine tournament worth right now for this weekend: Winning WCS Australia is worth 402.83 points. The prize for the winner has not be revealed, but WCS winners tend to get 40% of the prize pool, which here is $10,000, so I'm estimating at $4,000. Number will change should the prize change.
Now, how the players ranked lower than first will work: Second will get 50% of the points (201.42). In a single elimination, semifinals would get half of that, but with a 3rd/4th placement because of a double elimination, 3rd will get 66.6% of 2nd, 4th will get 33.3% (134.28 and 67.14). 5th/6th will get 25% of 2nd (50.36), 7th/8th 50% of that (25.18), 9-12 hafl that (12.59), and 13-16 half that (6.3). WCS Oceania will depend on who qualifies.
Rankings have been updated, they are up through WCS Australia. Last year's Latin American Battle.Net Invitational has been removed for now being over a year old.
By winning WCS Australia, PiG moves up to 124th on the rankings. mOOnGLaDe is very close, at 121st. 3rd place Mafia is down in 204th.
With WCS Oceania next up, there is a slightly bigger points prize for winning. Compared to Australia's 402.83, Oceania is worth 648.14 points!
As you can see, the winner of WCS Oceania WILL break into the top 100 (current gateholder of the top 100 is Sound with 566.95). JazBas, the highest ranked played coming in and the New Zealand champion, will make the top 50 (50th exactly) should he win. mOOnGLaDe winning Oceania would have him flirting with the top 50, in fact he would slot in between Monster and Bomber for 53rd. Australia champion PiG would become 55th with a win, right behind Keen. The player with the biggest to gain is tgun. His performance of 5th/6th at Australia was his first entry in the rankings, and he is currently the lowest ranked player at Oceania, down at 381st. A win would move him all the way up to 73rd, ahead of Creator.
On August 08 2012 06:28 Darkhoarse wrote: Just spotted one strange thing quickly and I'm not sure if there are other errors, but for example SeleCt got 2nd in an MLG. The scale says that 2nd in an MLG is worth 1200, but his total rating is only in the 700's. But definitely a cool concept.
The points degrade over time. So a 1200 point second gets smaller over time.
While watching TAC finals, I'll be working on some other tournaments! :D
A problem I've been having is WCS South Korea. How do I rate the KeSPA players? Just giving them 0 isn't fair, but giving them too high an amount doesn't work either. So what I am going to do is just not give points for the KeSPA players, act like they aren't there in determining the rankings.
Another tournament already going on (that I'll do the points for now) is the ongoing OSL. Similar to WCS SK, the problem is what do I do for the KeSPA players, so I'll just take the points from the non-KeSPA players. Another problem is the unknown prize pool, so I have the mean and median of the players (3069.73 mean and 1585.20 median). Prize pool is not known right now, but if you take a previous OSL for reference ($35362.24), that would make the OSL worth 4001.69 points, just a little less than a GSL!
Full points breakdown for TSL4 and WCS South Korea, among other events, can be found below in the spoiler.
Rankings have now been updated as of August 15th, 2012!
Also, with all players announced (assuming the two TBA in the each open bracket as just byes for the top koreans), here are the point spreads for IEM Cologne! :D IEM Cologne: 1st: 893.17 2nd: 446.59 3rd: 297.73 4th: 148.86 QF: 111.65 Ro12: 55.83 GS 4th: 27.92 GS 5th: 13.96 GS 6th: 6.98 Open Bracket 3rd: 3.49 Open Bracket 4th: 1.75 Open Bracket 5-6: 0.88 Open Bracket 7-8: 0.44
Yeah, the open bracket spots are REALLY not worth much, but considering how the tournament is structured, being in the open is the step below groups, so whatever. 893 points for winning is a decent amount, about half of winning TSL4. The main difference is the prize pool, but this makes sense, since this is just a tour stop for IEM. GL HF to all participants!
As for the actual tournament: let's talk some stats! -The lowest ranked players is the French Terran AureS, way down in 649th place. Even getting last place would move him up to 640th. The lower end of the rankings are VERY close. Last place (Retro at 768) has 0 points, 100 places up at 669 (Mkengyn) is only at 4.91.
-The highest ranked player is current #1, MC! MC dropped out of WCS South Korea to compete in this event, which originally may have meant giving up his #1 to close #2 DRG if he made a run, but with his surprising loss to EffOrt, MC will retain his top position regardless of what happens this weekend.
- The competition for top player from Oceanian heats up, with both #117, the Australian champion PiG (who crashed out of WCS Oceanian), and Australia runner up and Oceanian winner #52 MOOnGLaDe. mOOnGLaDe is 576.24 points ahead of PiG. The only way PiG catches up to pass mOOnGLaDe for top Oceanian player is with a win and mOOnGLaDe getting anything worse than 2nd. A more likely way for PiG to move up is by preventing a last place finish in the groups, that would move PiG ahead of Oceanian #2 (in both rank and WCS finish) MaFia.
-NaNiwa can break into the Top 10 (which is where many people think he should be, but only "good" results at GSL and not much in the foreigner tournaments is hurting his ranking) with a win here. With only a PvT and a PvP to get out of the Open Bracket pretty easily, but each group has at least 2 Zergs, so it will be tough for him to win.
2011 Chinese Battle.Net Invitational Removed 2012 IEM Cologne Added
Some Major changes: Nerchio +2: With his 2nd place, Nerchio goes from 12th to 10th, meaning there are now 2 foreigners in the top 10! VortiX +41: A run through to the Semis means that VortiX jumps from 142nd to 101st, he is now a mere .1 point from the top 100 (and he should get that soon due to other player's points dropping slowly). Bomber +3 sLivko +3 SuperNoVa +2 Kas +2 SortOf +5 ForGG +2 Minigun +11 PiG +3
A procedural update: for MLG Summer, I will delay publishing the weight for the tournament until Saturday. Due to the impossibility (and how badly it would degrade the ranking) of including all the open bracket players, I will only be counting the pool players. I will wait until Saturday, to include the players that make pools from the Open Bracket. As other MLG events, points will go down through to the first round of the Champ bracket, no further.
Tomorrow I will calculate the points for the new PokerStrategy.com League! The GIGABYTE NVIDIA Invitational being in tournament Limbo is going to bring down the rankings of Grubby and Kas, slightly hurting the PSL, but there is nothing I can do about that.
Am I right in assuming that there is no strength of field in your weightings?
This can play havoc with a true ranking. Winning an MLG where 10 of the top 100 attended (just hypothetical for logic) garners the same points as someone who wins one with 70 of the top 100? This can further be credited by breaking down top 10, 25, 50 etc... for truly accurate point distribution.
When I developed a world ranking for Counter-Strike, I instituted a multiplier based on strength of field to assist the general points per placing to ensure teams that were able to attend more events and lesser quality events didn't artificially pad their numbers.
I mean, CreatorPrime is like 20 places behind IdrA? I mean no offense to IdrA (he's a good player), but Creator is definitely much better than him.
Edit: I would weigh GSL more, and then figure out a better system for team leagues. Some players are really good at team leagues (some in fact rarely appear in individual leagues because of how intense Code A qualifiers are), so maybe take that into consideration.
I also want to know how you would weigh certain online tournaments, like ESV weeklies.
GSL should be weighted way more heavily than it is. GSL only being worth a little more than twice as much MLG/Dreamhack is very strange. It is definitely way more than twice as hard to win one.
Either the formula is bad or i am missing something, but MC or DRG don't have this for a year resume. August GSL #1 Date 9/10 5,000 points Geforce Pro #2 DAte 9/29 300 points Octover GSL #2 Date 10/22 2,499 points Blizzcon #1 Date 10/23 (2,000)?? points Arena of Legends #1 Date 11/13 500 points MLG Providence #4 11/20 500 points November GSL #3-4 Date 11/28 1,500 points WCG #1 Date 12/11 2000 points Blizzard Cup #3-4 Date 12/15 900 points 2012 GSL season 2 #1 Date 5/19 5,000 points IEM cologne #1 Date 8/19 500 points
Here is MC IEM cologne #2 Date 8/21 250 points MLG Orlando #2 Date 10/16 1200 points Blizzard Cup #3-4 Date 12/15 900 points HSC 4 #1 DAte 1/8 1500 points IEM WC #1 DAte 3/10 2000 points Red Bull #1 Date 5/27 500 points HSC 5 #3 Date 7/8 700 points NASL 3 #3 DAte 7/15 950 GSL season 3 #2 7/27 2,499 points Asus #2 Date 8/4 1,200
So i Messed up on the IEM i see now that you put the them at 1,500 non WC....but i made the same mistake for both so it equals out.....either way....this does not come close to the one posted above.....hence something is wrong.
For everyone that just commented: please read through the thread (especially the OP) before posting.
divito: Starting with Tournaments at August 10, 2012, all tournaments are weighted based on the players attending. I originally was following the Tennis style where every tournament is given a default point total, but based on feedback, I changed it to an equation that uses the players attending. All previous results prior to August 10, 2012 remain as pure points. It turned out to be useful for a baseline to start weighing tournaments.
glzElectromaster: Thing is, what has Creator done in the past year? Middling results in GSL, 9th in IPL4 (which was a few months ago), and a really good result in TSL4 that, because TSL4 is not done, has not been put in yet. As for team leagues, I'm open to suggestions, but everything I think of comes back to one problem: How can I create a way that rewards team success, but doesn't harshly punish players for the team they are on. What do I mean? Should Taeja get a boost because the rest of Liquid is not quite as good as he is right now? He would get far less all kills if someone else on Liquid got kills more often. Should someone like MC be punished because he is on a stacked team, thus rarely plays? Either it is too small a bonus to really matter, or it is too large and punishes players in certain team situations. And because there are far less team leagues than 1v1 events, there wouldn't be a large enough sample size to cancel out irregularites.
JJH777: People complain that GSL is worth too little, people complain that GSL is worth too much. I think this is a nice happy medium right now. Especially with Code S players constantly coming to MLG events.
Gotnerves: This does not count all events equally. Points decay over time, so events further in the past count less. Up to 365 days after the event, when they no longer count at all. Mvp has the most results over all time, but not the most results in the recent past. This is not a all time ranking, this is a CURRENT ranking. And Mvp, outside of his run at IEM, hasn't done much since GSL Season 2, whereas MC has gotten 3rd at HSC and NASL, 2nd in GSL S3, 2nd at ASUS Summer, and 4th at Spring Arena 2.
Oreo: Of course players that attend more tournaments will do better. But even then, players that do well in GSL get high rankings. Seed had never left Korea for Starcraft until TAC3 finals, and yet is in the Top 10. There is no requirement to go to many tournaments to be a top player, you have to do well in the tournaments you compete in.
On August 21 2012 13:26 Shinespark wrote: Shouldn't Homestory Cup give like, 4 points for the winner or something? I didn't see it listed.
No, that is not a typo, I mean 4 points.
1) HSC is in the spreadsheet. I don't explicitly say it in the points listing, because I just used the same scale as an MLG or something. Feel free to check, you can get the Excel sheet I use to calculate it all.
2) 4 points? You are going to sit there, and seriously say that winning a HSC does not matter? It's a more casual environment, but you mean to tell me that the players don't try or something?
On August 21 2012 13:26 Shinespark wrote: Shouldn't Homestory Cup give like, 4 points for the winner or something? I didn't see it listed.
No, that is not a typo, I mean 4 points.
1) HSC is in the spreadsheet. I don't explicitly say it in the points listing, because I just used the same scale as an MLG or something. Feel free to check, you can get the Excel sheet I use to calculate it all.
2) 4 points? You are going to sit there, and seriously say that winning a HSC does not matter? It's a more casual environment, but you mean to tell me that the players don't try or something?
No, I was just kidding. Gotta be worth at least 9 points, I mean, come on.
Kidding again. It's one of the major but unlisted tournaments so it gives 250 for the winner, right?
@Metalteeth I see, i didn't realize there was a decay over time as i only added the events during the last year, but i didn't calculate any decay. Good Job then.
On August 21 2012 13:26 Shinespark wrote: Shouldn't Homestory Cup give like, 4 points for the winner or something? I didn't see it listed.
No, that is not a typo, I mean 4 points.
1) HSC is in the spreadsheet. I don't explicitly say it in the points listing, because I just used the same scale as an MLG or something. Feel free to check, you can get the Excel sheet I use to calculate it all.
2) 4 points? You are going to sit there, and seriously say that winning a HSC does not matter? It's a more casual environment, but you mean to tell me that the players don't try or something?
No, I was just kidding. Gotta be worth at least 9 points, I mean, come on.
Kidding again. It's one of the major but unlisted tournaments so it gives 250 for the winner, right?
I used the Dreamhack scale for HSC, so it's worth 2000. It was changed slightly for the different style, but the point breakdown is the same, just where it says something like GS 5th, change it to 1st GS 4th or something.
On August 21 2012 13:26 Shinespark wrote: Shouldn't Homestory Cup give like, 4 points for the winner or something? I didn't see it listed.
No, that is not a typo, I mean 4 points.
1) HSC is in the spreadsheet. I don't explicitly say it in the points listing, because I just used the same scale as an MLG or something. Feel free to check, you can get the Excel sheet I use to calculate it all.
2) 4 points? You are going to sit there, and seriously say that winning a HSC does not matter? It's a more casual environment, but you mean to tell me that the players don't try or something?
No, I was just kidding. Gotta be worth at least 9 points, I mean, come on.
Kidding again. It's one of the major but unlisted tournaments so it gives 250 for the winner, right?
I used the Dreamhack scale for HSC, so it's worth 2000. It was changed slightly for the different style, but the point breakdown is the same, just where it says something like GS 5th, change it to 1st GS 4th or something.
So HSC is worth more the IEM stops? wow not sure i agree with that. I thought you clocked them both in at 1,500.
Gotnerves, if you think one tournament should be rated differently, good for you. That has been the majority of the 10 pages of this thread, and I am not changing it. All future tournaments are going to be calculated using the weight of the players, and changing a single event by 500 points will change nothing. This change was done based on feedback about the tournament weightings being subjective, and they will not be as much anymore. So it really doesn't matter anymore.
On August 21 2012 13:44 Metalteeth wrote: Gotnerves, if you think one tournament should be rated differently, good for you. That has been the majority of the 10 pages of this thread, and I am not changing it. All future tournaments are going to be calculated using the weight of the players, and changing a single event by 500 points will change nothing. This change was done based on feedback about the tournament weightings being subjective, and they will not be as much anymore. So it really doesn't matter anymore.
Well still good work, but common sense would agree with me but it's your rankings and formula. Impressed you took the time to do this
I'm sure its some if of all of the following points have been made before, but here are some issues I have with this system: The point drop of between places (especially first and second) seems a bit extreme. If two players finish 1st and second in the GSL, they may well be the top two players in the world, why does first get so many more points than second? Your system doesn't take into account difficulty. If the best 3 players in a tournament start in the same group of four, from which only two advance, is it really fair to punish the third place player so heavily? (this is an extreme example but you can see the point). Your weighting methodology is very subjective. It promotes quantity as a substitute for quality (this is fixed slightly by the drastic fall off in point values as players finish lower, but it remains an issue
I think the TLPD Elo system does an excellent job of covering all the points I've mentioned, and in a far more established manner. The one area where your system may be a viable substitute is as a measurement of tournament success. As I contend in my second point, this is related to, but not a perfect approximation of, skill. If you keep updating this I'll keep looking as it seems like an interesting system and fresh ideas are always great. I don't think it will be replacing Elo anytime soon though.
On August 07 2012 14:22 hunts wrote: GSL should definitely be worth more than it is IMO. Winning a GSL is a MUCH bigger accomplishment than something like an MLG or IPL which are all relatively close in points.
While I agree GSL is by far the hardest tournament, there's the problem with it being so regional based. Many players don't get the opportunities to try and qualify over and over. I know it's a super safe assumption to say that 99% of those players wouldn't be able to qualify anyways, given all the chances in the world, but who knows?
I think a really cool improvement to this idea (forgive me if its been mentioned already) would to be to use your rough prelimary results and use those to recalculate tournament weights and then show the updated list. This might make it a bit less subjective and I'd b e interested to see the results.
On August 21 2012 13:44 Metalteeth wrote: Gotnerves, if you think one tournament should be rated differently, good for you. That has been the majority of the 10 pages of this thread, and I am not changing it. All future tournaments are going to be calculated using the weight of the players, and changing a single event by 500 points will change nothing. This change was done based on feedback about the tournament weightings being subjective, and they will not be as much anymore. So it really doesn't matter anymore.
Well still good work, but common sense would agree with me but it's your rankings and formula. Impressed you took the time to do this
I think common sense here goes more or less:
1) If HSC weight change, he will have to recalculate all the weights of all the tournaments based on ranking of participants. So a lot of work. 2) It is a small change, and will probably make no significant difference to the ranking. 3) No matter the starting weights, over time it will converge to the same weights from player rankings. ie, 1500 or 2000 will matter less and less as time goes, not only because of point decay.
So even if there is a consensus on 1500 being a better weight than 2000, it is essentially not worth the effort to change it.
This doesn't factor in players that don't participate in many tournaments. NesTea for example pretty much only does GSL save a few foreign tournaments here or there. Cool that you did this though.
On August 21 2012 14:06 VediVeci wrote: I think a really cool improvement to this idea (forgive me if its been mentioned already) would to be to use your rough prelimary results and use those to recalculate tournament weights and then show the updated list. This might make it a bit less subjective and I'd b e interested to see the results.
You mean take what I have now, use the equations to calculate the new weight of all tournaments, then go through every event in the past year? Well, the thing is I doubt it would make all that much of a difference for the work. And those tournaments are worth less and less every day.
Haha Metalteeth, funny, i got alomst the same idea for world rankings like that a few days ago. Here is where our ideas differ (just for consideration, but i guess after 12 pages of criticism you won't change everything all over again)
1. I was aiming for an ATP-like ranking as well, so first i took their way of point distribution (which by the first post i assume you did too at first=
2. I weighed tournaments simply by prize-money in these categories taken from Liquipedia - Premier Events - more than 20.000$ - Major Events - 5.000 to 20.000$ - Minor Events - 2.000 to 5.000 $ - Challenger Events - 1.000 to 2.000$ - Challenger 2 Events - 500 to 1.000$ - Challenger 3 Events - 100 to 500 $
I did not enter all results yet (and won't do it since you already did the same), so maybe one or two of the bottom categories would be too much work.
Points for the specific categories are taken from ATP point distribution. So Premier Events equal Grand Slams, Major Events equal the ATP World Tour Masters 1000 etc.
3. I only counted events of this year
4. There is no regression. If a tournament was played before Dec 31 2011, it is simply not counted. For the future, i would have only counted events that are not older than one year.
I know much of these points for already discussed here so you don't need to answer to my post again. Systems like this are always a little depended on the "taste" of their creator and all have their weaknesses, so it's almost pointless discuss every detail. I think you did a great job with yours, so keep it up. E-Sports really needs reliable world rankings, not only in Starcraft!
As for team leagues, I think I have a possible solution, but would be open to feedback. To calculate how much a tournament would be worth, first each team would be calculated. This is the total sum of the top 5 players from that team AT THE START OF THE TOURNAMENT. Same equation for all tournaments at that point, only with the change with dividing by 15 (to account for the much larger points pool), with the same points spread similar to other tournaments. To assign points for individual players, they will receive a percentage of the points the team receives, equal to the percentage of total game wins by that player throughout the tournament. For example, if a team wins 30 individual games over a tournament, and 1 player wins 15 of those games, he will receive 50% of the points for the tournament for that team.
Holy points! The likely winner (Scarlett) will get a HUGE boost from this. MLG will be calculated tomorrow, as we wait for the players from the open bracket to make pools. Tournament will be calculated from players in the pools ONLY.
Creator jumps to from 75th to 26th with his win of WCS SK, and SuperNova goes from 23rd to 13th with his win of Campus Part EU!
WCS NA has some HUGE possibilities to change the standings. VIBE would become 21st, Scarlett 22nd with a win of NA. Once the tournament is added, Major will break into the Top 100 (add get even better once TSL4 finishes).
MLG Summer will be calculated tonight, once the players from the open bracket enter the pools! :D
This MLG is actually lower than it normally would be. A lot of top Koreans stayed behind to close out WCS Korea, and some of the top Europeans were at Campus Party EU. This lead to a lot of lower level US players being seeded (or coming through open bracket), making the weight of the players pretty low. So winning this MLG is worth only 200 points or so more than WCS NA. Without every player that is below 500 points (approximately 100th, and there are 15 players below that), MLG would be worth over 2900 points.
That one tournament, with so many unknown British players, added like 15 players to the rankings. JonnyRecco, winning i46, will go up about 50 spots to around 120th from 175th. I will publish the results once MLG finishes. Expect some big changes. Scarlett, will break into the Top 25 with her win of WCS NA. Stephano or Taeja winning should bring them to the Top 3. HerO winning should break him into the Top 10 again, after losing it to Nerchio.
EPT
