|
Some of you may know that we released a website a couple of weeks ago with a progamer ranking as well as several statistic subpages about balance, prizemoney distribution etc. (SC2Charts.net).
In the post that I created on TL about it the question was raised whether top tournaments like MGL and GSL should be awarded with more ranking points if a player is able to perform well in them. There have been a couple of pros and cons but as the topic itself doesnt really adress the issue I wanted to have these ideas in a separate thread as I'd love more opinions on the matter.
Poll: Should matchwins in major tournaments be given more points?? Yes, there should be an emphasize on major tournaments in the ranking (218) 81% No, one should treat all matches the same for the ranking (50) 19% 268 total votes Your vote: Should matchwins in major tournaments be given more points?? (Vote): Yes, there should be an emphasize on major tournaments in the ranking
(Vote): No, one should treat all matches the same for the ranking
Here are two of the posts mentioned:
Primadog:- There lacks existing, community agreed upon definition of major/minor tournaments. The closest of which is the BIG EVENT calender on TL. Frankly, even that is rather a crap-shoot.
- I been keeping an eye on the TLPD Elos closely for the past few months, and noticed that the Elo values have tendency to precede reputation and major tournament wins, especially in the EU arena, where weekly cups are much more prolific. I suspect the same pattern can be observed even more clearly with this improved algorithm.
The discrepency between korean and foreigner rating are as much result of regional isolation as the simple lack of recorded games in the Korean side. There's still only 2 major tournaments (GSTL or GSL) for a korean player to participate in, so if they're knocked out early, that's it for datapoints for a month.
- No amount of weighting will resolve this clear lack of data from the Korean side. Giving greater weight to GSTL and GSL will not give a cleaner rating, but instead magnify the variance that is already fairly terrible, nature intrinsic to the Korean arena. This is a mathematical limitation inherent to any ordering problem, as we learned information theory 101.
Redemption:
Well, the only reason I brought that up is that I assumed the larger, more prestigious tournaments have more money on the line and more publicity (huge audiences watching you play live), and as a result, there is more pressure on the player to perform. I feel that the players that do perform well under this type of environment should be rewarded more for their victory than someone that wins a small $100 online tournament from the comfort of his bedroom while wearing nothing but his underwear and munching on a bag of Doritos.
To go back to my tennis analogy, there's no reason to think players are not playing as hard in the smaller Masters Tours compared to the Grand Slams. They play their heart out every match. But everyone recognizes the huge size, prestige, and monetary compensation associated with the Grand Slams, and so they are weighted more.
I would love if we could gather some additional opinions on that topic. In general it should be no problem to adjust the algorythm so that for example tournaments with a certain amount of prize money (10.000?) would be weighted higher than others. The main question is would that make sense to you and be preferable to the current solution?
Kind regards, Khaldor
|
Hey Khlador! I'm that guy who responded to you on Twitter on like the first day SC2Charts was released! Back then I seriously questioned its accuracy, but looking at it now it seems to be more realistic.
Primadog's a genius when it comes to stats, so I'd really value whatever he says. The only serious advice I could offer, is based on what the Charts are like now.
The Top 6 seem very accurate, if not for their order. I was surprised to see NesTea come in at No 6, especially since anecdotally at least people always claim he's probably the best player in the world.
Not sure what MarineKing is doing up so high. How long of a memory does SC2Charts have? If this was like three months ago then MarineKing would certainly be in the Top 10, but at the moment he's still in Code A.
From all this, I dunno: my only suggestion would be to look at how long the Charts "remember" games. How much is, say, a GSL win from May worth compared to a GSL win in July?
|
Well, the games that are taken into consideration are all the games in our database (which is quite a lot and dates back to the beta). Players lose points over time if they dont play, very slowly though in order to keep a fair balance.
If you want to see why Marineking has so many points right now just click on his name and you can check his latest results. If you are interested you can even list all the results of all the games hes ever played in tournaments/events that are in our database. Nestea has won a lot of games but most of them were again supposedly "weaker" players than for example the opponents Puma and DRG hat to face lately. That's why he's not #1.
Nonetheless I hope we can focus on the topic I adressed in the opening post. I can clearly seen why people would want to emphasize big tournaments by weighting them over smaller ones but I'm still undecided in whether it'd be a good thing to do or not. Any additional input is more then welcome.
|
I'd say that as long as you're weighting the value of a win by the opponent, then it doesn't really matter what the context is. For example, beating Bomber in some little tournament seems (to me) as valuable as beating Bomber in a big tournament.
Winning a big tournament is more valuable that winning a small tournament, but that will already be reflected in the quality of the individual opponents. So, as long as tournament finishes aren't a separated category for the rankings (which my small understanding of the system says is not the case), I'd say you should not weight games by the tournaments in which they take place.
|
Absolutely GSL, MLG, etc should be weighed more heavily than small online weekly cups. There's a reason why we call SC2 an e-sport, and it's not just vanity or hype. Being able to win games in the comfort of your own bedroom is not the same thing as winning tournaments in a foreign country where hardly anyone speaks your language, big money on the line, jet-lagged 8 times zones, with opponents guaranteed to be using their best strategies, several cameras in your face and hundreds of fans watching in the same building. I could care less if you can beat Nestea on ladder 7 times out of 10 or win $100 dollar cups that nobody watches, your status as a progamer depends on whether you can do it when it counts.
|
On September 04 2011 14:47 red4ce wrote:
Absolutely GSL, MLG, etc should be weighed more heavily than small online weekly cups. There's a reason why we call SC2 an e-sport, and it's not just vanity or hype. Being able to win games in the comfort of your own bedroom is not the same thing as winning tournaments in a foreign country where hardly anyone speaks your language, big money on the line, jet-lagged 8 times zones, with opponents guaranteed to be using their best strategies, several cameras in your face and hundreds of fans watching in the same building. I could care less if you can beat Nestea on ladder 7 times out of 10 or win $100 dollar cups that nobody watches, your status as a progamer depends on whether you can do it when it counts.
All well and good, but I don't think the assumptions of this post are correct. Ladder games don't go into rankings, and players like NesTea don't populate the little tournaments.
|
Winning the GSL is much harder than winning anything else, and this is already taken in account in the algorythm. Not the GSL itself, but you won't find the same level nor quantity of super strong players anywhere else, so, not directly, but I believe this factor is already being considered. In short, if you win a weakly cup by defeafing the same players that Nestea does on the GSL, it SHOULD grant you the same points. And btw there is the money factor, which is already considered on the money ranking, plus the Live TV factor, which I have no idea how to consider; perhaps a 0.8 to 1.2 corrective factor with 1.2 being a GSL live event and 0.8 being an offline unknown tourney.
|
On September 04 2011 14:38 Wren wrote:
I'd say that as long as you're weighting the value of a win by the opponent, then it doesn't really matter what the context is. For example, beating Bomber in some little tournament seems (to me) as valuable as beating Bomber in a big tournament.
Winning a big tournament is more valuable that winning a small tournament, but that will already be reflected in the quality of the individual opponents. So, as long as tournament finishes aren't a separated category for the rankings (which my small understanding of the system says is not the case), I'd say you should not weight games by the tournaments in which they take place.
I don't agree. Just for example, Bomber on Friday played Lucky, and clearly screwed around a lot of the game and didn't really care as the game meant nothing to them. They had already qualified for the playoffs. Are you telling me that win, where Bomber clearly wasn't playing his best in an unimportant match, should be worth as much as beating Bomber in a GSL Final?
|
You should weight the points according to the general level of the participating players.
E.g. if there are a lot of good players in a tournament, a win should count higher.
|
You need to define major tournaments by prize pool if you do this. MLG circuit tournaments should NOT count as a "major tournament". Though this year's championship should.
|
My suggestion is as follows:
I think if you weight anything more... you should weight in person matches a bit more.
For example, maybe a game is worth 100% weight when played. But, if the game is played in person (DreamHack, NASL Championship, MLG, GSL, HomeStoryCup, and so on) then the game is worth 110% of normal weight.
The reason I say this is because there is relatively little lag factor and is more of a judge of skill.
Edit: And for major tournaments. I would consider the following as Major:
GSL Code S
GSL Code A
Blizzcon
TSL
NASL
MLG Events and Grand Finals
DreamHack
IEMs
HomeStoryCup
IPL
|
Yes, big tournaments should be weighted stronger than other tournaments.
The ones directly coming to my mind is the following which always have fantastic competition is the following: IPL, MLG, NASL, DreamHack, Assembly, Blizzcon, IEM, HSC, GSTL, GSL and TSL.
Regarding prize money I think you should manually decide tournaments based on competition+prize money. If you only look at prize money MLG Pro Circuit may not be included but as we know the competition there are one of the best in the world.
|
I have a question, is there a way to define a "major" tournament by the number and quality of participants? Like if the tournament reaches a minimum threshold of participants, and the combined or average ELO of the players was above a certain value, than maybe it should be considered major.
For example, the HD World Tournament players probably have a high ELO average, but the tournament doesn't have enough participants to be considered major.
|
I think a quick and easy definition of a "major tournament" would be that it satisfies two of these three requirements:
1. The prize pool is above $20.000
2. The tournament contains at least 3 players from NA, 3 players from EU and 3 players from KOR.
3. The tournament is played "offline", i.e. at a lan, arena or similar location.
That definition would make it include every event that I would consider major. The GSL is included even though there might not be American or European participants every season. TSL is included even though it isn't an offline tournament. MLG is included even though the prize money is lower than other major tournaments.
|
Yeah, I agree in having "majors" because then we can create the idea of a "grand slam" like in golf or tennis and I think that those should be weighted with a little more points just because the level of competition will be greater, but you must define those terms first. I don't know how we can create those definitions without some sort of league or governing body though, as in tennis.
Without that body, the idea of the "major" is too subjective and subject to change.
|
Do it like they do with FIFA ranking for international soccer, you get a certain amount of points based on who you beat, although there are too many players and games that it would be next to impossible to rank and record every single player. A player who gets X amount of points for winning a european tournament while another gets the same X amount of points for winning a korean tournament is obviously imbalanced. Although europe is good, winning something in korea is obviously much more difficult and therefore should give more points. Same goes with winning in MLG, when only 10 koreans show up and you only play 2 koreans to win, it just isn't the same as someone beating 6 koreans to win the GSL. If you wanted to do it for those who play in the GSL and GSTL and only those who make it into code A get points recorded that is somewhere to start.
To the point of those in GSL/GSTL not getting that many points because if they lose in Ro32 they will fall drastically behind, isn't that the point? Obviously from the korean point of view, beating a foreigner is much less of an accomplishment than another korean because of the skill level difference at the moment. I just don't see it being able to be measured on any scale that matters because you can have a player stay in NA and win literally everything with no koreans in it have a very high rating because he never loses. Not playing against the best don't deserve the same points.
To the exact point of more points for the prize pool, it kinda goes back to the same thing. The Dota 2 tournament had a 1 million dollar prize pool, if this ever happened for StarCraft is the player that wins that the #1 forever? If it is a tournament for 50$ but it has every good korean playing in it, it deserves more points than a stupid little NA only tournament for 10,000$.
Sorry for vomit thoughts on page, just seems too difficult to create a fair ranking system in the current scheme of things as it is such a wide spread game.
|
Philadelphia, PA10406 Posts
I agree with Primadog. I think that a ranking that highly rates major tournaments wouldn't be useful, because it would largely parrot what the community thinks about various players. Because more people watch the bigger tournaments, the results from these tournaments have a huge impression on who the community believes are the best players. But this does not usually reflect reality.
I think he's is right to point out that the TLPD ELO's often predict up-and-coming players before the community becomes aware of them. Any good ranking should have this same effect. Before a player becomes commonly regarded as among the elite, they are first noticed by the statistics and people who are closer to the scene. A ranking that doesn't do much but reflect what the community at large thinks isn't too helpful.
EDIT: To the poster below me:
On September 04 2011 17:48 J.E.G. wrote:
bigger tourny's bring better players. If you are beating better players, you should get more points.
This isn''t really the case though. Some European online tournaments have insanely strong pools, and even low money weekly events generally have at least a couple elite EU players.
|
bigger tourny's bring better players. If you are beating better players, you should get more points.
|
Weigh the average skill level of the tournament participants when determining rank value. You could rank every tournament individually or separate them into tiers based on participant skill level. All things being equal, bigger tournaments will tend to have higher skill level participants. It follows then that these tournaments should matter more for ranking.
|
If Machine beats Artosis in a major tournament such as the NASL, should he be awarded more points than DIMAGA beating Socke in a minor EU cup?
From the responses it seems by "major tournament" you mean GSL only. There is really no need to do this, if in the future there is more interaction between koreans and foreigners, and the koreans are much better than the foreingers, the ranking system will reflect this without artificial inflations. If the korean scene gets more and more isolated then there is no point in a combined rating system.
|
|
|
|
|
|
EPT
