|
On November 12 2010 16:03 Azzur wrote:
In my opinion, the purpose of a tournament is not to "find the best player". It is to provide entertainment for both players and fans. If "finding the best player" were the only criteria, then we would just use a round-robin format containing many games.
Thus, since the extended series is deemed as confusing and is disliked by many fans, then it should not be used.
I'd argue that making sure good players get into the quarter/semi/finals is pretty important in providing entertainment for both fans and players. If a tournament was extremely random and you end up with two extremely mediocre players in the finals or a lopsided matchup, that wouldn't be very fun to watch would it? As the OP pointed out, the extended series doesn't do too much really either way, so it's really a judgment call.
|
I just love teamliquid! :D
This is noteworthy for sure
|
On November 12 2010 20:18 teamsolid wrote:Show nested quote +On November 12 2010 16:03 Azzur wrote:
In my opinion, the purpose of a tournament is not to "find the best player". It is to provide entertainment for both players and fans. If "finding the best player" were the only criteria, then we would just use a round-robin format containing many games.
Thus, since the extended series is deemed as confusing and is disliked by many fans, then it should not be used. I'd argue that making sure good players get into the quarter/semi/finals is pretty important in providing entertainment for both fans and players. If a tournament was extremely random and you end up with two extremely mediocre players in the finals or a lopsided matchup, that wouldn't be very fun to watch would it? As the OP pointed out, the extended series doesn't do too much really either way, so it's really a judgment call.
Of course, finding good players in the latter stages of the tournament is part of the entertainment itself. That's why single elimination bo1 may be exciting but doesn't fare well on the entertainment criteria.
In my mind, a double elimination bo3 does serve this purpose (finding good players, resulting in entertainment) well. Hence, for me, there's no reason of using the extended series (since it's confusing and disliked).
|
Please delete this double post
|
You know what would sort this issue out.. a league lol. There's a reason most competitions start as a round robin or a league, and then either use that to determine rank or then go into single elimination. We saw with machine getting unlucky 2 tournaments in a row.. which left him unseeded. In a league format this would not have been an issue, but in a knockout tournament even though he is better than other players who advanced further his rank and therefore his seeding in the next tournament didn't show this.
Its unreasonable to run a full league system such as is employed in football. But the most successful spectator sports all involve tournaments based upon round robin into single elimination, NFL, World Cup Football, European Champions League. Admittedly this may be easier if MLG were to run a team comp but i believe If they ran league systems on the friday and saturday. 8 people per group.. round robin. so everyone plays 7 games over 2 days. top 2 advance from each group decided by games won if there's a draw.. (same as world cup) into a single elimination.. either pre decided like world cup, seeded b position like american sports ie.. nba's playoff system. or staright up draw out the hat.
This is a world recognised format. easy to follow, allows a lot of exciting matchups and upsets, and still should allow the best players to at least get through to the single elims. None of this looking at the brackets completely lost for 10 minutes figuring out who's playing who.
I mean to add.. best of 3's in round robin.. best of 5's for KO stages until final BO7. Which makes it at least similar to how GSL run too, making the standard SC2 format easy.
|
On November 12 2010 18:06 Kazang wrote:Show nested quote +On November 12 2010 17:30 scion wrote:On November 12 2010 16:38 Kazang wrote:
Your so called statistical analysis is filled with biased side notes and totally neglects a number of the the real reasons behind using the extended series.
Notably that it provides a better ranking for all players not simply better chances of just the best player winning, it prevents things like the 2nd best players going out in the first round. Which your analysis doesn't take into account.
Your "scope" contains "questions" that are not questions, not even rhetorical, but statements that haven't been proven within your analysis or even supported by other statements that are proven.
Your math is solid but it's isolated and applies or has been applied to only certain circumstances ranges without really taking into account the varying factors that have to be considered. Such as were does seeding fit into this?
It is used and without that being factored into any math it is essentially worthless.
That's all mostly negative criticism but you don't need anyone to tell you what you got right, you already know that. Wow people still posting to point out flaws in model? Read the thread before posting, YES there are flaws in the model, we can't have a perfect model. Again, simplest model is best model to make decisions with, because we can't just add in these non-quantifiable factors in the model and expect it to work. Sounds like you just wanted to pint out flaws. Where is the bias in his analysis or scope? Read the scope, which specifically says its NOT concerned with the questions listed. Can we actually talk about his conclusion, that is, this simple model tells us that while extended series contributes a little bit to making the tournament outcome "fairer", in reality, it does not have big enough effect to be absolutely certain, considering all other factors. So whether they should continue with what they are doing, or just do regular double elimination since statistically its not SO different, or come up with a whole new tournament method. Yeah no wonder people are pointing out flaws in the model...... It is kind of important if you are going to use it as evidence or basis for a decision. You don't need a mathematical model to point out the logical benefits of a extended series. So why apply a flawed mathematical model at all if you are going to then factor in other external factors? If the model is flawed, which it most certainly is, how can you logically use it as an argument for anything? The scale of the model is also quite ridiculous, round robin for a 128 man tournament like MLG Dallas is 8128 games, how the hell can you compare that to single elimination of 126 games in the same graph? The scale is insane, the accuracy difference is far bigger than than those little jpgs show since the number of games is the biggest factor at work in a live tournament. The comparison is more misleading than anything to someone who hasn't already thought about this. I also read the scope clearly, I'm pointing out a mistake in the writing; it says "here are questions" then lists statements. Of course I want to point out the flaws, duh..... You cannot base an argument or discussion on a flawed premise, if you do it's just pointless. As it is the model shows nothing of value, other than the fact this the extended series is better even when not taking into account the full range of benefits the extended series offers. So then what is the point of it?
NOT concerned with. No you didn't read it.
This post is an in-depth analysis of the statistical performance of
different tournament formats. It is not concerned with many other
important questions, for example:
He is NOT using this model for an argument. It whether you think flawed or not, does a good job of simplifying how a league works. And he's not using math to show benefit of extended series, he's showing how much benefit it provides.
Also he put round robin and double elimination in the same graph so that we can just compare between the two. Hardly a suggestion that MLG should adopt a round robin system. Sigh, if you don't get stats, don't post about it.
Wow, a lot of haters in this topic that don't understand the point of modelling. Models are not meant to be a 100% perfect representation of reality but are used to draw out insights into the behaviour of a system. Every model has limiting assumptions and these should be should considered when interpreting the results and drawing conclusions, just as nzb has done. Also, just because some assumptions are limiting does not mean they will necessarily have a large effect on the results - this should be reasoned through. Anyways good job nzb.
this guy gets it.
Also, if people are gona point out flaws, it has to be an effect that affects one side. As in, if you can't prove that in Extended Bo7, Winner's bracket guy has a clear psychological advantage over loser guy, its not going to effect the stat itself in any meaningful way.
I feel like we've already concluded this thread in page 3 and now people are just adding meaningless objections to the model.
|
Nice work on the analysis. It proves what most probably expected: that double elimination with extended series is just a minor tweak to the normal double elimination system and doesn't dramatically change the results. I feel that the main point that people want to discuss was left unaddressed - there was no comparison between the extended series and a normal bo3 in letting the more skilled player advance. Anyway, people seem to emphasize these hypothetical "skill scores" that they assign players in their heads a lot. Tournaments will always be about whoever wins the matches on that day, and the best we can do is give them fair chances to win the tournament if they just win games. If you want to know who is the best player, no system will allow you to determine it from just 1 tournament. What you are looking for are point/Elo systems that take the results of multiple tournaments.
I've probably followed nearly 100 double elimination tournaments in various games since the year 2000 and this is the first time I've seen anybody complain about unfairness in a standard bracket and start counting single maps from previous series to try to point out some kind of injustice. Before MLG I never saw anybody use a system that would count the scores of the previous match in a later match. I see it a bit like starting a soccer match in the finals from the score you finished a group stage match at, or (from my post in the previous thread):
It's OSL, and Flash and Jaedong are in the same group in the group stage. Stork is in another group, and all 3 advance with Flash beating Jaedong 2-0. If Flash and Jaedong meet in the finals, should Flash start 2-0 up in a bo7? If he meets Stork, should it start from 0-0? What you are arguing is that Jaedong needs to prove he's the better player by resuming the previous series and winning 4 games out of 5.
Also you could imagine a double-elimination bracket played as bo1's to eliminate this argument of losing 0:2 then winning 2:1 to eliminate someone. In this case a score of 1:1 is the only one that eliminates the player that won the first match. Looking at their other results, the winner of the first match lost another game while the loser didn't, so that breaks the tie. The reason we choose to go to bo3's from this system is to allow players to overcome an unfavorable map or build order loss, not to write map wins down and sometimes use them later depending on who the players play against.
|
On November 12 2010 13:07 nzb wrote:
I didn't talk about this in the main post, because its just my opinion and wasn't backed by any numbers, but I think a good format would be:
- Play swiss-style tournament to determine the top 8-16 players.
- Play single elimination to get champion.
This would be a very reliable way to determine the top 8 or 16, and then would switch into overdrive to determine the champ. It would be very exciting, similar to how the NCAA does March Madness. I would love it if we could get someone to do some special event using this format just to try it out.
Lets explore this idea, because I can see there being problems with using a swiss pairing system for Starcraft.
+ Show Spoiler +
For those who are unfamiliar with a swiss pairing system, it works by having a scheduled number of rounds, with every player playing every round.
For example, we have IdrA, Nony, Ret and Nazgul (names off the top of my head)
First round IdrA plays Nony, Ret plays Nazgul. IdrA and Ret win.
Second round IdrA plays Ret because they won, Nony plays Nazgul because they lost. IdrA and Nony wins.
The third round Ret would play Nony, because they each have 1 win, and so on.
Over a predetermined number of rounds, this system will sort the players from best to worst theoretically, and requires fewer games than a round robin system.
The first and easiest to solve problem would be doing the pairings for the first round. You could do something like use peoples sc2ranks scores, with the caviat that teammates won't play each other. Pretty simple.
After that, lets say we have a 64 player tournament. Swiss pairing requires 6 rounds to sort 64 players, or 5 rounds if you use accelerated pairing.
You'd then have to figure out a points system for wins and losses. Probably the easiest way is to give 2 points for a win, 1 point for a draw (however rare) and 0 points for a loss.
The problem then becomes that the point numbers are too low to realistically sort players. Assuming each pairing plays 3 games, you will after the first round have 32 winners who have either 4 or 6 points. The system is supposed to pair those players together, but it can't because there will be too many tie scores.
Sorting ties by countback would be impossible for round 2, because everyone with 6 points opponent would have 0 points, and everyone with 4 points opponent would have 2 points. You can't sort the ties by their sc2ranks stats because those were just used to sort them for the first round, and there won't be enough tie games to balance the system because ties in Starcraft almost never happen at high level play.
You could increase the number of games played to increase variance in points totals, but this puts you closer and closer into the territory of having far too many games played to organize a tournament. As is, a single elimination BO3 series tournament requires between 96 and 144 games to get from the round of 64 to round of 16 like you want, but a swiss pairing tournament of this size requires 480 games using accelerated pairing, or 576 without. Increasing the number of rounds to say 5 to mitigate the issue of ties in round 2 jumps you to 800 games needed to sort your players.
Another possibility would be to have people compete in the swiss pairing bracket by team. In a 64 player tournament, if each team were allowed to send 8 players, you could have 8 teams in the tournament, with teams playing other teams, and the score could be a combined team score. Then the top 2 teams could compete single elimination style in the final brackets. This system would definitely work with swiss pairing, but only teams that could send the required number of people would be able to participate.
The other possibility would be to create some system that can give people more or fewer points for a win, and losing players some points for losing depending on how well they did. This would be nebulous territory, because there'd be a lot of arguing over it within the community, and any system imposed would change the way the game is played. Count points by number of expansions? Well that favors Zerg. Count points by how long the game goes on, assuming that a longer game is a closer game? That disadvantages (or possibly advantages depending on the round in the tournament) early rush strategies.
In short, I think a swiss pairing starcraft tournament would be very difficult to do for single players. For teams I think it would work quite well.
|
Swiss rounds kind of fails in SC2 because you need to wait for the longest match in a round to finish before you know the matchups for the next round, and you can't exactly set a time limit for a match. You could do it in a long-term tournament like the GSL but then you cut into the practice time of the players since you'll announce matchups later.
|
On November 13 2010 01:47 Teddyman wrote:
Swiss rounds kind of fails in SC2 because you need to wait for the longest match in a round to finish before you know the matchups for the next round, and you can't exactly set a time limit for a match. You could do it in a long-term tournament like the GSL but then you cut into the practice time of the players since you'll announce matchups later.
I <3 Teddyman, he seems to always have my back.
This is an additional issue with Swiss Pairing, it would be extremely difficult to play the entire thing out in one day. It would have to be a multi day tournament.
You could speed it up by having all 32 pairs play at the same time, but then you'd only be able to cast 3% of the games, which would be unacceptable to the average viewer. Unacceptable to me at least.
|
In short, I think a swiss pairing starcraft tournament would be very difficult to do for single players. For teams I think it would work quite well.
Another thing to consider is that for a swiss-style tournament, you could drop the BO3's and just accelerate the pairing between players based on individual games (not series).
This would let you ahve 3x the games to sort people with, and rely on the swiss format to correct the additional randomness of not using BO3's. I think it might be able to work.
|
On November 13 2010 01:52 Ketara wrote:Show nested quote +On November 13 2010 01:47 Teddyman wrote:
Swiss rounds kind of fails in SC2 because you need to wait for the longest match in a round to finish before you know the matchups for the next round, and you can't exactly set a time limit for a match. You could do it in a long-term tournament like the GSL but then you cut into the practice time of the players since you'll announce matchups later. I <3 Teddyman, he seems to always have my back. This is an additional issue with Swiss Pairing, it would be extremely difficult to play the entire thing out in one day. It would have to be a multi day tournament. You could speed it up by having all 32 pairs play at the same time, but then you'd only be able to cast 3% of the games, which would be unacceptable to the average viewer. Unacceptable to me at least.
Well, I'd agree with you except that this seems to be what MLG does anyway. 
|
That would not solve the problem of having 32 players with the same score in round 2.
I've used swiss pairing to organize tournaments for different games before, and I do not see how you'd be able to sort that.
Edit: I gotta go but we should continue this discussion!
|
On November 13 2010 01:58 Ketara wrote:
That would not solve the problem of having 32 players with the same score in round 2.
I've used swiss pairing to organize tournaments for different games before, and I do not see how you'd be able to sort that.
Edit: I gotta go but we should continue this discussion!
I imagine you would just use the initial seeds to sort between people with equivalent scores, and having 3x the number of rounds would quickly break people into categories.
I know that Google used this exact format (swiss->single elim) for their internal tournament at the end of the beta.
|
On November 12 2010 13:24 nzb wrote:
I would caution, however, what would people's opinions be if instead Liquid`Tyler had beat Painuser 2-0 in the winners' bracket, and then lost 1-2 to him in the losers'. He would have gone 3-2 against him in the tournament, but been knocked out. Where would TL.net stand on the issue then?
It doesn't matter if the effective result is 3:2 or 4:0, as individual games are not important.
At the start of the tournament both players have a record of 0:0. After the first match, first round player A can be considered the better player, because he's ahead with 1:0. But when they meet again later both players have played n matches, have n-1 wins and 1 loss. They both have beat the same amount of players, have won the same amount of matches and technically no one should be favored, because they're back to equal. Every other matchup in the tournament has exactly these initial conditions n-1:1 (or n:0 in the winners bracket).
If one of them wins he's the better player no matter what, because his record will remain n-1:1 while the other will be eliminated with n-2:2. How he lost at that point or would have performed in a different tournament mode isn't important. But, but..just no.
Take a look at other players they can 2:0 every game and lose one 2:1, but in the losers bracket the players are treated equal even if his opponent lost one match 0:2 and barely clawed back with 2:1 in every other match. Yet the 'better' player won't get favored through extended series unless they've met before and that makes it complete BS. You either favor every player that is considered better or none at all, everything else isn't fair..period.
The only one who benefits from ES is the lucky player who gets this huge advantage, eventhough the match record clearly shows that he isn't better and he wouldn't be favored if he's unlucky and meets a different player. And because of the way how SC2 works a weaker player can put a better player into serious trouble if he's favored by ES, because he can utilize dirty tactics to force wins and reduce the decision making process.
A tournament needs to be fair to everyone and give them the same conditions to win, but the ES fails to do that and artifically favors a single player for absolutely no reason. By that logic the ES must be enforced on every match where a player has a better individual game win/loss ratio, because he has to be the better player, but that would be really stupid. Because like I said only the match record is a consistent measurement of skill in a DE tournament.
The reason why ES probably works well for halo is, because a game takes 15-30m and contains thousands of individual decisions, where the better team probably makes more good decisions and takes the win. But SC2 doesn't work that way a cheese/all-in reduces it to exactly one decision, you either scout it in time and win or you lose. The ability to force or deny a map pick is more subtle, but all these things heavily favor the winner of the first match without any justification to claim he's the better player.
If they played 2 bo3 in a row, yes it would be dumb, but they don't. They meet again several rounds later in the tournament with exactly the same match record, which resets their individual win/loss back to 0:0, because that's the rule for everyone in the tournament and there can't be exceptions or else it won't be fair to all players.
On November 12 2010 22:17 scion wrote:
He is NOT using this model for an argument. It whether you think flawed or not, does a good job of simplifying how a league works. And he's not using math to show benefit of extended series, he's showing how much benefit it provides.
If you use math to prove a point, then people need to be allowed to question&critizise the method. You can't just pull numbers out of a flawed simulation and say. "Look, that's the benefit it will have" and ignore everything else, because that isn't more scientific than claiming evolution doesn't exist because the bible says so.^^
So, the simulation either gets adjusted for these issues or the conclusion needs to be changed to something that makes clear that this simulation only works for generic tournaments and can't be applied to SC2, because of these issues.
Or else people will take these numbers as truth and assume that the extended series works fine, which it doesn't.
The only good thing about ES is, that most players are probably not aware of their own skill level and don't recognize that they just were lucky in the first match. And since only mlg has this rule, they probably haven't thought about how powerful a instant 2:0 lead in a bo7 is.
But of course that doesn't makes ES any better, because the claimed minimal increase in result accuracy doesn't make up for the risk of abuse/false-positives..it just isn't worth it and messes up the fairness of the tournament for everyone that wasn't favored in a ES. And if a tournament isn't fair then there is no point to have one at all.
I'd rather have the best player in the world kicked out once in a while instead of frequently getting weak players finishing better than they should. The initial seeding already causes the bracket to be a bit more unfair to players at times, why introduce even more luck into the tournament to make it even worse? A clean bo5 is good if you want higher accuracy between 2 players, but a extended series that resumes the first bo3 is just bad in SC2.
|
On November 13 2010 02:07 Nienordir wrote:
But of course that doesn't makes ES any better, because the claimed minimal increase in result accuracy doesn't make up for the risk of abuse/false-positives..it just isn't worth it and messes up the fairness of the tournament for everyone that wasn't favored in a ES. And if a tournament isn't fair then there is no point to have one at all.
I'd rather have the best player in the world kicked out once in a while instead of frequently getting weak players finishing better than they should. The initial seeding already causes the bracket to be a bit more unfair to players at times, why introduce even more luck into the tournament to make it even worse? A clean bo5 is good if you want higher accuracy between 2 players, but a extended series that resumes the first bo3 is just bad in SC2.
The beauty of this post is then, from your perspective, that ES doesn't help much anyway, so we should get rid of it.
|
idra and incontrol were arguing against extended series, but they actually didn't realize that their arguments contradicted each other while they verbally agreed with each other.
idra explained how because if player A screwed up against player B, and player B screws up against player C, why should player A be penalized when playing player B in the losers' bracket? But then incontrol said that each of these Bo3's are isolated events. Any previous result should not affect the current Bo3. When you look at these two arguments, they clearly contradict each other as idra uses events elsewhere in the tournament as an argument, while incontrol states that each Bo3 is isolated.
Sorry, I just wanted to throw that out there lol
|
On November 13 2010 02:20 cHaNg-sTa wrote:
idra and incontrol were arguing against extended series, but they actually didn't realize that their arguments contradicted each other while they verbally agreed with each other.
idra explained how because if player A screwed up against player B, and player B screws up against player C, why should player A be penalized when playing player B in the losers' bracket? But then incontrol said that each of these Bo3's are isolated events. Any previous result should not affect the current Bo3. When you look at these two arguments, they clearly contradict each other as idra uses events elsewhere in the tournament as an argument, while incontrol states that each Bo3 is isolated.
Sorry, I just wanted to throw that out there lol
You're an idiot. These statements don't contradict each other at all...
|
On November 13 2010 02:13 nzb wrote:Show nested quote +On November 13 2010 02:07 Nienordir wrote:
But of course that doesn't makes ES any better, because the claimed minimal increase in result accuracy doesn't make up for the risk of abuse/false-positives..it just isn't worth it and messes up the fairness of the tournament for everyone that wasn't favored in a ES. And if a tournament isn't fair then there is no point to have one at all.
I'd rather have the best player in the world kicked out once in a while instead of frequently getting weak players finishing better than they should. The initial seeding already causes the bracket to be a bit more unfair to players at times, why introduce even more luck into the tournament to make it even worse? A clean bo5 is good if you want higher accuracy between 2 players, but a extended series that resumes the first bo3 is just bad in SC2. The beauty of this post is then, from your perspective, that ES doesn't help much anyway, so we should get rid of it.
Well, yeah it's a fancy way to say it, but I think my post explains the flaws of the extended series if you apply it to a game like SC2. =)
But I don't mind the idea that's behind the extended series. It's in everyones interest to have the best players go far into a tournament. If 2 players meet again, then increase the amount of games played to bo5 or bo7 if the tournament schedule allows it and we'll get a result with higher accuracy.
At the same time I would propose a motion to change the grand finals too and make it a straight up bo5/7, because both players earned their way throught the tournament and the extended series there is even worse than inside the rest of the tournament. There is no reason why the player from the winners bracket should lose two matches, because it's the grand final. The losers bracket ends there and there isn't a losers grand final that the player could drop down to..that just doesn't make sense and you don't want to get the finals messed up by the flaws that ES inherits.^^
|
Fucking excellent OP. I'm sorry for the profanity, but seriously... phenomenal. Wish more people were top-notch analysts, but I'm sure with examples like this some will get better.
|
|
|
|
|
|
EPT
