|
motbob
United States12546 Posts
tl;dr: In double elimination brackets, a 1-0 advantage in Grand Finals for the team coming from Winner's increases the chance of the better team winning the tournament.
In Dota, double-elimination brackets are almost always used, and the grand finals are almost always Bo5. Tournaments have not agreed, however, on whether to give teams from the Winner's Bracket an advantage in Grand Finals. For example, when Alliance and Na`Vi played in the TI3 finals, the series started 0-0, but when those two teams played in Starladder Season 8, Alliance started up 1-0 because they came from the Winner's Bracket.
I'm under the impression that spectators don't like the 1-0 start, but some tournaments (D2CL and Starladder most notably) employ it nonetheless.
Being a massive nerd, I have these various brackets simulated in Excel, so I decided to do some tests and try to test how, in theory, a winner's bracket advantage affects the tournament outcome.
The best team doesn't always win a tournament. Dota is a game with a lot of variance involved, and it only takes a glance at Dota2lounge bet odds to see that. There is a 100% chance that Secret is a better team than M5, but the odds of Secret winning against M5 are not 100%. Nor is the chance of Secret winning a tournament against 7 other scrub teams 100%.
I think that an implicit goal of tournament organizers is to create a format where the best team has a good chance to win. Spectators generally want this. An uproar would surely result if a tournament advanced the second place team to bracket, rather than the first place team, or made the Grand Finals a Bo1. A caveat: spectators want to see good teams earn the win, which is probably why 1-0 advantages leave a bad taste in their mouths.
So if tournament organizers want to create a tournament format where the best team wins most often, spectators be damned, they should create a simulated bracket with teams assigned Elo values (representing "true" skill), run the simulation 10,000 times, and see how many times the best team won with (1) a 1-0 advantage in Grand Finals and (2) no advantage! Or let me do it.
First, I simulated a bracket with two good teams and a bunch of scrubs (1500 Elo, 1480, and a bunch of 1300s). The best team won 51.6% of the time without a Grand Finals advantage, and 52.2% with a 1-0 advantage. That's a 0.6% increase. (Note that the only number we really care about is the increase.)
Second, I simulated an Elo distribution that resembled TI4, meaning that there were a few teams clustered near the top and some semi-competitive teams just afterwards. Here we saw an increase of 1.7% in the best team's win chance from no advantage to 1-0 advantage.
Third, I simulated a very steady drop in Elo (1500, 1490, 1480, 1470...). With this distribution, the best team saw a 1.4% chance increase in winning.
To clarify: one thing to note about the above simulations is that I'm simulating the whole tournament, not the grand finals. In some runs of the simulation where the best team ended up winning, the team lost in Winner's and won GF coming from Loser's. In other runs, the team won Winner's and then won GF.
So with these different distributions of Elo, creating a 1-0 advantage increased the chance of the best team winning the whole tournament. I can't say for sure that that would be true for any combination of teams, but I think that's what these results imply. If y'all want me to test unusual Elo distributions or weird tournament formats (e.g. Bo5 WF instead of Bo3), ask in the comments.
The conclusion I derive from these results is this: if tournament organizers are concerned solely with creating a format where the best team wins, they should have GF with a 1-0 advantage. But the difference between formats seems small enough that, if I were an organizer, I would just keep doing what spectators want (no advantage).
 
|
My thinking for starting 0-0 is:
-Winner team deserves it
-Loser team deserves it anyway because they fell down yet showed great psychological strenght and managed to reach the finals anyways.
As you said, from a spectator point of vie starting 1-0 is meh :/
|
IMHO all e-sports should do what FGC already does and give the player coming from the winners a full match advantage. Although, scheduling and time issues would be a big problem. So I guess stick to 1 game advantage. I think allowing 1 team to drop a series and another not is unfair.
|
On March 14 2015 19:52 SoSexy wrote:
My thinking for starting 0-0 is:
-Winner team deserves it
-Loser team deserves it anyway because they fell down yet showed great psychological strenght and managed to reach the finals anyways.
As you said, from a spectator point of vie starting 1-0 is meh :/
The winning team does not deserve zero advantage.
The losing team doesn't deserve it because they already got their second chance.
The whole point is that you have a DOUBLE elimination bracket in these tournaments... right up to the final game where you suddenly decide that it's single elimination.
That means that all the hard work done by one team to not lose a single series is for nothing, as basically everything resets.
The winner team should have an advantage because they've earned it by not losing.
The "other" way is to have two BoXs, where the losing team has to win both, the winning team only has to win one. That's the true double elimination right up to the end of the competition. What's so hard about just using that method?
What impact does that also have on your calculations for differences, since that's the REAL way to complete a double elim tournament?
|
I'm more curious about how often the team from the Upper (winner's) bracket won the GF with a 1-0 lead compared to 0-0. - to me that's more important. (Why reward the team that's slightly better "on paper" than the team that's possibly already beat them.)
If it's just about "setting up for the best team to win" isn't a seeded single elimination bracket best?
|
On March 14 2015 20:38 Lonyo wrote:Show nested quote +On March 14 2015 19:52 SoSexy wrote:
My thinking for starting 0-0 is:
-Winner team deserves it
-Loser team deserves it anyway because they fell down yet showed great psychological strenght and managed to reach the finals anyways.
As you said, from a spectator point of vie starting 1-0 is meh :/ The winning team does not deserve zero advantage. The losing team doesn't deserve it because they already got their second chance. The whole point is that you have a DOUBLE elimination bracket in these tournaments... right up to the final game where you suddenly decide that it's single elimination. That means that all the hard work done by one team to not lose a single series is for nothing, as basically everything resets. The winner team should have an advantage because they've earned it by not losing. The "other" way is to have two BoXs, where the losing team has to win both, the winning team only has to win one. That's the true double elimination right up to the end of the competition. What's so hard about just using that method? What impact does that also have on your calculations for differences, since that's the REAL way to complete a double elim tournament?
This used to be done in some foreign BW tournaments. While I agree that this method would be the most fair for the team coming from the WB finals, it has a few glaring faults, which is generally why tournaments opt to not employ it and instead recompense the team with a 1-0 advantage.
First of all, it takes a long, long time. Potentially forcing the teams to play 8 games (assuming a bo3 and a bo5), which for dota would mean a grand finals which could easily span the better part of 11-12 hours. To be honest, now that I think about it, knowing dota tournaments it would probably take 2 days at least.
Taking that into consideration one can interpolate that it would probably also have a negative impact on the viewership/ad revenue to cost of the event ratio.
While it's desirable from a purely competitive standpoint, the logistical problems it'd pose to play that many additional games usually just make it so that tournament organisers shy away from it.
|
United States24495 Posts
motbob can you also run simulations with single elimination? It would be interesting to see how the two double-elimination formats above compare to single elimination in odds of the best team winning the tournament.
|
motbob
United States12546 Posts
Kupon and I had a nice discussion on LiquidDota about these simulations. He pointed out that, if teams have very different adaptation capabilities during a tournament, my definition of "best team" becomes questionable. Is the best team the team which started out with the best value, or the team that adapted to the "tourney meta" (especially important at TI/DAC) and performed the best at the end?
Kupon recommended that I change the simulation to reflect this possibility. It turns out that with a dramatic adaptation variable (teams have a 50% of being either "good" or "bad" adapters, gaining a constant 20 or 5 points per round, respectively, with 5 rounds), a 1-0 advantage system does hurt the best team's chance of winning if the best team is defined as the team with the highest initial Elo and also the 20 point adaptation. A lower adaptation variable (2.5/1) resulted in the "best team," similarly defined, benefiting from the 1-0 advantage.
|
motbob
United States12546 Posts
On March 14 2015 23:19 micronesia wrote:
motbob can you also run simulations with single elimination? It would be interesting to see how the two double-elimination formats above compare to single elimination in odds of the best team winning the tournament.
With 8 teams spaced 20 Elo apart each, it's a 2-3% difference between single and double elim.
|
Are these changes significant? And how did you test?
|
Double elimination has quite a few problems and this is one of them, almost all real sports use a combination of round robin and single elimination and the only exception I can think of is college baseball.
There's also the problem of the 4-player group where player A beat player B, player B went 1-1 with a winning record in matches over player C, player C beat player D, and player A beat player D.
A > B > C > D and A and C advance.
Also in large bracket the player coming from the loser's bracket can end up playing twice as many games as the winner's bracket player, this creates a huge disparity in player fatigue.
|
On March 14 2015 23:44 motbob wrote:
Kupon and I had a nice discussion on LiquidDota about these simulations. He pointed out that, if teams have very different adaptation capabilities during a tournament, my definition of "best team" becomes questionable. Is the best team the team which started out with the best value, or the team that adapted to the "tourney meta" (especially important at TI/DAC) and performed the best at the end?
Kupon recommended that I change the simulation to reflect this possibility. It turns out that with a dramatic adaptation variable (teams have a 50% of being either "good" or "bad" adapters, gaining a constant 20 or 5 points per round, respectively, with 5 rounds), a 1-0 advantage system does hurt the best team's chance of winning if the best team is defined as the team with the highest initial Elo and also the 20 point adaptation. A lower adaptation variable (2.5/1) resulted in the "best team," similarly defined, benefiting from the 1-0 advantage.
How about if the best team is defined as the one with the highest ELO after?
|
Cascadia1753 Posts
Yea, I also don't understand why double elim brackets end with a bo5 instead of two bo3s. It only changes scheduling in the worst case, and gives consistency across the entire bracket..
Mind running a simulation for that?
|
It changes the schedule from 3 to 5 games to 2 to 6 games. That's a lot. Plus people don't like it for the same reason they dislike the 1 game advantage.
|
I'm not entirely sure I understood your point. There are a few things I really can't grasp. Anyhow, don't try to sound smug here, my statistics knowledge is more than just limited and I'm not that great when it comes to mathematics.
First off, I don't really get the question behind it. Imo it doesn't matter what kind of mode you use for a tournament, the assumption that there is a "best" team will tell you that this best team will win more often than any other team, as long as the circumstances are even for all teams. That's like trivial. It should also be somewhat obvious that longer distances, in theory, support the better team.
Now you take ELO as measurement of skill, which in itself sounds kind of overcomplicated. Why not just align values from 0 (worst team in the tour) to 1 (best team). Basically, that's the idea, no?
Might be my mathematics being strange.
However, related to that point, I don't think the changes in the outcome of what you tried to calculate have any meaning to them. The distances in skill are arbitrary. I'm not even sure anyone could tell you what a difference of 10 points on the ELO scale would mean - for your tournament, for the entire player/team base or anything. You can only losely relate gaps in such a ranking. That being said, a change in the outcome of win% per mode in the range of 0.x - 2% seem... I don't know. Not much? Especially without T-Test behind it.
|
Hong Kong9148 Posts
Let's put aside Elo chess assumptions being set up in a sample size as small as a one-off tournament in games that are not-chess not being reliable at all and go with this: you note an increase of .6%
That doesn't sound statistically significant, even with your highest stated increase. You do no testing to show whether it is. We have you rejecting the null hypothesis here without actually giving a good reason why.
Edit: sniped by gecko. hi gecko.
|
On March 15 2015 07:14 itsjustatank wrote:
...you note an increase of .6% That doesn't sound statistically significant, even with your highest stated increase.
This was exactly my thought as I finished reading the OP, however I strongly believe that this topic warrants further testing and discussion because there is obviously dissension about whether the 1-0 advantage is necessary. The real question is"What is the real motivation behind the 1-0 advantage? Is it really to help the better team win or, as the OP suggested, is it actually beneficial because of the way the brackets and numbers work out?" Hopefully Motbob can hammer away and help us plebs figure out what's what.
|
motbob
United States12546 Posts
I don't think there's any dissension here. If you read anything in the post, you should have read the conclusion: if I were a tournament organizer, I would stick with no advantage.
|
Hong Kong9148 Posts
Your generated Elo predictions based on arbitrary distribution choices resulted in differences that do not seem statistically significant. You do no test to prove that they are statistically significant, you just give the differences in observed percentages.
Null hypothesis: there is no statistically significant difference between starting a double-elimination finals 1-0 versus 0-0
Alternate hypothesis: there is a statistically significant difference between starting a double-elimination finals 1-0 versus 0-0
You have not proven whether or not what you got is noise and whether or not there really is a difference between a 1-0 start and a 0-0 start. You just want one of the two, clearly, and think this is enough to want to make a change.
Your argument is completely non-falsifiable right now. Sure, it may work for the internet, but unless you do that extra work you are pissing in the wind with a cloak of statistics making your advocacy look smart to people who do not know what they are reading.
|
I still don't get it or why you needed math to make a point.
Like you start out with something like this:
- You have a team, which is better than any other team participating
- This team therefore wins with a higher likelihood against any other team
- If the gap in between the "true skill" is not that large, the distances / modes a tournament uses gets important
That's somewhere in the blog already as far as I understood. What's left out is:
- The longer distances (Bo3 vs. Bo9 etc) are, the more certain (? sry, English) it is the better team will win within one tournament
- If every team plays exactly the same modes, the better team, under the assumption skill won't ever change, will win in more tournaments if you look at enough samples
Now something happens in your trail of thought. E.g. you want to ensure the best team wins, for whatever reasons possible. You entirely miss however, that as long as you don't drastically introduce one sided changes, any mode will support the best team already.
Like, it should be kind of obvious with a 1-0 advantage, that:
- the best team will advance through the WB to the Grand Finals more often and therefore more often starts with a 1-0 lead
- even in cases they need their second chance via the LB route to Grand Finals the better team has a somewhat larger chance to win with a 0-1 disadvantage
Grant you, it'd be propably interesting, from a very theorycrafting point of view, how much influence this 1-0 has. However, you will never know, even if you test your results (the differences you list). Why you already explained:
- You can not possibly meassure skill
- All indicators for skill do not tell you how much better a team is, even indirectly via ELO. There's always a large margin of error involved, those estimators operate with them. Hence, the statements like "twice as good" are just your very subjective view on that matter
Hence, it's not really suprising that your results mostly tell you that the better team wins more likely. That's all I could learn in what you wrote.
Disregard all that, it'd probably comes down to other points. People already pointed out that a DE format is designed to give a second chance. Therefore the only logical follow-up is to set up the Grand Finals as 0-0 and double Best of X. If the LB Team wins, they have to endure a second Grand Final Best of X - because the WB Team never got a second chance.
Since this takes much time - as pointed out - the 1-0 lead is in place, depending on the game. Setting it entirely to 0-0 is - tournament design wise - just silly.
Btw, if you're interested in the topic itself, try to google for interviews of Barry Hearn and the PTC Snooker series. He changed tons of professional billard tournaments to shorter distances (iirc Bo9-Bo17 to Bo7 only). He tries to explain why that is - without any math - and just summarizes it as: "it's the only way to get all games done in a short time frame".
|
|
|
|
|
|
EPT
