|
motbob
United States12546 Posts
I am not sure that MLG's extended series provision does a good job of allowing the best player to win. In fact, my simulations show that it does exactly the opposite.
MLG_Lee once told me a story that explained the philosophy behind extended series. At a Halo event, he said, one of the best teams demolished an up-and-coming team 2-0 or 3-0 in the winners' bracket. The two teams met again in the losers' bracket, and the worse team eked out a close win against the favorites. Extended series, Lee said, would have prevented that from happening.
If MLG's goal is to help shepherd the best player to victory, extended series, in some (and perhaps all) cases, do not achieve that goal. If MLG's goal is to help the better player to win just in those matches where they have already played their opponent, extended series, in some (and perhaps all) cases, do not achieve that goal. If MLG's goal isn't either of those two things, what is it?
I made an Excel workbook that simulates a major part of MLG's 2012 tournaments: an 8-player group that features double elimination and extended series. MLG's Spring, Summer, and Fall tournaments each featured these types of groups, and those groups might make a return in Columbus after the fluke formats of 2013 Winter and Spring. I ran Monte Carlo simulations (that is, simulating the group over and over again thousands of times) in this workbook.
Whether the players were close or distant in skill... whether the players were seeded by skill or randomly seeded... whether there was a single dominant player or a more even field... in all these cases, the simulations all came up with the same result: Extended series make it less, not more, likely for the better player to win the bracket.
That's not all. I ran a different set of simulations: If the first- and second-best players in the group met up in the losers' finals, would the best player be more likely to win in a bracket with extended series or one without? Again, the result of the simulations run counter to MLG's wishes: Extended series resulted in the better player being knocked out more often, not less often. Sometimes, the extended series rule worked in the best player's advantage exactly as MLG had designed, allowing them to close out the match against an inferior foe. But other times, the winners' bracket match was a fluke, with the worse player winning -- and in the rematch, that fluke carried over, giving the worse player a better chance to advance! A real world example of this would be ToD vs Ryung in an MLG qualifier, where ToD was probably the worse player of the two. ToD won in the winners' bracket 2-1, and in the rematch managed to finish Ryung off 4-3.
Typically, the difference in the rate of the best player winning the bracket was 0.5% - 1.0%. Fluctuations here were, I assume, due to ELO distribution. The difference in winrate in "would the best player win in this particular matchup" tests was 0.2%-0.8%. In the second case, fluctuations came from how often an extended series match was played: If the first- and second-best players were matched up in the first round, thus guaranteeing that an extended series would be played in a losers' bracket matchup, then the difference between extended series and non-extended series formats was relatively large.
Note that this is NOT definite proof that extended series do not achieve MLG's goals in all situations. I am not going to simulate a 128-player bracket, and I didn't simulate all possible combinations of ELOs or player seeding. However, I'm not sure why things would be much different in a larger bracket, especially since most extended series in MLG events happen near the end of the bracket.
If extended series do not achieve what MLG intends them to achieve, what good are they?
You can see the excel workbook here. (Please don't look at the formula I used to pull ELOs in column B. Yes, I know I could have just used INDEX/MATCH.)
+ Show Spoiler [how the excel spreadsheet works] +Player ELOs are input. Players play winners' bracket matches, and match results are recorded in the table at the top right. Later, in losers' bracket matches, it is determined whether an extended match is appropriate (cell A65, for example, serves this purpose) and, if an extended series needs to be played, an initial score for the match is called. This pattern continues to the grand finals, where MLG's weird grand finals format is appropriately simulated.
  
|
So, wait, did you use truly random numbers in the simulation?
|
motbob
United States12546 Posts
The RAND function is pseudo-random, but it is fine for my purposes. Older versions of RAND in Excel were not appropriate for Monte Carlo simulations, but Excel 2007 improved it.
http://support.microsoft.com/kb/828795
|
Hey, cool stuff.
Im just curious what alternative did you use instead of extended series? Just a normal Bo3?
|
motbob
United States12546 Posts
On July 22 2013 23:44 FlubberMan wrote:
Hey, cool stuff.
Im just curious what alternative did you use instead of extended series? Just a normal Bo3?
The alternative was what would have been played under MLG rules if the two players hadn't previously played.
|
That's very interesting! Thanks for posting this.
|
On July 22 2013 23:51 motbob wrote:Show nested quote +On July 22 2013 23:44 FlubberMan wrote:
Hey, cool stuff.
Im just curious what alternative did you use instead of extended series? Just a normal Bo3? The alternative was what would have been played under MLG rules if the two players hadn't previously played.
Okay thanks for the quick answer. Im not sure what the MLG rules are but I think somehow you have to have some kind of rule that give an advantage to the guy in the winnersbracket playing against someone from the loosers bracket, no matter what makes the best player win the tournament more often.
|
Extended series has to be the most overblown, ridiculous controversy in all of StarCraft2.
I understand the argument that the extra matches you have to play to get through the loser bracket is punishment enough but I think the scenario of Player A losing 0-2 the first time and then winning 2-1 the second time and advancing with the 2-3 total is suboptimal.
Excuse me while I go watch NaNiwa vs Flash Game 7.
|
Correct me if I'm wrong (I may be missing some complexity or nuance here), but the following is true, and is independent of whether or not the series is continuous or broken into a Bo3 and extended into a Bo7.
If player A is better than B, say having a 51% chance of winning any single game vs. player B, then the more games they play, the law of large numbers will push the winrate of each player towards the projected underlying probability. Which is to say that smaller samples are noisier than large ones.
So, using a simple binomial distribution with the set of binomial parameters:
% chance of success (probability that the better player will win):
# successes needed (required # of wins in a series):
# trials (# of games in a series)
The probability of the better player winning always goes up with a larger set of games.
For example, with the initial binomial parameters (for a Bo3 series with 2 extremely evenly matched players):
% chance of success (probability that the better player will win): 51%
# successes needed (required # of wins in a series): 2
# trials (# of games in a series): 3
The better player wins that series 51.499% of the time.
If the series is extended to 7 games however:
% chance of success (probability that the better player will win): 51%
# successes needed (required # of wins in a series): 4
# trials (# of games in a series): 7
The better player wins 52.19% of the time.
For larger discrepancies between skill level (70% to 30% chance of winning between players), the dichotomy between short and long series increases even further:
% chance of success (probability that the better player will win): 70%
# successes needed (required # of wins in a series): 3 (short), 7 (long)
# trials (# of games in a series): 2 (short), 4 (long)
The better player wins the short series 78.4% of the time, and the long series 87.4% of the time.
This trend is also independent of how the series is broken up. For example, using our 70/30 split again. Here are the chances of certain outcomes in a Bo3 series:
Better player wins Bo3: 78.4%
Better player wins 2-0: 49%---------------------------------->Better player goes on to win Bo7 extended series 96.9% of the time
Better player wins 2-1: 29.4%-------------------------------->Better player goes on to win Bo7 extended series 91.6% of the time
Worse player wins Bo3: 21.6%
Worse player wins 2-0: 9%------------------------------------>Better player goes on to win Bo7 extended series 52.8% of the time
Worse player wins 2-1: 12.6%------------------------------->Better player goes on to win Bo7 extended series 65.2% of the time
Using the % chance of each Bo3 result to weight the Bo7 results:
87.4% chance of the better player winning the Bo7 extended series overall, which is identical to the result for continuous Bo7 series
|
motbob this spreadsheet is so big it's hard to figure out what's going on. Isn't is sufficient to just simulate 2 players meeting in the losers bracket? Extended vs not.
|
We aren't looking for a continuous Bo7 series though. If Player B beats Player A 2-1, they play again... player B starts as 2-1 so only needs to win 2 games, where Player A has to win 3 games. Compare that to Player B beats player A 2-1, they play again in another Bo3. Either one only needs to win 2 games. So if player A is the better player, he would have a better chance at winning only 2 games, rather than needing to win 3. Simple.
|
First let me explain why 'extended series' exists:
Players A and B meet in the winner's bracket and losers bracket.
Scenario 1:
Player A wins 2-0 in winner's bracket. Player B wins 2-1 in loser's bracket rematch. Player A leads 3-2 overall in maps, but Player B advances. I consider this unfair.
Scenario 2;
Player A wins 2-0 in winner's bracket. Player B wins 2-0 in loser's bracket rematch. The series is a tie 2-2 overall in maps, but Player B advances solely based on the order of the matches. A won earlier and B won later, but B advances. Again, I consider this unfair.
Now allow for extended series. This resolves Scenario 1 and 2 by forcing either Player A or B to win in map score. Therefore, if you think Scenario 1 & 2 are unfair, like I do, then there should be a rule to resolve the problem. MLG uses extended series to solve this problem.
With regards to the simulation in the OP:
I think what you are saying is, "Better players lose the extended series when they are playing from the 'behind' position." This is a function of win rates. If you use win rates hovering around 50% for both players (typical Starcraft 2 win rates), then the discreteness of being behind 1 or 2 games entering the extended series will frequently result in an overall loss for the 'better' player.
E.g. Player A and Player B have a 48/52 win rate split. Player A finishes the winner's bracket series up 2-1. Now they meet in an extended series. Player A has to win 2 games and Player B needs to win 3 games. Given the win rates, it is no surprise that once the results of the original series are revealed we can easily predict that Player B will still lose the series more often than Player A loses the series (even though Player B is objectively better!)
TLDR: Better players not coming from behind to survive through extended series has nothing to do with why extended series has been implemented; to resolve unwanted map score advancement scenarios.
|
You are ignoring the fact of # of series lost. One player, even with better map score, has lost two series. Player B has only lost one. We aren't looking for the best player vs player B, but the best player, which takes into account all of the matches. The same thing happens in GSL.
Player A 2-0 Player B
C 2-0 D
C > A
B > D
B 2-1 A
Does A still deserve to advance just because he has a better map score vs player B? He still lost one more series than B did, therefore as an overall player he is not as skilled on that day.
|
Hong Kong9165 Posts
On July 23 2013 01:15 KissMeRed wrote:
With regards to the simulation in the OP:
I think what you are saying is, "Better players lose the extended series when they are playing from the 'behind' position." This is a function of win rates. If you use win rates hovering around 50% for both players (typical Starcraft 2 win rates), then the discreteness of being behind 1 or 2 games entering the extended series will frequently result in an overall loss for the 'better' player.
E.g. Player A and Player B have a 48/52 win rate split. Player A finishes the winner's bracket series up 2-1. Now they meet in an extended series. Player A has to win 2 games and Player B needs to win 3 games. Given the win rates, it is no surprise that once the results of the original series are revealed we can easily predict that Player B will still lose the series more often than Player A loses the series (even though Player B is objectively better!)
TLDR: Better players not coming from behind to survive through extended series has nothing to do with why extended series has been implemented; to resolve unwanted map score advancement scenarios.
it's not observable win rates, but an inference based on Elo values
|
On July 23 2013 01:15 KissMeRed wrote:
First let me explain why 'extended series' exists:
Players A and B meet in the winner's bracket and losers bracket.
Scenario 1:
Player A wins 2-0 in winner's bracket. Player B wins 2-1 in loser's bracket rematch. Player A leads 3-2 overall in maps, but Player B advances. I consider this unfair.
Scenario 2;
Player A wins 2-0 in winner's bracket. Player B wins 2-0 in loser's bracket rematch. The series is a tie 2-2 overall in maps, but Player B advances solely based on the order of the matches. A won earlier and B won later, but B advances. Again, I consider this unfair.
Now allow for extended series. This resolves Scenario 1 and 2 by forcing either Player A or B to win in map score. Therefore, if you think Scenario 1 & 2 are unfair, like I do, then there should be a rule to resolve the problem. MLG uses extended series to solve this problem.
With regards to the simulation in the OP:
I think what you are saying is, "Better players lose the extended series when they are playing from the 'behind' position." This is a function of win rates. If you use win rates hovering around 50% for both players (typical Starcraft 2 win rates), then the discreteness of being behind 1 or 2 games entering the extended series will frequently result in an overall loss for the 'better' player.
E.g. Player A and Player B have a 48/52 win rate split. Player A finishes the winner's bracket series up 2-1. Now they meet in an extended series. Player A has to win 2 games and Player B needs to win 3 games. Given the win rates, it is no surprise that once the results of the original series are revealed we can easily predict that Player B will still lose the series more often than Player A loses the series (even though Player B is objectively better!)
TLDR: Better players not coming from behind to survive through extended series has nothing to do with why extended series has been implemented; to resolve unwanted map score advancement scenarios.
well said.
on a different note i never actually thought that extended series was used to resolve potential map score issues. The rules makes a little more sense now. However, I still dislike it from a spectator point of view.
|
Hong Kong9165 Posts
On July 23 2013 02:05 Theberlinwall wrote:Show nested quote +On July 23 2013 01:15 KissMeRed wrote:
First let me explain why 'extended series' exists:
Players A and B meet in the winner's bracket and losers bracket.
Scenario 1:
Player A wins 2-0 in winner's bracket. Player B wins 2-1 in loser's bracket rematch. Player A leads 3-2 overall in maps, but Player B advances. I consider this unfair.
Scenario 2;
Player A wins 2-0 in winner's bracket. Player B wins 2-0 in loser's bracket rematch. The series is a tie 2-2 overall in maps, but Player B advances solely based on the order of the matches. A won earlier and B won later, but B advances. Again, I consider this unfair.
Now allow for extended series. This resolves Scenario 1 and 2 by forcing either Player A or B to win in map score. Therefore, if you think Scenario 1 & 2 are unfair, like I do, then there should be a rule to resolve the problem. MLG uses extended series to solve this problem.
With regards to the simulation in the OP:
I think what you are saying is, "Better players lose the extended series when they are playing from the 'behind' position." This is a function of win rates. If you use win rates hovering around 50% for both players (typical Starcraft 2 win rates), then the discreteness of being behind 1 or 2 games entering the extended series will frequently result in an overall loss for the 'better' player.
E.g. Player A and Player B have a 48/52 win rate split. Player A finishes the winner's bracket series up 2-1. Now they meet in an extended series. Player A has to win 2 games and Player B needs to win 3 games. Given the win rates, it is no surprise that once the results of the original series are revealed we can easily predict that Player B will still lose the series more often than Player A loses the series (even though Player B is objectively better!)
TLDR: Better players not coming from behind to survive through extended series has nothing to do with why extended series has been implemented; to resolve unwanted map score advancement scenarios. well said.
except he's wrong.
|
On July 23 2013 02:11 itsjustatank wrote:Show nested quote +On July 23 2013 02:05 Theberlinwall wrote:On July 23 2013 01:15 KissMeRed wrote:
First let me explain why 'extended series' exists:
Players A and B meet in the winner's bracket and losers bracket.
Scenario 1:
Player A wins 2-0 in winner's bracket. Player B wins 2-1 in loser's bracket rematch. Player A leads 3-2 overall in maps, but Player B advances. I consider this unfair.
Scenario 2;
Player A wins 2-0 in winner's bracket. Player B wins 2-0 in loser's bracket rematch. The series is a tie 2-2 overall in maps, but Player B advances solely based on the order of the matches. A won earlier and B won later, but B advances. Again, I consider this unfair.
Now allow for extended series. This resolves Scenario 1 and 2 by forcing either Player A or B to win in map score. Therefore, if you think Scenario 1 & 2 are unfair, like I do, then there should be a rule to resolve the problem. MLG uses extended series to solve this problem.
With regards to the simulation in the OP:
I think what you are saying is, "Better players lose the extended series when they are playing from the 'behind' position." This is a function of win rates. If you use win rates hovering around 50% for both players (typical Starcraft 2 win rates), then the discreteness of being behind 1 or 2 games entering the extended series will frequently result in an overall loss for the 'better' player.
E.g. Player A and Player B have a 48/52 win rate split. Player A finishes the winner's bracket series up 2-1. Now they meet in an extended series. Player A has to win 2 games and Player B needs to win 3 games. Given the win rates, it is no surprise that once the results of the original series are revealed we can easily predict that Player B will still lose the series more often than Player A loses the series (even though Player B is objectively better!)
TLDR: Better players not coming from behind to survive through extended series has nothing to do with why extended series has been implemented; to resolve unwanted map score advancement scenarios. well said. except he's wrong.
thanks for the explanation
|
On July 23 2013 02:11 itsjustatank wrote:Show nested quote +On July 23 2013 02:05 Theberlinwall wrote:On July 23 2013 01:15 KissMeRed wrote:
First let me explain why 'extended series' exists:
Players A and B meet in the winner's bracket and losers bracket.
Scenario 1:
Player A wins 2-0 in winner's bracket. Player B wins 2-1 in loser's bracket rematch. Player A leads 3-2 overall in maps, but Player B advances. I consider this unfair.
Scenario 2;
Player A wins 2-0 in winner's bracket. Player B wins 2-0 in loser's bracket rematch. The series is a tie 2-2 overall in maps, but Player B advances solely based on the order of the matches. A won earlier and B won later, but B advances. Again, I consider this unfair.
Now allow for extended series. This resolves Scenario 1 and 2 by forcing either Player A or B to win in map score. Therefore, if you think Scenario 1 & 2 are unfair, like I do, then there should be a rule to resolve the problem. MLG uses extended series to solve this problem.
With regards to the simulation in the OP:
I think what you are saying is, "Better players lose the extended series when they are playing from the 'behind' position." This is a function of win rates. If you use win rates hovering around 50% for both players (typical Starcraft 2 win rates), then the discreteness of being behind 1 or 2 games entering the extended series will frequently result in an overall loss for the 'better' player.
E.g. Player A and Player B have a 48/52 win rate split. Player A finishes the winner's bracket series up 2-1. Now they meet in an extended series. Player A has to win 2 games and Player B needs to win 3 games. Given the win rates, it is no surprise that once the results of the original series are revealed we can easily predict that Player B will still lose the series more often than Player A loses the series (even though Player B is objectively better!)
TLDR: Better players not coming from behind to survive through extended series has nothing to do with why extended series has been implemented; to resolve unwanted map score advancement scenarios. well said. except he's wrong.
Country A plays Country B in world cup pool play. Country A beats Country B 2-0. They meet again in knockout stages, Country A starts with 2-0 advantage having already beaten Country B by that margin. This sounds fair right?
|
On July 23 2013 01:15 KissMeRed wrote:
First let me explain why 'extended series' exists:
Players A and B meet in the winner's bracket and losers bracket.
Scenario 1:
Player A wins 2-0 in winner's bracket. Player B wins 2-1 in loser's bracket rematch. Player A leads 3-2 overall in maps, but Player B advances. I consider this unfair.
Scenario 2;
Player A wins 2-0 in winner's bracket. Player B wins 2-0 in loser's bracket rematch. The series is a tie 2-2 overall in maps, but Player B advances solely based on the order of the matches. A won earlier and B won later, but B advances. Again, I consider this unfair.
Now allow for extended series. This resolves Scenario 1 and 2 by forcing either Player A or B to win in map score. Therefore, if you think Scenario 1 & 2 are unfair, like I do, then there should be a rule to resolve the problem. MLG uses extended series to solve this problem.
With regards to the simulation in the OP:
I think what you are saying is, "Better players lose the extended series when they are playing from the 'behind' position." This is a function of win rates. If you use win rates hovering around 50% for both players (typical Starcraft 2 win rates), then the discreteness of being behind 1 or 2 games entering the extended series will frequently result in an overall loss for the 'better' player.
E.g. Player A and Player B have a 48/52 win rate split. Player A finishes the winner's bracket series up 2-1. Now they meet in an extended series. Player A has to win 2 games and Player B needs to win 3 games. Given the win rates, it is no surprise that once the results of the original series are revealed we can easily predict that Player B will still lose the series more often than Player A loses the series (even though Player B is objectively better!)
TLDR: Better players not coming from behind to survive through extended series has nothing to do with why extended series has been implemented; to resolve unwanted map score advancement scenarios.
Why would games from a diferent stage in the tournament matter? It doesn't matter if you won 2-0 or 2-1 in the groups stage, if you had an easy win or a hard win, if you won in 50mins of 15mins, if you meet the guy again in a later part of the tourney, it's a diferent match that should start from scratch.
A Bo3 is a single match, done in this way to avoid some of the randomness of the game. If you were to count every game as a single match, you would have to always play the 3 games of the series, else you would be giving an advantage to whoever wins earlier, the exact thing you are saying is bad. The odds of winning O-X-O and winning O-O-X should be very similar. By how much you win a match has never been important in any kind of sport, except for tiebreakers.
It's like counting goals scored in football. It can be used as a tiebreaker, the same way games won is used as a tiebreaker, but it is absolutelly never used in diferent stages of the tournament. If Arsenal beats Barça 3-0 in the groups stage and they meet again in the finals, it will obviously start from scratch. Saying that match should start 3-0, or that Barça doesn't deserve the win if they win 1-0 make zero sense.
There's also the issue that you are saying that's the reason it exists, when the MLG guys who implemented it said otherwise.
|
Imagine a tournament with double elimination Bo1, and another tournament with double elimination Bo1 extended into Bo3 in rematches.
Now take two players who meet in the first round and will always meet each other in the loser's bracket regardless of who wins the initial game. In the first tournament, the better player will have x chance to advance, where x is his chance of winning a Bo1 against the worse player.
In the second tournament, if the better player wins the first match, he has (1 - (1-x)^2) chance of advancing. (1 minus the chance of losing 2 Bo1s in a row) If the worse player wins the first match, the better player has x^2 chance of winning the extended series (2 Bo1s in a row)
So for the second tournament, the better player has x*(1 - (1-x)^2) + (1-x)*x^2 chance to advance, based on simple conditional probability. This is larger than x for all x between 0.5 and 1, as seen here:
http://www.wolframalpha.com/input/?i=x*(1 - (1-x)^2) + (1-x)*x^2 = x
It really looks like extended series benefits the better player to me, and I don't see why this would change with Bo3 extended to Bo7.
|
|
|
|
|
|
EPT
