A Critique of TI Scheduling

VGhost

United States3606 Posts

August 14 2017 19:11 GMT

Summary

The International 2017 broke with both precedent and appearance of fairness in two aspects of the group stage: assignment of teams to groups, and schedule of play. Additionally, the schedule on which the Main Stage bracket is played has certain flaws which introduce uneven schedules and thus ought to be addressed in future years.

Personal Interest in the Subject

I began watching DotA regularly after about halfway through the TI6 Main Event, when word of the EG-EHOME series leaked on to the main Team Liquid site where I'd been following Brood War for years. Of course I'd known about DotA's existence, but actually watching it I got hooked immediately as a spectator - though I confess my occasional ventures into the games tutorials have left me mostly baffled.

I was looking forward TI7 with great anticipation: and the games mostly lived up to the hype. Maybe I'll get out to Seattle someday; on stream, though, the presentation of the Main Event was a little lacking. (I specify the Main Event because I quite liked the minimalist approach to the group stage matches.) The opening ceremony was flat; team introductions, information, or bios were presented sporadically or not at all; and the segments run between games did a poor job of showcasing any atmosphere at the event. But it was really the irregularities in scheduling that bothered me most of all.

Brief History of The International's Format

In 2011 The International was introduced with a huge prize pool considering it was essentially a publicity event for a game still in beta. The tournament was played with groups of four teams for seeding followed by a double-elimination bracket. Matches were all single games until the last four, of which the upper and lower bracket finals were played best of three, while the grand final was best of five with advantage. In 2012 and 2013 the group stage was played in groups of eight, the entire upper bracket was played best of three, and the grand final was played without advantage, which has continued to be the standard. In 2013 best of three was also extended one round further back in the lower bracket.

In 2014, a radically different format was used. Nineteen teams were qualified, of which the last four were placed in a play-in bracket to eliminate three. The remaining sixteen teams then played a single group, of which the last six were eliminated, the next four played each other for elimination and then against the next four for seeding, with the top two automatically seeded, all into an eight-team double-elimination bracket. Group matches were best of one, while all bracket play was best of three except the grand final which was best of five.

Current Format of The International

The format of The International has been essentially the same for three years now. In my comments below I refer only to data and results from these three years, as the previous arrangements of the tournament are not readily comparable.

Two round-robin groups are played for seeding into a sixteen-team double-elimination bracket: all matches in the groups are played in two games, with extra tie-breakers played if elimination or upper vs. lower bracket results are required; in other cases tie-breaking criteria are used based on results of matches already played. The tournament begins with 18 teams: in 2015 and 2016, a preliminary play-in bracket was played between the last four qualifiers, while in 2017 all teams were included in one of the round robin groups and the last place team in each did not advance to bracket play.

In bracket play, all matches are best out of three games, except for the lower bracket play-in matches which are a single game each, and the final match which is played best out of five.

Group Assignment and the 2017 Failure

In 2015, ten teams - a majority - were invited. Five were placed in each group, as was one of the wild card teams, with the remaining two teams in each groups being regular qualifiers. In 2016 and 2017, only six teams were invited. In 2016, invited teams and wild card teams were again evenly split, while the three teams which had qualified all the way from open qualifying were split, two in Group A and one in Group B.

However, in 2017, teams' entry status into the tournament was not a deciding factor in group assignment. The six invited teams were split two and four. The two teams qualifying from open play were not split but were placed instead in the same group. Although the groups were "balanced" by placing the open qualifying teams (presumably, though not historically, weaker than the rest) in Group B which had four of the invited teams, the uneven appearance of the groups is a fault in attention to detail.

In fact there would have been no difficulty in constructing groups which provided this attention to all details. Invictus Gaming's main team was invited, while their "junior" squad Vitality also qualified: they were placed in separate groups, but with IG in Group B with three other invited teams. That ought to have been reversed. Similarly, one team which qualified from open qualifying matches, Execration, could easily have been exchanged from the actual groups with either of the other Southeast Asian teams, TNC or Fnatic. (Fnatic, with only one third place result in any significant tournament since the most recent major update, would have been the obvious candidate as being most similar: and in fact Fnatic were the second-worst team at The International this year.)

The problems with the group construction this year, both in terms of actual appearance and non-conformance to precedent, I trust are obvious.

Group Stage Scheduling and the 2017 Failure

Leaving this point alone, I move on to the schedule of games. Despite superficial similarities in terms of number of teams invited and overall format, schedules for group play over the last three years have been erratic. In an unfortunately developing theme, we find this year was by far the worst.

In 2015 and 2016, the wild card play-in was conducted on the first day of offline play in Seattle. In 2015 the resulting 8-team round-robin groups were played over four days (for a total of five days of preliminary play), but in 2016 this was compressed to only three days (for a total of only four days of preliminaries). This condensation of schedule - with the same 56 matches to play - was acheived in 2016 by running four matches simultaneously instead of three. The length of the day scheduled remained the same - five sequential scheduled match times except the last day of play.

In 2015 and 2016, each team played seven matches in the group stage. In 2015, the four-day schedule gave all teams two matches to play on three days and one match on the other. Though teams were sometimes playing back to back and sometimes had a break, a maximum of two matches meant a reasonable schedule.

+ Show Spoiler [Games per Day in Group Stage, 2015] +

Group A
Cloud 9: 2-2-2-1
compLexity: 2-2-1-2
Fnatic: 2-2-1-2
Invictus Gaming: 2-2-2-1
LGD Gaming: 2-2-2-1
MVP Phoenix: 2-2-2-1
Natus Vincere: 2-2-2-1
Secret: 2-2-2-1

Group B
CDEC Gaming: 1-2-2-2
Evil Geniuses: 2-2-2-1
EHOME: 1-2-2-2
Team Empire: 2-2-2-1
MVP Hot6ix: 2-1-2-2
Newbee: 2-1-2-2
Virtus Pro: 2-2-2-1
Vici Gaming: 2-2-2-1

In 2016, the reduction to a three-day schedule created a crush: all teams played at least two matches every day, with Group A teams playing 3-2-2 and Group B playing 2-3-2. All matches were played back-to-back within groups, with Group A matches scheduled before Group B every day. In fact on the days when three matches were played, matches were scheduled back-to-back-to-back with only short between-game breaks.

Still, while the schedule may have been difficult, all teams were treated impartially. The one potential complaint might be with regard to scheduling - but whether playing close to the same time every day is better or worse than alternating schedules to evenly distribute earlier mornings and later nights I have no idea.

In 2017 the addition of the last qualifiers to the groups resulted in 72 matches, requiring that even with four matches played simultaneously, and the five-match-per-day schedule continued, the fourth day of group play had to be used as well. Overall this ought to have been a positive development. With groups of nine teams, and four days to play the group, each team's eight matches should, rationally, be laid out 2-2-2-2. Naturally this would take some attention to detail to schedule cleanly, as it requires at least three match times per day per group: but it should not have been insuperable, as the groups could overlap and only 18 (of 20 total possible) slots per day would be required.

However the schedules were not evenly distributed. Instead, the assignment of matches was noticeably uneven. The basic match schedule aimed for was to give most teams a heavy day with three matches, a light day with only one, and two matches the other days. Twelve of the teams (six in each group) actually got this schedule: the remaining three in Group B were assigned the even 2-2-2-2 schedule, but in Group A two teams were assigned a 3-1-3-1 schedule and the remaining team played 3-2-3-0. In all, four different scheduling patterns were used.

+ Show Spoiler [Games per Day in Group Stage, 2017] +

Group A
Evil Geniuses: 3-1-3-1
Team Empire: 3-2-3-0
Fnatic: 3-1-3-1
IG Vitality: 2-2-3-1
Infamous: 2-2-3-1
LGD Gaming: 3-2-2-1
Team Liquid: 3-2-2-1
TNC Pro Team: 2-2-3-1
Secret: 3-2-2-1

Group B
Cloud 9: 2-2-2-2
Digital Chaos: 2-2-2-2
Execration: 2-3-2-1
Hellraisers: 1-3-2-2
Invictus Gaming: 1-3-2-2
LGD Forever Young: 2-3-2-1
Newbee: 2-2-2-2
OG: 2-3-1-2
Virtus Pro: 2-3-1-2

I am not, as mentioned before, qualified to judge psychological or physiological impacts of different schedules. If one were to look for an immediate impact, it would in any case be difficult to make a judgment, as no correlation exists between schedule and placement in group and then in bracket. But I do find it interesting that of the top six in the Main Event, all but one played a 3-2-2-1 in some arrangement in groups, and the remaining team played 2-2-2-2. If we take the entire top half of the bracket, we add one team with the most common schedule, and one which played the irregular 3-2-3-0.

There is no clear conclusion to draw about the effect of the schedule on teams: but from the point of view of avoiding any dispute or speculation and ensuring fair conditions - and almost as important, the appearance of fair conditions - the schedule this year is a disappointment.

Flaws in the Main Event Schedule

I turn next to a problem which was, sadly, not simply introduced for the first time this year. As a result, it may be much harder to fix due to the event's inertia. The schedule or the "Main Event" has been identical for the past three years. It would admittedly seem difficult to prove any particular effect of the unfair scheduling with only three years of data. But in fact the schedule does have significant built-in inequalities.

The bracket is played out over six days. The schedule for the past three years has been:

Day 1: Upper Bracket Round 1 Matches 1 and 2; Lower Bracket Round 1 Play-In (Bo1)
Day 2: Upper Bracket Round 1 Matches 3 and 4; Lower Bracket Round 2 Matches 1 and 2
Day 3: Upper Bracket Round 2; Lower Bracket Round 2 Matches 3 and 4
Day 4: Lower Bracket Round 3; Lower Bracket Round 4 Match 1
Day 5: Lower Bracket Round 4 Match 2; Upper Bracket Final; Lower Bracket Semifinal
Day 6: Lower Bracket Final; Grand Final (Bo5)

In the upper bracket, the teams winning Round 1 Matches 3 and 4 have only a day to prepare for their next matches, while the teams winning Matches 1 and 2 get two days. Fortunately 1 and 2 play each other while 3 and 4 play each other, and after that the Upper Bracket is equal.

But in the lower bracket, teams winning matches 1 and 2 of the Round 1 Play-In have a single day to prepare for their next opponent, while the teams winning matches 3 and 4 have a two days (though only a day with knowledge of the specific opponent).

Subsequently, Lower Bracket Round 2 winners of matches 1 and 2 have two days to prepare, while the winners of matches 3 and 4 have only one. These are offsetting preparation time advantages if a team has made it through the lower bracket this far - but any teams losing Upper Bracket Round 1 Matches 3 or 4 but winning their lower bracket match are playing repeatedly with only a day to prepare.

After Lower Bracket Round 3, the winner of Match 1 has to immediately play the loser of Upper Bracket Round 2 Match 2 - who have now themselves had to play three days in a row. Meanwhile, the winner of Lower Bracket Round 3 Match 2 has a day to prepare for a known opponent - who also has had a day off between every match.

In the Lower Bracket Semifinal, the team coming from Round 4 Match 1 has had a day to prepare, while the team from Match 2 is playing again the same day.

Results of Schedule

To my surprise, in matches between teams playing with one day to prepare and teams playing with two days, I found no significant effect.

2015: 1-2 (compLexity loss to Virtus Pro), 2-1 (Team Secret over Invictus Gaming), 0-2 (Virtus Pro loss to LGD Gaming)
2016: 0-2 (Alliance loss to Fnatic), 1-2 (Newbee loss to Team Liquid), 2-0 (Fnatic over MVP Phoenix)
2017: 0-2 (TNC Pro Team loss to OG), 2-0 (LGD Gaming over Digital Chaos), 2-0 (LGD Gaming over Invictus Gaming)

Teams playing the next day are 4-5 (10-11) in all matches (all Bo3) against teams playing with a day off.

However, an oddity which I noticed is that since adopting this schedule, the winner of the Lower Bracket Final has twice also won the Grand Final and the title. Going back to examine all similar matches, the data suggest that a team playing a second time in a day (having won their first match) has a clear advantage over a team that has not played.

2015: 2-0 (Vici Gaming over EHOME), 2-0 (LGD Gaming over Vici Gaming), 3-1 (Evil Geniuses over CDEC Gaming)
2016: 2-0 (Digital Chaos over EHOME), 0-2 (Fnatic loss to Digital Chaos), 1-3 (Digital Chaos loss to Wings Gaming)
2017: 2-1 (Team Liquid over Virtus Pro), 0-2 (LGD Gaming loss to Team Liquid), 3-0 (Team Liquid over Newbee)

Teams playing a second game in a day are 6-3 (15-9) in all matches. (In no cases have teams both played twice in a day in the bracket.)
4-2 (8-5) in Bo3
2-1 (7-4) in Bo5

When I began looking for any noticeable results, the first thing I noticed was actually that the upper bracket team playing in Lower Bracket Round 4 Match 1 has never won the match, and in three years has only once won a game. This is notable mainly as trivia but also because this team will always be playing a third day in a row. So a final category I looked at was teams playing 3 or more days in a row. Due to the schedule, this will always be a lower bracket team, where a loss means elimination.

2015: Team Secret eliminated Day 3 (1-2 to Virtus Pro); EHOME eliminated Day 3 (0-2 to Vici Gaming); Virtus Pro eliminated Day 3 (0-2 to LGD)
2016: EHOME eliminated Day 3 (0-2 to Digital Chaos); Fnatic eliminated Day 3 (2-0 MVP Phoenix, 0-2 to Digital Chaos); Digital Chaos eliminated Day 3 (2-1 Evil Geniuses, 1-3 to Wings Gaming)
2017: LGD Gaming won Day 3 (2-0 OG), eliminated Day 4 (2-0 Invictus Gaming, 0-2 to Team Liquid); Virtus Pro eliminated Day 3 (1-2 to Team Liquid); Team Liquid won Day 3 (and event) (2-1 LGD Forever Young, 3-0 Newbee)

Teams playing three or more days in a row are 6-8 (15-19) in all matches. In no cases have teams both played three days in a row though it is possible in theory.

Two of these wins are from Team Liquid's event victory in 2017. Three of the wins are from teams which were eliminated in a second match the same day. And the final win is from LGD Gaming's third straight day of play in 2017, in a run which saw them eliminated playing on a fourth straight day.

This suggests a comparison of results. I found that the only losses by teams playing a second game in one day are by teams that have played three or more successive days! In fact teams playing a second game on the same day, and not having played more than two days in a row, are 5-0 over the three years this schedule has been used. This leads to two theses:

A team playing a second match in a day has a significant advantage over a team not previously playing that day.
A team playing three or more days in a row is at a significant disadvantage against a team having played fewer days in a row.

Repairing the Schedule

Unfortunately both of these situations have occur repeatedly in The International's Main Event as currently played, and almost certainly have had an outcome on at least some results. I therefore believe the schedule ought to be adjusted. When advantages and disadvantages are known, there are three principles to adopt in working to alleviate them:

Avoid advantage or disadvantage if possible.
If not possible, ensure advantages or disadvantages are equal if possible.
If not possible, ensure advantages or disadvantages occur as appropriately as possible.

I believe the schedule could be reconstructed fairly easily keeping these in mind. The schedule I would propose is:

Day 1: Lower Bracket Round 1 (Bo1 or Bo3)
Day 2: Upper Bracket Round 1
Day 3: Lower Bracket Round 2
Day 4: Upper Bracket Round 2; Lower Bracket Round 3
Day 5: Lower Bracket Round 4; Upper Bracket Final; Lower Bracket Semifinal
Day 6: Lower Bracket Final; Grand Final (Bo5)

Unfortunately, from Day 3 in the lower bracket and Day 4 in the upper bracket teams are playing every day: without stretching the schedule over more days this cannot be avoided completely. But on this plan, it applies as evenly as possible to all teams.

Additionally, with the clear advantage to a team playing a second time in a day, I would suggest that the advantage (either in the form of a default win or in a requirement that the lower bracket team win by two games - preferable but potentially impracticable) ought to be returned to the upper bracket team in the Grand Final.

Mafe

Germany5966 Posts

August 14 2017 21:26 GMT

While you are pointing out a few flaws indeed, there are some question marks over your analysis:
-You say that teams were placed in groups. Is that indeed the case? I didnt pay too much attention but did valve release any information about how the groups were created? Maybe it was a draw, according to whatever rules?
-Groupstage were surely created to include time for potential tiebreaking matches on day 4. Therefore on day 4 fewer matches were scheduled. Is that still possible with a 2-2-2-2 schedule for all teams? I dont think so.
-TI2&3 were usingly largely the same schedule for the double elimination system (though a lot of the matches were bo1 instead of bo3). Did you look at those?
-The sample sizes are small. Maybe still too small to draw any definite conclusions. For one of your examples, if you were to draw the outcome of 9 matches as random, a 6-3-or-more-extreme result has a ~50% of happening. ThereforeI claim that a 6-3 result proves absolutely nothing.
-If teams play on three games in a row, as you pointed out this can only happen if the team is in lower bracket for three matches. Then it will meet a team from the upper bracket, right? If so, I would expect the team from the upper bracket to be better, and therefore it will should be logical that a team playing on its third day in a row has a relatively high chance of losing. But that is more likely due to the fact that they are facing a better team, and not because of schedule imabalances.

Disclaimer: I have not understood everything you said as I'm quite tired and/or some parts are could be formulated more clearly.

Laserist

Turkey4269 Posts

August 15 2017 15:28 GMT

Well, as far as i understand you draw some statistical conclusion from a very low amount of data points, basically a couple of matches.
It is impossible reach to a meaningful conclusion i.e

"the winner of the Lower Bracket Final has twice also won the Grand Final and the title" argument only feeds from 3 sets of matches from 3 different years which is extremely low to draw any conclusion.

Aznupdown

Canada318 Posts

August 15 2017 16:33 GMT

Day 1: Lower Bracket Round 1 (Bo1 or Bo3)
Day 2: Upper Bracket Round 1
Day 3: Lower Bracket Round 2
Day 4: Upper Bracket Round 2; Lower Bracket Round 3
Day 5: Lower Bracket Round 4; Upper Bracket Final; Lower Bracket Semifinal
Day 6: Lower Bracket Final; Grand Final (Bo5)

With this scheduling there would only be 4 matches played on the first day since the first round LB is bo1 for elimination. What would be the filler? Or would you just end the day as is?

VGhost

United States3606 Posts

August 16 2017 13:24 GMT

On August 16 2017 00:28 Laserist wrote:
Well, as far as i understand you draw some statistical conclusion from a very low amount of data points...

Right - and I said this repeatedly. There were a couple results that stood out but it's true it could just be a fluke - like three heads in a row flipping coins. I'm still interested in possible conclusions, but more data would only make me happier.

On August 15 2017 06:26 Mafe wrote:
-You say that teams were placed in groups. Is that indeed the case? I didnt pay too much attention but did valve release any information about how the groups were created? Maybe it was a draw, according to whatever rules?

I don't know how the placement was done. It's rumored that it was based on an internal Valve ranking system, which if so certainly would alleviate some of my concern, though I'd prefer it if it were public.

On August 15 2017 06:26 Mafe wrote:
-Groupstage were surely created to include time for potential tiebreaking matches on day 4. Therefore on day 4 fewer matches were scheduled. Is that still possible with a 2-2-2-2 schedule for all teams? I dont think so.

You're right, the 2-2-2-2 would mean all days would be of equal length, and that would cause problems for the tie-breakers. I presume that was the motivation for the 3-2-2-1 standard. But I don't know whether not everybody got a 3-2-2-1 because it was impossible (I haven't yet tried to construct that schedule myself), or for other reasons.

On August 15 2017 06:26 Mafe wrote:
-TI2&3 were usingly largely the same schedule for the double elimination system (though a lot of the matches were bo1 instead of bo3). Did you look at those?

TI 1-3 all used the sixteen-team bracket, with varying numbers of Bo3 matches. I did look at them briefly, but as they all differed both from each other and from the standard schedule used for TI 5-7 I wasn't certain the comparison would be much use. At TI2 equivalent matches were mostly played on the same day; TI3 seems to have introduced the idea of playing half-brackets, but there two days were half the upper bracket, then two rounds (Bo1) for the lower bracket. That does wind up with some of the same differences, but with the Bo1s I'm not sure it's quite the same question.

On August 15 2017 06:26 Mafe wrote:
-The sample sizes are small. Maybe still too small to draw any definite conclusions. For one of your examples, if you were to draw the outcome of 9 matches as random, a 6-3-or-more-extreme result has a ~50% of happening. ThereforeI claim that a 6-3 result proves absolutely nothing.
-If teams play on three games in a row, as you pointed out this can only happen if the team is in lower bracket for three matches. Then it will meet a team from the upper bracket, right? If so, I would expect the team from the upper bracket to be better, and therefore it will should be logical that a team playing on its third day in a row has a relatively high chance of losing. But that is more likely due to the fact that they are facing a better team, and not because of schedule imabalances.

Disclaimer: I have not understood everything you said as I'm quite tired and/or some parts are could be formulated more clearly.

Thanks for the question: I think I've found a mistake. I may try to go back and edit it in a bit, but let me try to lay out what I found.

Of the three categories I looked at, the only one which suggested an advantage/disadvantage was the 6-3 result for teams playing a second series on the same day. Even that, as you say, is not conclusive: to get an extreme result over 9 series assuming a 50% chance in any single series does have about a 50% chance, though the probability of a positive extreme result, 6-3 or better, is only about 25%

But I had also noticed an overlap with one of the other criteria I had looked at. I found 9 matches played by teams as their second match in a day; I found 14 matches played by a team on the third (or greater) straight day of play. Four of those matches overlapped, in the sense that a team playing for the second time in one day was also playing for three or more straight days: and in these cases, the record was 1-3. This resulting overlap also meant that if a team was playing for a second time in one day without also playing on a third straight day, they have been undefeated. Thus my theses that (a) the team playing the second time in one day had an advantage and (b) the team playing three or more days in a row had a disadvantage. On closer inspection, that second idea doesn't hold up; I think the first may.

Consider a composite table of my categories "playing for a second time in one day" and "playing three or more days in a row" (imagine a Venn diagram with the middle row the overlap):

Teams playing for a second time in one day: 5-0
Teams playing for a second time in one day and playing three or more days in a row: 1-3
Teams playing three or more days in a row: 5-5

Now in fact that suggests that, by itself, playing repeated days in a row isn't particularly a factor. But why I started poking at that criterion (and made an apparently unfounded assumption) was the weird fact that the team losing Upper Bracket Round 2 Match 2 has never won their next (Lower Bracket) match, which ought to be against a worse team (either lower bracket to begin with, or upper bracket start that lost earlier).

But thanks to your question, reviewing the results I see my initial impulsive conclusion that the repeated days of play were the culprit looks unsustainable.

On August 16 2017 01:33 Aznupdown wrote:
Day 1: Lower Bracket Round 1 (Bo1 or Bo3)
Day 2: Upper Bracket Round 1
Day 3: Lower Bracket Round 2
Day 4: Upper Bracket Round 2; Lower Bracket Round 3
Day 5: Lower Bracket Round 4; Upper Bracket Final; Lower Bracket Semifinal
Day 6: Lower Bracket Final; Grand Final (Bo5)

With this scheduling there would only be 4 matches played on the first day since the first round LB is bo1 for elimination. What would be the filler? Or would you just end the day as is?

I'd leave it just as is: perhaps combine it with a less half-assed opening ceremonies, or with the media day, or something. In fact I'd really like to have a Day 7, so that the upper bracket final, lower bracket semi, and lower bracket final were all played on Day 6, and the grand final on Day 7 - but then I have no idea what constraints Valve is working with, so I worked out my suggestion assuming six days only. It's entirely probable that better refinements can be suggested.

Please or register to reply.

A Critique of TI Scheduling

Completed

Ongoing

Upcoming