|
Following the publication of ladder game data analysis in Kraekkling's thread, here is an extensive analysis of data from all Starleague (ASL/KSL) games played between 2016 and 2023.
The analysis is based on a new, manually compiled dataset of the 1,906 on-stage games of ASL seasons 1-16 and KSL seasons 1-4. The findings relate to many different aspects including matchup dynamics (by year, game duration, spawn locations etc.), player rankings (winrates, Elo), map balance rankings, as well as sundry other enquiries including but not limited to testing out the famous "If I win game 1, I'll win the whole series" claim.
I have written up the findings on a web page with a lot of graphs and tables. Check it out in the link below:
All the stats from the ASL/KSL era
|
While I appreciate the analysis, I think the limitation of sample size and more importantly, the cup format of ASL and KSL, make the methodology a bit flawed, which leads to some misleading conclusions.
For example, ain't no way Sylphid is the most balanced map ever made. If anything it's one of the worst. You can check its stats (with number of games in the range of thousands) on eloboard and see.
The reason: with 141 games as a total sample size, excluding the mirror matchups maybe we have like 100 games left (just hypothetically), that leaves us with 33-34 games per matchup which means you have a ratio of 17:17 for a totally balanced map (at 50%) and only a 3-game swing makes it become a totally unbalanced map with 40% win rate for one race. And that's for Sylphid with a total games of 141. As for maps like Vermeer with 46 games, I can only imagine you have a sample size of 10 for each matchup.
The second problem is the cup format which, combined with the sample size, skew the win rate of players in every direction. This is because the quality of players in the Ro24 is very different to Ro16, let alone Ro8, 4 and finals. For example, let's say a player has a record of 8-4 before the semifinals, but then loses the semi 0-4 (to Flash for example). He will end up with a win rate of 50%, which is not better than a player with a record of 2-1 in Ro24 and 1-2 in Ro16.
You would think it will eventually even out after 20 seasons, but clearly not enough. That's why you end up with stuff like:
- Mong and Sea are better TvP players than Light and Last? And Last at 7th place nonetheless.
- Ample is the 5th best TvZ player in ASL+KSL history?
- Jaehoon and Horang2 are better PvT players than Bisu and... Rain?
- Best is a better PvZ player than Bisu and Mini?
- Sacscri as the 5th best ZvT player?
- Queen only at 8th spot in the ZvP table? (probably because he lost 2-8 to Mini in two semifinals which makes his stats look worse)
Note that you don't see those irregularities in the ranking for 3 mirror matchups? That's quite self explanatory.
|
Great work! Nicely formatted, and some interesting stats to explore there. Thank you for sharing 
|
|
|
On October 24 2023 09:38 TMNT wrote:
While I appreciate the analysis, I think the limitation of sample size and more importantly, the cup format of ASL and KSL, make the methodology a bit flawed, which leads to some misleading conclusions.
For example, ain't no way Sylphid is the most balanced map ever made. If anything it's one of the worst. You can check its stats (with number of games in the range of thousands) on eloboard and see.
The reason: with 141 games as a total sample size, excluding the mirror matchups maybe we have like 100 games left (just hypothetically), that leaves us with 33-34 games per matchup which means you have a ratio of 17:17 for a totally balanced map (at 50%) and only a 3-game swing makes it become a totally unbalanced map with 40% win rate for one race. And that's for Sylphid with a total games of 141. As for maps like Vermeer with 46 games, I can only imagine you have a sample size of 10 for each matchup.
The second problem is the cup format which, combined with the sample size, skew the win rate of players in every direction. This is because the quality of players in the Ro24 is very different to Ro16, let alone Ro8, 4 and finals. For example, let's say a player has a record of 8-4 before the semifinals, but then loses the semi 0-4 (to Flash for example). He will end up with a win rate of 50%, which is not better than a player with a record of 2-1 in Ro24 and 1-2 in Ro16.
You would think it will eventually even out after 20 seasons, but clearly not enough. That's why you end up with stuff like:
- Mong and Sea are better TvP players than Light and Last? And Last at 7th place nonetheless.
- Ample is the 5th best TvZ player in ASL+KSL history?
- Jaehoon and Horang2 are better PvT players than Bisu and... Rain?
- Best is a better PvZ player than Bisu and Mini?
- Sacscri as the 5th best ZvT player?
- Queen only at 8th spot in the ZvP table? (probably because he lost 2-8 to Mini in two semifinals which makes his stats look worse)
Note that you don't see those irregularities in the ranking for 3 mirror matchups? That's quite self explanatory.
I agree with this, especially that historical sum total winrates don't translate to a skill ranking. In the case of Sea, he was one of the strongest Terrans in 2017 but he's very far from that now. The total winrates obviously don't pay any attention to time. The Elo ranking does, though, so that should give a better picture of who is stronger now.
But yes, the sample rate means that most of the stats can't be interpreted as evidence of strong underlying trends, and some of them are nothing but hints. But the stats are what they are, and it's up to each of us what we will make of them.
About
"- Ample is the 5th best TvZ player in ASL+KSL history?"
and
"- Jaehoon and Horang2 are better PvT players than Bisu and... Rain?"
Well, they have a higher winrate in those matchups.I can't measure whether they are "better" in general, and my gut instinct is definitely to agree with you that they are probably not. Maybe it's useful to think of the winrate rankings more as a scoreboard than as an accurate reflection of skill levels.
|
Good work! One request - instead of measuring how "fast" or "slow" a player is by their game length, can you only count the duration of games they won?
|
Wow this looks like it took a lot of work and you've formatted it well too! Thanks for your time and Effort
|
this is really amazing. thank you. BW is extremely balanced
|
|
|
On October 24 2023 16:21 Zealgoon wrote:
Good work! One request - instead of measuring how "fast" or "slow" a player is by their game length, can you only count the duration of games they won?
Interesting idea. I'll note that down and let you know when I've had time to look into it. You can see the winrates of a few selected players divided into time intervals in the article, which I guess is kind of close to what you're asking. The players selected are the ones with high variance between the different intervals.
Edit: actually this was really easy to look up by modifying the other script very slightly. This is what the table of average win duration by player looks like.
|
Amazing work. Read through all of it, really interesting, but not really surprising :D
|
Well done. It must have been a huge amount of work to collect data on game length and spawn locations by hand from 2 thousand afreeca vods, huge respect for your determination!
As discussed in the thread, I agree that we should be careful not to draw too many conclusions from stats which are based on a small sample of games, like for example balance stats for some of the maps.
Overall I think there are many valuable insights here. In particular, to me it was remarkably interesting to see how much Terran seems to rely on close spawns on 4-player maps.
|
On October 24 2023 21:35 Kraekkling wrote:
Well done. It must have been a huge amount of work to collect data on game length and spawn locations by hand from 2 thousand afreeca vods, huge respect for your determination!
As discussed in the thread, I agree that we should be careful not to draw too many conclusions from stats which are based on a small sample of games, like for example balance stats for some of the maps.
Overall I think there are many valuable insights here. In particular, to me it was remarkably interesting to see how much Terran seems to rely on close spawns on 4-player maps.
Thank you. Yeah, I created an interface for putting in the data and then just filled it in little by little at times when I wasn't doing anything else important, over a year or so. I'm sure there was a smarter approach 😂 But hey, it worked out.
|
United States9943 Posts
Rain with an 84% PvP win rate is just crazy to me. Further noting that TvT and PvP are definitely more "skilled" mirror matchups while Zerg, with not one player having over 55% win rate, is definitely a coin flip matchup with the most amount of RNG based off build order wins. Sad to see Jaedong, which we used to nickname the matchup JvZ, be only at 33% WR, which is likely due to him being older and can't rely on his insane micro to level the game after a build order loss.
Also, it seems that 3 player maps are the most balanced overall for all 3 races (PvZ being the most imbalanced). This might be because 3p maps are just better for every race, or because RNG of spawn location, thus evening out the matchups (unlike 4p maps where cross spawns give no one a true advantage, 3p maps will always have someone spawn in a "favorable" position).
Really cool post overall, but I do agree with small sample sizes that some of the conclusions can't be totally drawn out from the games.
|
Average game length to me was very interesting. Fun to see most zergs average game length under 13 minutes
|
The pies and half donuts look tasty! ; D
I think we should not forget the finals either (where it really matters in terms of cash)
Terrans have won as much as the other 2 races COMBINED!!!!!!!!!!! T = Z + P ( T 10W = Z 6W + P 4W)
In terms of raw first place cash won T>Z+P
Terran: $ 613,466 (ASL $ 474,347) > Z+P Total: $ 549,839
Zerg: $ 338,040 (ASL $ 267,026)
Protoss: $ 211,799 (ASL $ 143,911)
*according to liquipedia and my calculations excluding VANT36.5 National Starleague and HungryApp Starz League with Kongdoo who are literally NOT named ASL or KSL (1 Z and 1 T victories there).
|
On October 25 2023 03:08 LUCKY_NOOB wrote:The pies and half donuts look tasty! ; D I think we should not forget the finals either (where it really matters in terms of cash) Terrans have won as much as the other 2 races COMBINED!!!!!!!!!!! T = Z + P ( T 10W = Z 6W + P 4W)In terms of raw first place cash won T>Z+PTerran: $ 613,466 (ASL $ 474,347) > Z+P Total: $ 549,839Zerg: $ 338,040 (ASL $ 267,026) Protoss: $ 211,799 (ASL $ 143,911) *according to liquipedia and my calculations excluding VANT36.5 National Starleague and HungryApp Starz League with Kongdoo who are literally NOT named ASL or KSL (1 Z and 1 T victories there). + Show Spoiler +
I wish this was a good way to see how terrans perform, but ASL seasons do not have the say pay out each season. Majority pay around 22k, however, ASL4 paid out 52,650 which FlaSh won.
|
I totally see TMNT's point about sample size but it is a good indication about which players do better or worse in certain MUs during ASL. For example I would normally Light over any Zerg in Proleague or KCM as he's basically the strongest general TvZer but he seems to do worse in offline planned matches. Snow has been the strongest general Protoss player for the last year but has really struggled to get out of group stages in ASL for the last few seasons.
Also damn yea flash really was GOAT!
|
On October 24 2023 19:16 JackyVSO wrote:Show nested quote +On October 24 2023 16:21 Zealgoon wrote:
Good work! One request - instead of measuring how "fast" or "slow" a player is by their game length, can you only count the duration of games they won? Interesting idea. I'll note that down and let you know when I've had time to look into it. You can see the winrates of a few selected players divided into time intervals in the article, which I guess is kind of close to what you're asking. The players selected are the ones with high variance between the different intervals. Edit: actually this was really easy to look up by modifying the other script very slightly. This is what the table of average win duration by player looks like.
Thanks for the quick reply. Free is looking a lot less slow now
|
"Reverse sweeps
The last series stat I’ve registered is the probability of reverse sweeps. In 197 Best-of-5s, we have seen six reverse sweeps. The probability of making a reverse sweep in a Best-of-5 (i.e. the probability that if you’re down 0-2, you go on to win the series) stands at 6%."
Maybe im confused, but how can 6/197 be 6%? Shouldn't it be around 3%? Or is it that only 100 games ended up being a 0-2 to start with. That was the only part that confused me here
|
|
|
|
|
|
EPT![[image loading]](https://i.imgur.com/raAWXQe.png)
![[image loading]](https://i.imgur.com/DLpk8Ew.png)
