|
On October 12 2023 00:27 TT1 wrote:
yes well the problem is eapm is also a flawed measure in BW which can easily be spammed, it has to do with unit movement/attack commands (protoss has less of this so their eapm is usually less than t/z), so a zerg player will normally have more eapm just because they're constatantly microing mutas etc... but i guess it's better than nothing
Actually in the data it can be seen that Protoss have higher eapm than Zergs. + Show Spoiler +
to me a sample size of like ~1k hand picked games of high lvl players is worth way more than that type of data
The problem with that is that with such a small sample, if you'd repeat this type of win rate vs game time analysis, you'll get completely dominated by statistical fluctuations so your data starts to look (almost) random.
Just for the win rates (no time dependence), you can refer to eloboard, where you have a sample of pro games.
|
Interesting. I have to agree with TT1, though; as a P main, I always had higher EAMP when off-racing as Z, mostly because mutalisks.
|
On October 12 2023 00:46 Kraekkling wrote:Show nested quote +On October 12 2023 00:27 TT1 wrote:
yes well the problem is eapm is also a flawed measure in BW which can easily be spammed, it has to do with unit movement/attack commands (protoss has less of this so their eapm is usually less than t/z), so a zerg player will normally have more eapm just because they're constatantly microing mutas etc... but i guess it's better than nothing
Actually in the data it can be seen that Protoss have higher eapm than Zergs. + Show Spoiler +Show nested quote +
to me a sample size of like ~1k hand picked games of high lvl players is worth way more than that type of data
The problem with that is that with such a small sample, if you'd repeat this type of win rate vs game time analysis, you'll get completely dominated by statistical fluctuations so your data starts to look (almost) random. Just for the win rates (no time dependence), you can refer to eloboard, where you have a sample of pro games.
yes Z won't have mutas most of the time and the type of muta micro is much different in ZvP <> ZvT
P has sairs/zeals which are heavy movement/attack units, it depends on the playstyle and the stage of the game (map control is key here), the early to middgame will be P eAPM intensive (relative to say PvT but that also depends on the playstyle, i.e: forge opener will be less eAPM intensive than 1g expo in PvZ) and most games will end around that stage as well (depending on whether or not P manages to slow Z down enough/bust)
like i said eAPM means something at high/pro lvl because players aren't sacrificing their macro for good unit movement, at lower levels they're sacrificing their macro the majority of the time tho
basically at lower lvls it comes down to how that player enjoys playing the game, some are more macro heavy others more micro heavy and that'll have a big impact on their eAPM, but ultimately in both cases their play is extremely flawed
|
Btw, from my gut feeling I also expected Protoss players to have lower eapm than Zergs.
|
Eapm has a massive anti Zerg bias. It counts pressing select larva multiple time as spam, even though that is an effective way to grab larva right as it hatches. Other races can just queue. Mutalisks and lings which are used in every match up require a lot of move commands. If you can give Zerg and exception for move commanding mutalisk, lings, and pressing select larva then it will be more accurate
|
This is great stuff, thank you for putting this together. Really looking forward to the spawn/skill plots.
Question: how do you detect who won the match? I thought this information was not included within the replay.
Also, more technical question here, just curious, I noticed you included error bars, are these the sample std deviation of your data?
|
Lol, tons of work man! Good job.
|
Thank you very much for the work!
|
I've added plots to the cross-spawn part and the apm group comparisons.
On October 12 2023 01:38 Volka wrote:
This is great stuff, thank you for putting this together. Really looking forward to the spawn/skill plots.
Question: how do you detect who won the match? I thought this information was not included within the replay.
Also, more technical question here, just curious, I noticed you included error bars, are these the sample std deviation of your data?
Yes its the standard error of the bin content.
|
With regards to mmr/eapm and such, couldn't you get a lot of leverage a simplistic ELO implementation? Is there enough info about player handles/accounts to do that?
Perhaps it's just easier to wait for mmr data, seems that will be forthcoming.
|
After looking at the apm vs winrate figure, I feel like there's something wrong with the data or treatment of data.
For example:
+ Show Spoiler +
In the overall figure, in the 14-15th minute interval, the win rate is exactly at 50%
+ Show Spoiler +
But in the apm figure, the win rates in that interval for the <150 and >150 apm groups are both over 50%. That can't be right?
Also you can see here that the curves for the >150 and >180 apm groups are almost the same. It shows you that apm progression doesn't reflect skill progression. Meanwhile in the stats posted by cwal, you can see the win rate shifts quite strongly with the progression of mmr:
+ Show Spoiler +
(image taken from reddit)
|
On October 12 2023 05:52 TMNT wrote:After looking at the apm vs winrate figure, I feel like there's something wrong with the data or treatment of data. For example: + Show Spoiler +In the overall figure, in the 14-15th minute interval, the win rate is exactly at 50% + Show Spoiler +But in the apm figure, the win rates in that interval for the <150 and >150 apm groups are both over 50%. That can't be right?
I can't give you a certain answer to this now but we are throwing away games where players do not fit any of the brackets, for example games where one player has an effective apm of 120 and the other of 160. This might account for it, I'm not sure why it should though. I might take a deeper look into whats happening here.
Also you can see here that the curves for the >150 and >180 apm groups are almost the same. It shows you that apm progression doesn't reflect skill progression. Meanwhile in the stats posted by cwal, you can see the win rate shifts quite strongly with the progression of mmr: + Show Spoiler +(image taken from reddit)
Feel free to draw whatever conclusion you find most interesting from the data - I'm just presenting the data.
On October 12 2023 05:01 sophisticated wrote:
With regards to mmr/eapm and such, couldn't you get a lot of leverage a simplistic ELO implementation? Is there enough info about player handles/accounts to do that?
Perhaps it's just easier to wait for mmr data, seems that will be forthcoming.
Yeah I'd rather use the mmr data if possible.
|
On October 11 2023 19:05 Branch.AUT wrote:Show nested quote +On October 11 2023 13:37 imBLIND wrote:On October 11 2023 08:32 TMNT wrote:
I'm very much waiting for the analysis on player skill influence, as I expect we will some notable changes in the shape of those curves around S rank On October 11 2023 12:21 TT1 wrote:
Is it possible to do this with progamer replays or 2500+ ladder games from cwal (the ladder games would prob be way better cus of sample size)? I know the sample size would be way less but the quality of games is way more important for this type of analysis. Thanks for this tho, great work. Just to add on to this, and I'm sure someone else has already done this, but could you show TvZ and TvP win rates with and without Flash? Assuming equal distribution between zvp/tvz/pvt, we're talking about 2.3Million games in each matchup. Excluding any one player, no matter the name, shouldn't make a dent. Out of the sheer volume of games analyzed. Great work OP! It's very interesting to see the winrate evolution over game time.
On October 11 2023 19:18 TMNT wrote:Show nested quote +On October 11 2023 13:37 imBLIND wrote:On October 11 2023 08:32 TMNT wrote:
I'm very much waiting for the analysis on player skill influence, as I expect we will some notable changes in the shape of those curves around S rank On October 11 2023 12:21 TT1 wrote:
Is it possible to do this with progamer replays or 2500+ ladder games from cwal (the ladder games would prob be way better cus of sample size)? I know the sample size would be way less but the quality of games is way more important for this type of analysis. Thanks for this tho, great work. Just to add on to this, and I'm sure someone else has already done this, but could you show TvZ and TvP win rates with and without Flash? You already have it. T win rate without Flash is literally the current win rate on eloboard, since the site started around the time he left for military. Then compare it with whatever you have before that point. I looked up a while ago and they're basically unchanged.
You guys are absolutely right, Flash's games won't make a dent in 8.3 million total games. I should've been more specific in saying I'd like to see Flash's T v P/Z rates versus the other S-rank Terran's T v P/Z rates. Unfortunately, it appears that Kraekkling is unable to get just S-rank games, so my request is rather useless anyways T-T...
|
On October 12 2023 05:52 TMNT wrote:After looking at the apm vs winrate figure, I feel like there's something wrong with the data or treatment of data. For example: + Show Spoiler +In the overall figure, in the 14-15th minute interval, the win rate is exactly at 50% + Show Spoiler +But in the apm figure, the win rates in that interval for the <150 and >150 apm groups are both over 50%. That can't be right? Also you can see here that the curves for the >150 and >180 apm groups are almost the same. It shows you that apm progression doesn't reflect skill progression. Meanwhile in the stats posted by cwal, you can see the win rate shifts quite strongly with the progression of mmr: + Show Spoiler +(image taken from reddit)
I think it's caused by different samples. First graph counted all games; second one counted only those on 4p maps.
Speaking of which, I'd be very interested in winrate-by-length graphs for 2p, 3p and 4p maps respectively.
|
|
|
On October 12 2023 05:52 TMNT wrote:After looking at the apm vs winrate figure, I feel like there's something wrong with the data or treatment of data. Also you can see here that the curves for the >150 and >180 apm groups are almost the same. It shows you that apm progression doesn't reflect skill progression. Meanwhile in the stats posted by cwal, you can see the win rate shifts quite strongly with the progression of mmr: + Show Spoiler +(image taken from reddit)
PvT's taken into equation with eAPM are +/- 10% of PvTs from the 1st graph, and I think S rank is for 1% only if i'm not mistaken. eAPM is not super effective tool for measuring MMR as well. So here is your answer probably.
|
Until you break down the balance at an S rank these analyses are not useful at all. If you're C rank and struggle to beat a certain race, just get better at the game. It really is not out of anyones reach to get to S rank with any race. Then the problems start, because Terran becomes so overpowered that it dominates every high level ladder/high level tournament and that is an issue of balance.
|
Awesome analysis. Quite unfortunate that build orders are not available as it would be even cooler if we could break these stats down by build order matchups.
|
This is fantastic, looks great and is a great read thanks for the cool analysis!
I can understand if you are not accepting requests but if you have a todo list I would suggest two points.
As a visually regressed person it would be great to have the mirrored graphs for the matchups
If you are still going to look at the data in the future would you check some derivations of these plots?
- P(vZ) has around 10 minutes when the race is above 50%? From the looks of it out of 38 mins or so 1-8 -> Z, 9-19 -> P, 20-40 -> Z ~28 minutes go to zerg, and ~10 to protoss. But by the win percentage volume of those 10 minutes for P I'm not even sure it cancels out the first 8 minutes (and the hydra bust specifically) of Z slaughter
- Accumulating the win rates over time would also be an interesting comparison to those graphs
- Average rate of change for winrates and their direction could perhaps iindicate which parts of the matchup need to be revolutionised in the next 8 million games :D
Sorry, I got carried away by looking at your graphs. Really well done thanks for your good work!!
|
Certainly, those are a lot of games. Good statistics, man.
|
|
|
|
|
|
EPT![[image loading]](https://i.ibb.co/4p1MNH4/effapm-allraces.png)
![[image loading]](https://i.ibb.co/qBv8LW4/racial-win-rate-vs-time-pvt.png)
![[image loading]](https://i.ibb.co/583C3RC/racial-win-rate-vs-time-pvt-selectioncomparison-eapm-Groups.png)
![[image loading]](https://i.imgur.com/ZloyhHr.png)
