ZvP is imbalanced - Page 38

On October 21 2009 05:46 heyoka wrote:
YOUR STANDARD DEVIATION CANNOT BE 49%

THAT IS NOT YOUR VARIATION

YOU ARE LOOKING AT ONE DATA POINT

THERE IS NO OTHER WAY I CAN SAY THIS SIMPLY, UNLESS I DO IT IN CAPS PLUS BOLD

On October 21 2009 05:46 Matrijs wrote:
The bottom line here is that, despite what Day[9] said in his video analysis of Bisu's recent ZvP loss to Shine, the recent higher rate of Zerg wins in the matchup is not a coincidence. It cannot be attributed to random factors. What this means is that there's something going on that's changing the "natural" ZvP win rate.

I don't have the kind of expert qualifications necessary to undertake match analysis of recent games to try to figure out what the problem is, but I would point out one thing: I don't think it's the maps.

Consider: if we were to expect that particular maps are making a difference, we would probably see that the newer maps are where the zergs are piling up wins, and that the older maps have lower Zerg win rates. That's just not what we observe. Destination is a relatively old map - it dates back prior to the surge in Zerg wins. We would expect a lowish Zerg win rate - instead, we see Zerg winning at a 60% clip. Heartbreak Ridge is a new map, we should expect a high zerg win rate - instead we see only 58%, lower than the mean. God's Garden is a new map - only a 56% win rate. Outsider is 60% for Zerg, but it's the exception, not the rule. We can see the same thing happening in maps with new versions. Medusa has a higher Zerg win rate than Neo Medusa over the period we're studying. Byzantium 2 has a higher Zerg win rate than Byzantium 3. It seems to me that you could make a strong case that the win rates on the new maps aren't all that much different from the win rates on the old maps.

On October 21 2009 05:46 heyoka wrote:
YOUR STANDARD DEVIATION CANNOT BE 49%

THAT IS NOT YOUR VARIATION

YOU ARE LOOKING AT ONE DATA POINT

THERE IS NO OTHER WAY I CAN SAY THIS SIMPLY, UNLESS I DO IT IN CAPS PLUS BOLD

On October 21 2009 06:12 Black Gun wrote:



u forget one thing: that it might take time until a race figures out how to abuse the maps to win. maybe the maps had what it needs to be imba in zvp, but it was so subtle that the zergs needed several months to figure it out. for example hbr: first it was good for protoss. then came lurker contain and it was relatively balanced. then came the abuse of the excess gas for muta snipes which make mass hydra roll any protoss army. hbr turned into a protoss graveyard. the map hasnt changed, it had the potential to be a protoss graveyard since the very beginning, zergs just didnt know.

on the other hand, maybe the maps do allow for a good protoss counter to 5hatch hydra with muta snipe, and the tosses just dont know yet.

On October 21 2009 06:13 motbob wrote:

Take it up with the programmers of Excel, not me. As you can see below, I'm asking Excel to give me the standard deviation of the dataset, and it's giving me ~0.49

On October 21 2009 06:09 mahnini wrote:

you didnt bold newb!

btw i took stats like a year ago and i'm terrible at math but i'm pretty sure you're supposed to have a control group in order to prove anything and that's pretty much impossible because it's impossible to control / keep external factors constant so all this math seems kind of useless.

On October 21 2009 06:14 Black Gun wrote:




use a small dataset: 10 data points, 8 times a "1", 2 times a "0". the percentage is 0.8. look what excel tells u about the sd.

On October 21 2009 06:14 motbob wrote:

Nah, you're thinking of controlled experiments. This is just data analysis.

On October 21 2009 06:18 mahnini wrote:

yeah but all of your data doesnt really prove a point because it's not taking into account factors other than win/lose so there's really no point.

On October 21 2009 06:21 motbob wrote:

I'm just trying to show that it's not a coincidence that zergs have been winning. It's not random chance. There's an "external factor," as you put it.

On October 21 2009 06:58 Gustav_Wind wrote:
The standard deviation for a single event that has 55% probability is in fact 0.49. That is obtained by the simple calculation sqrt(p(1-p)). To get the standard deviation that we want to use in calculating z-score, divide that value by the square root of the sample size. so .49/sqrt(885).

Also, pointing to a big column of excel data for your evidence is somewhat unnecessary, and kind of undermined your credibility as someone with a good grasp of stat. The standard deviation of a single bernoulli event is sqrt(p(1-p)), which is the same as what that excel calculation was doing.

On October 21 2009 07:07 motbob wrote:

It's more correct to call this the standard error...

http://en.wikipedia.org/wiki/Standard_error_(statistics)

http://en.wikipedia.org/wiki/Standard_deviation

There's a subtle but important difference. Calling both of these things the standard deviation would be really confusing. So most statisticians call the SD of the sampling distribution (which is SD_pop/sqrt(n)) the "standard error" in order to reduce that confusion.

*shrug* I'm used to working w/ excel spreadsheets w/ data sets that aren't just filled with binary data. So it's second nature for me to just draw up a data set and use the Excel command. It only took 30 seconds to create the data set of 1's and 0's. Note that I didn't just use Excel to get the stdev... Excel also makes it really easy to take all the variables and do the z-test itself.

On October 21 2009 06:58 Gustav_Wind wrote:
Okay guys, the calculation that motbob did and the one Black Gun did are the same, and correct.

59% over 885 games vs an expected 50% (or an expected 55%) is statistically significant. That should be common sense. Think about flipping a coin 885 times and getting heads almost 6/10 times.

motbob, you just did a bad job in explaining/justifying your process. .49 is the standard deviation of ONE zvp game; .49/sqrt(885) is the standard deviation of 885 zvp games. That is what jwd/heyoka were trying to say.

Also, pointing to a big column of excel data for your evidence is somewhat unnecessary, and kind of undermined your credibility as someone with a good grasp of stat. The standard deviation of a single bernoulli event is sqrt(p(1-p)), which is the same as what that excel calculation was doing.

On October 21 2009 07:55 Gustav_Wind wrote:


As I understand it, standard error is used as an estimate of the standard deviation of the true population, right?

But we are assuming that p = .55 in our null hypothesis test, aren't we? So isn't it fine to use the term standard deviation since we can derive that from our assumption?

And .49 and .49/sqrt(885) are both standard deviations. .49 is the standard deviation of the variable (one zvp game), whereas .49/sqrt(885) is the standard deviation of the variable (number of zvp wins in 885 games/885), or in other words, the ratio of zvp wins in 885 games.