| Forum Index > BW General |
motbob
United States12546 Posts
On October 21 2009 03:14 zulu_nation8 wrote: ok motbob you win, the standard deviation is clearly 50% from a mean of 55%. Actually yeah it is. Like I said, go to excel and punch in a bunch of binary data if you don't believe me. You might think that "common sense" tells you that that's impossible, but it's not. Go try it. | ||
|
EtherealDeath
United States8366 Posts
On October 21 2009 03:14 zulu_nation8 wrote: ok motbob you win, the standard deviation is clearly 50% from a mean of 55%. K, so. Motbob's standard deviation is fucking crazzzzy because it appears Excel decided to use the % wins on each individual map as its individual data points (obviously since we do not have any other 7 month periods of data looked up, that was the only option). Of course, many of the maps do not have enough games to be relevant, but the point is: MAPS ARE IMPORTANT TO BALANCE therefore standard deviation is ridiculous as you have noted. Now you may also have noticed that another analysis was done using the entire set of games, all 885 of them, and calculating the probability of something even more extreme occurring (that would be the p-value), which turned out to be really ridiculously low. In other words, the probability of zerg winning at least 59.21% of the games when they are supposed to win 50% (I think 53% was actually used as the null hypothesis) is really really low, aka unexpected. Of course, this analysis does NOT take maps into consideration and instead treats individual game as purely identical. So result? Chances of zerg winning so much = low as hell if ZvP is supposed to be balanced Maps important? Yes, they heavily affect the win% Of course, relative player skill is not taken into account by this model. If you want a more accurate one, go find all the elos involved in every game and post it plz, cause I don't know that TLPD has a convenient interface for doing so. | ||
|
Black Gun
Germany4482 Posts
On October 21 2009 03:20 motbob wrote: Actually yeah it is. Like I said, go to excel and punch in a bunch of binary data if you don't believe me. You might think that "common sense" tells you that that's impossible, but it's not. Go try it. the formula for standard deviation is sqrt(p*(1-p)). its pure coincidence that for p close to 0.5, this sd is about the same as p itself. an example: 1 1 1 1 1 0 1 1 0 0 1 1 0 1 1 1 1 1 0 1 1 1 1 0 0 0 1 1 1 1 1 1 1 1 0 1 0 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 0 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 0 0 1 1 1 1 1 0 0 1 1 1 1 1 1 1 1 1 0 1 these are numbers created with a probability of 0.8. the sample mean is 0.81. the sd is 0.39 though. enter it in excel if u dont believe me. | ||
|
motbob
United States12546 Posts
On October 21 2009 03:25 EtherealDeath wrote: K, so. Motbob's standard deviation is fucking crazzzzy because it appears Excel decided to use the % wins on each individual map as its individual data points (obviously since we do not have any other 7 month periods of data looked up, that was the only option). Of course, many of the maps do not have enough games to be relevant, but the point is: MAPS ARE IMPORTANT TO BALANCE therefore standard deviation is ridiculous as you have noted. Actually, I used 885 data points in my test, just like the other study. I didn't break it down by maps or anything. I just had one column full of 1's and 0's. | ||
|
Day[9]
United States7366 Posts
what happened to null hypothesis tests? : [ | ||
|
Holgerius
Sweden16951 Posts
 | ||
|
zulu_nation8
China26351 Posts
On October 21 2009 03:25 EtherealDeath wrote: K, so. Motbob's standard deviation is fucking crazzzzy because it appears Excel decided to use the % wins on each individual map as its individual data points (obviously since we do not have any other 7 month periods of data looked up, that was the only option). Of course, many of the maps do not have enough games to be relevant, but the point is: MAPS ARE IMPORTANT TO BALANCE therefore standard deviation is ridiculous as you have noted. I think that would be a way to see if a certain map is actually significant imbalanced compared to the other maps played in the same period? That would be a good test if maps are we are trying to focus on. And of course we would need to set a minimum number of games played. | ||
|
zulu_nation8
China26351 Posts
On October 21 2009 03:28 motbob wrote: Actually, I used 885 data points in my test, just like the other study. I didn't break it down by maps or anything. I just had one column full of 1's and 0's. ok i'll try to explain it in another way, the null hypothesis you used is a percentage, but the data points are not percentages. Therefore your SD is wrong. | ||
|
Black Gun
Germany4482 Posts
On October 21 2009 03:34 Day[9] wrote: i'm reading so much about standard deviation what happened to null hypothesis tests? : [ they were performed  the result: the zvp winning percentage of the last 7 months significantly exceeds 55%. so even if the historical race imbalance would be as high as 55% zerg wins, the recent trend would still be much higher than that, so that it cant be explained as a fluke. | ||
|
EtherealDeath
United States8366 Posts
On October 21 2009 03:28 motbob wrote: Actually, I used 885 data points in my test, just like the other study. I didn't break it down by maps or anything. I just had one column full of 1's and 0's. Oh, well if you do it that way then the standard deviation isn't really meaningful I don't think. I'd prefer the p-value instead :O | ||
|
zulu_nation8
China26351 Posts
On October 21 2009 03:40 Black Gun wrote: they were performed the result: the zvp winning percentage of the last 7 months significantly exceeds 55%. so even if the historical race imbalance would be as high as 55% zerg wins, the recent trend would still be much higher than that, so that it cant be explained as a fluke. if i was to plot the data of zvp win % over every 800 games in the history of bw, and find the standard deviation. And then plug that into a z test for the current 800 game period and have the null be 55%, would that be a better test to explain if the current trend is significant? | ||
|
Black Gun
Germany4482 Posts
On October 21 2009 03:43 EtherealDeath wrote: Oh, well if you do it that way then the standard deviation isn't really meaningful I don't think. I'd prefer the p-value instead :O when comparing with a truly balanced 50% chance to win, the p-value is 0. when comparing with historical imbalances, (which we dont know, so i substituted 55% for the historical "race imbalance equilibrium"), its 0.0059. | ||
|
EtherealDeath
United States8366 Posts
EDIT: wait nvm, the matchup elo of each player itself would be dependent on the balance, and then I am not sure how you resolve this interdepency of variables >.> Perhaps instead we look at the matchup elos of the players of each race, and compare the resulting distributions? EDIT2: need more minor league games, there aren't THAT many players with enough games for me to trust in their elo -.- | ||
|
EtherealDeath
United States8366 Posts
On October 21 2009 03:45 Black Gun wrote: when comparing with a truly balanced 50% chance to win, the p-value is 0. when comparing with historical imbalances, (which we dont know, so i substituted 55% for the historical "race imbalance equilibrium"), its 0.0059. Yea I know, that is one ridiculously low p-value. | ||
|
motbob
United States12546 Posts
On October 21 2009 03:34 Day[9] wrote: i'm reading so much about standard deviation what happened to null hypothesis tests? : [ You need the SD to perform a null hypothesis test. | ||
|
motbob
United States12546 Posts
On October 21 2009 03:46 EtherealDeath wrote: Would be cool though to get the race matchup elo of each player in each game and then use that to determine the probability of the zerg winning each game, and then use that to determine the probability of zerg winning at least as many games as they did. If only there were a simple to use automated process for this... EDIT: wait nvm, the matchup elo of each player itself would be dependent on the balance, and then I am not sure how you resolve this interdepency of variables >.> Perhaps instead we look at the matchup elos of the players of each race, and compare the resulting distributions? EDIT2: need more minor league games, there aren't THAT many players with enough games for me to trust in their elo -.- Hahahaha statistics nerds unite! This whole concept would make for a really cool empirical paper... I wouldn't want to try to do literature review though LOL | ||
|
EtherealDeath
United States8366 Posts
On October 21 2009 03:44 zulu_nation8 wrote: if i was to plot the data of zvp win % over every 800 games in the history of bw, and find the standard deviation. And then plug that into a z test for the current 800 game period and have the null be 55%, would that be a better test to explain if the current trend is significant? I'd imagine that taking games in a 400 game radius around each game, and plotting the win % in that range continuously would be better. That way, we have ~30k data points. | ||
|
EtherealDeath
United States8366 Posts
On October 21 2009 03:52 motbob wrote: Hahahaha statistics nerds unite! This whole concept would make for a really cool empirical paper... I wouldn't want to try to do literature review though LOL I'm a comp sci nerd who took stats for fun with my friend who is a stats major ;( | ||
|
zulu_nation8
China26351 Posts
On October 21 2009 03:53 EtherealDeath wrote: I'd imagine that taking games in a 400 game radius around each game, and plotting the win % in that range continuously would be better. That way, we have ~30k data points. right so if the current sample comes out as insignificant what would that mean? And what does the test black gun did mean over a sample of 800 games? | ||
|
Black Gun
Germany4482 Posts
On October 21 2009 03:46 EtherealDeath wrote: Would be cool though to get the race matchup elo of each player in each game and then use that to determine the probability of the zerg winning each game, and then use that to determine the probability of zerg winning at least as many games as they did. If only there were a simple to use automated process for this... still, there would remain the question how to scale elo differences. same elo means winning percentage of 50%, obviously. but how strong is the effect of lets say 50 points difference in elo? and does the effect of elo differences change for different elo regions? to tackle this, u would need to perform a logistic regression that would surely be interesting, but i dont think many guys would understand it, especially if the effect of elo differences is not constant over time, ie would have to be modelled nonparametrically. | ||
| ||
StarCraft 2 StarCraft: Brood War Rain League of Legends 3519GuemChi 1841Shuttle 1182Hyuk  371ProTech146 Rush  123Larva 110sOs 22ZergMaN 19Icarus 9Counter-Strike Super Smash Bros Other Games summit1g11506 WinterStarcraft335 Cr1tdota295 C9.Mang0293 Fuzer  89NeuroSwarm85 ViBE52 ROOTIllusion16 |
|
The PondCast
KCM Race Survival
OSC
IntoTheTV X SOOP
Reynor vs Solar
ByuN vs SHIN
Clem vs MaxPax
herO vs Rogue
IPSL
WorsT vs nOmaD
Dewalt vs andersOn
AI Arena Tournament
IntoTheTV X SOOP
IPSL
Azhi_Dahaki vs ggnore13
Double vs WIZARD
Replay Cast
Replay Cast
[ Show More ] |
|
|
EPT
