|
On October 21 2009 03:55 Black Gun wrote:Show nested quote +On October 21 2009 03:46 EtherealDeath wrote:
Would be cool though to get the race matchup elo of each player in each game and then use that to determine the probability of the zerg winning each game, and then use that to determine the probability of zerg winning at least as many games as they did. If only there were a simple to use automated process for this... still, there would remain the question how to scale elo differences. same elo means winning percentage of 50%, obviously. but how strong is the effect of lets say 50 points difference in elo? and does the effect of elo differences change for different elo regions? to tackle this, u would need to perform a logistic regression that would surely be interesting, but i dont think many guys would understand it, especially if the effect of elo differences is not constant over time, ie would have to be modelled nonparametrically.
ELO Difference Calculation
Well, about the thought concerning graphing player elos instead. Presumably the charts should look pretty similar if ZvP were balanced. Now, since we don't have enough pro games to do this well, what if we took every A level and above ICCUP game as well? Don't know how closely that skill level compares to the average progamer though.
|
On October 21 2009 03:55 zulu_nation8 wrote:Show nested quote +On October 21 2009 03:53 EtherealDeath wrote:On October 21 2009 03:44 zulu_nation8 wrote:On October 21 2009 03:40 Black Gun wrote:On October 21 2009 03:34 Day[9] wrote:
i'm reading so much about standard deviation
what happened to null hypothesis tests? : [ they were performed  the result: the zvp winning percentage of the last 7 months significantly exceeds 55%. so even if the historical race imbalance would be as high as 55% zerg wins, the recent trend would still be much higher than that, so that it cant be explained as a fluke. if i was to plot the data of zvp win % over every 800 games in the history of bw, and find the standard deviation. And then plug that into a z test for the current 800 game period and have the null be 55%, would that be a better test to explain if the current trend is significant? I'd imagine that taking games in a 400 game radius around each game, and plotting the win % in that range continuously would be better. That way, we have ~30k data points. right so if the current sample comes out as insignificant what would that mean? And what does the test black gun did mean over a sample of 800 games?
If it were insignificant, then almost certainly the historical chance of a zerg beating a toss is pretty high, more so than you would expect if the matchup were balanced. The test black gun did was to determine the probability that zerg wins at least as much as they do, assuming they have an expected win % of 55%. That probability turned out to be just under 0.6%, which means for the usual significance levels, the null hypothesis of 55% must be rejected, and replaced by something higher.
|
On October 21 2009 04:05 EtherealDeath wrote:Show nested quote +On October 21 2009 03:55 zulu_nation8 wrote:On October 21 2009 03:53 EtherealDeath wrote:On October 21 2009 03:44 zulu_nation8 wrote:On October 21 2009 03:40 Black Gun wrote:On October 21 2009 03:34 Day[9] wrote:
i'm reading so much about standard deviation
what happened to null hypothesis tests? : [ they were performed the result: the zvp winning percentage of the last 7 months significantly exceeds 55%. so even if the historical race imbalance would be as high as 55% zerg wins, the recent trend would still be much higher than that, so that it cant be explained as a fluke. if i was to plot the data of zvp win % over every 800 games in the history of bw, and find the standard deviation. And then plug that into a z test for the current 800 game period and have the null be 55%, would that be a better test to explain if the current trend is significant? I'd imagine that taking games in a 400 game radius around each game, and plotting the win % in that range continuously would be better. That way, we have ~30k data points. right so if the current sample comes out as insignificant what would that mean? And what does the test black gun did mean over a sample of 800 games? If it were insignificant, then almost certainly the historical chance of a zerg beating a toss is pretty high, more so than you would expect if the matchup were balanced. The test black gun did was to determine the probability that zerg wins at least as much as they do, assuming they have an expected win % of 55%. That probability turned out to be just under 0.6%, which means for the usual significance levels, the null hypothesis of 55% must be rejected, and replaced by something higher.
ok but im still having trouble understanding how, if zerg is expected to win 55% of games, and they win 59% over 800 chances, theres a 0.6% chance of that happening. It would seem that it happens all the time in BW. Of course if you were to have a two sided coin and say heads is expected to come 55% of the time, it would see very improbable to get 59% over 800 times. So what would this mean to BW? That each game can not be counted as a separate event?
|
|
|
motbob
United States12546 Posts
On October 21 2009 04:10 zulu_nation8 wrote:Show nested quote +On October 21 2009 04:05 EtherealDeath wrote:On October 21 2009 03:55 zulu_nation8 wrote:On October 21 2009 03:53 EtherealDeath wrote:On October 21 2009 03:44 zulu_nation8 wrote:On October 21 2009 03:40 Black Gun wrote:On October 21 2009 03:34 Day[9] wrote:
i'm reading so much about standard deviation
what happened to null hypothesis tests? : [ they were performed the result: the zvp winning percentage of the last 7 months significantly exceeds 55%. so even if the historical race imbalance would be as high as 55% zerg wins, the recent trend would still be much higher than that, so that it cant be explained as a fluke. if i was to plot the data of zvp win % over every 800 games in the history of bw, and find the standard deviation. And then plug that into a z test for the current 800 game period and have the null be 55%, would that be a better test to explain if the current trend is significant? I'd imagine that taking games in a 400 game radius around each game, and plotting the win % in that range continuously would be better. That way, we have ~30k data points. right so if the current sample comes out as insignificant what would that mean? And what does the test black gun did mean over a sample of 800 games? If it were insignificant, then almost certainly the historical chance of a zerg beating a toss is pretty high, more so than you would expect if the matchup were balanced. The test black gun did was to determine the probability that zerg wins at least as much as they do, assuming they have an expected win % of 55%. That probability turned out to be just under 0.6%, which means for the usual significance levels, the null hypothesis of 55% must be rejected, and replaced by something higher. So what would this mean to BW? That each game can not be counted as a separate event?
Sorry... not sure what this means 
|
On October 21 2009 04:15 motbob wrote:Show nested quote +On October 21 2009 04:10 zulu_nation8 wrote:On October 21 2009 04:05 EtherealDeath wrote:On October 21 2009 03:55 zulu_nation8 wrote:On October 21 2009 03:53 EtherealDeath wrote:On October 21 2009 03:44 zulu_nation8 wrote:On October 21 2009 03:40 Black Gun wrote:On October 21 2009 03:34 Day[9] wrote:
i'm reading so much about standard deviation
what happened to null hypothesis tests? : [ they were performed the result: the zvp winning percentage of the last 7 months significantly exceeds 55%. so even if the historical race imbalance would be as high as 55% zerg wins, the recent trend would still be much higher than that, so that it cant be explained as a fluke. if i was to plot the data of zvp win % over every 800 games in the history of bw, and find the standard deviation. And then plug that into a z test for the current 800 game period and have the null be 55%, would that be a better test to explain if the current trend is significant? I'd imagine that taking games in a 400 game radius around each game, and plotting the win % in that range continuously would be better. That way, we have ~30k data points. right so if the current sample comes out as insignificant what would that mean? And what does the test black gun did mean over a sample of 800 games? If it were insignificant, then almost certainly the historical chance of a zerg beating a toss is pretty high, more so than you would expect if the matchup were balanced. The test black gun did was to determine the probability that zerg wins at least as much as they do, assuming they have an expected win % of 55%. That probability turned out to be just under 0.6%, which means for the usual significance levels, the null hypothesis of 55% must be rejected, and replaced by something higher. So what would this mean to BW? That each game can not be counted as a separate event? Sorry... not sure what this means
it means youre ignoring that not every game is played under the same conditions like a coin flip, if the test is to become more accurate, factors such as map, time, tv/non tv games would have to be taken under consideration, for example delete all the non official games from the 855 games and do another z-test and see what comes out.
|
On October 21 2009 04:10 zulu_nation8 wrote:Show nested quote +On October 21 2009 04:05 EtherealDeath wrote:On October 21 2009 03:55 zulu_nation8 wrote:On October 21 2009 03:53 EtherealDeath wrote:On October 21 2009 03:44 zulu_nation8 wrote:On October 21 2009 03:40 Black Gun wrote:On October 21 2009 03:34 Day[9] wrote:
i'm reading so much about standard deviation
what happened to null hypothesis tests? : [ they were performed the result: the zvp winning percentage of the last 7 months significantly exceeds 55%. so even if the historical race imbalance would be as high as 55% zerg wins, the recent trend would still be much higher than that, so that it cant be explained as a fluke. if i was to plot the data of zvp win % over every 800 games in the history of bw, and find the standard deviation. And then plug that into a z test for the current 800 game period and have the null be 55%, would that be a better test to explain if the current trend is significant? I'd imagine that taking games in a 400 game radius around each game, and plotting the win % in that range continuously would be better. That way, we have ~30k data points. right so if the current sample comes out as insignificant what would that mean? And what does the test black gun did mean over a sample of 800 games? If it were insignificant, then almost certainly the historical chance of a zerg beating a toss is pretty high, more so than you would expect if the matchup were balanced. The test black gun did was to determine the probability that zerg wins at least as much as they do, assuming they have an expected win % of 55%. That probability turned out to be just under 0.6%, which means for the usual significance levels, the null hypothesis of 55% must be rejected, and replaced by something higher. ok but im still having trouble understanding how, if zerg is expected to win 55% of games, and they win 59% over 800 chances, theres a 0.6% chance of that happening. It would seem that it happens all the time in BW. Of course if you were to have a two sided coin and say heads is expected to come 55% of the time, it would see very improbable to get 59% over 800 times. So what would this mean to BW? That each game can not be counted as a separate event?
Well, one problem I see with using all the data in TLPD and then constructing a probability distribution out of that are the map changes, and the changes in the metagame. It's not quite the same coin that we keep flipping, it changes over time. Though, if we consider that the metagame is mostly influenced by the maps (eh may or may not be true, but simplifies analysis), we could use that data to get the effect that maps have on the game. Of course, if the data were pretty consistent over the entire TLPD history, then ZvP is really balanced regardless of the maps, but I think we all know that's not true.
It's one of the reasons I prefer using just this 7 month period, with more or less consistent maps. Now we are flipping the same coin, and it seems to be zerg tilted.
One thing you can use though, using all the games, is to calculate the probability of zerg's win % being under 50% at any point in time, and compare that to toss's. Off the top of my head, they should be relatively close if it the chance/magnitude of map imbalance were not tilted toward zerg, but I'd have to be more careful about that before saying anything conclusive. Too many possible pitfalls.
|
On October 21 2009 04:18 zulu_nation8 wrote:Show nested quote +On October 21 2009 04:15 motbob wrote:On October 21 2009 04:10 zulu_nation8 wrote:On October 21 2009 04:05 EtherealDeath wrote:On October 21 2009 03:55 zulu_nation8 wrote:On October 21 2009 03:53 EtherealDeath wrote:On October 21 2009 03:44 zulu_nation8 wrote:On October 21 2009 03:40 Black Gun wrote:On October 21 2009 03:34 Day[9] wrote:
i'm reading so much about standard deviation
what happened to null hypothesis tests? : [ they were performed the result: the zvp winning percentage of the last 7 months significantly exceeds 55%. so even if the historical race imbalance would be as high as 55% zerg wins, the recent trend would still be much higher than that, so that it cant be explained as a fluke. if i was to plot the data of zvp win % over every 800 games in the history of bw, and find the standard deviation. And then plug that into a z test for the current 800 game period and have the null be 55%, would that be a better test to explain if the current trend is significant? I'd imagine that taking games in a 400 game radius around each game, and plotting the win % in that range continuously would be better. That way, we have ~30k data points. right so if the current sample comes out as insignificant what would that mean? And what does the test black gun did mean over a sample of 800 games? If it were insignificant, then almost certainly the historical chance of a zerg beating a toss is pretty high, more so than you would expect if the matchup were balanced. The test black gun did was to determine the probability that zerg wins at least as much as they do, assuming they have an expected win % of 55%. That probability turned out to be just under 0.6%, which means for the usual significance levels, the null hypothesis of 55% must be rejected, and replaced by something higher. So what would this mean to BW? That each game can not be counted as a separate event? Sorry... not sure what this means it means youre ignoring that not every game is played under the same conditions like a coin flip, if the test is to become more accurate, factors such as map, time, tv/non tv games would have to be taken under consideration, for example delete all the non official games from the 855 games and do another z-test and see what comes out.
We could just do say... HBR only, or Desti only and see. I don't have any stats software with me though ;/
Pretty sure the stats will come out heavily zerg favored though.
|
On October 21 2009 04:21 EtherealDeath wrote:Show nested quote +On October 21 2009 04:10 zulu_nation8 wrote:On October 21 2009 04:05 EtherealDeath wrote:On October 21 2009 03:55 zulu_nation8 wrote:On October 21 2009 03:53 EtherealDeath wrote:On October 21 2009 03:44 zulu_nation8 wrote:On October 21 2009 03:40 Black Gun wrote:On October 21 2009 03:34 Day[9] wrote:
i'm reading so much about standard deviation
what happened to null hypothesis tests? : [ they were performed the result: the zvp winning percentage of the last 7 months significantly exceeds 55%. so even if the historical race imbalance would be as high as 55% zerg wins, the recent trend would still be much higher than that, so that it cant be explained as a fluke. if i was to plot the data of zvp win % over every 800 games in the history of bw, and find the standard deviation. And then plug that into a z test for the current 800 game period and have the null be 55%, would that be a better test to explain if the current trend is significant? I'd imagine that taking games in a 400 game radius around each game, and plotting the win % in that range continuously would be better. That way, we have ~30k data points. right so if the current sample comes out as insignificant what would that mean? And what does the test black gun did mean over a sample of 800 games? If it were insignificant, then almost certainly the historical chance of a zerg beating a toss is pretty high, more so than you would expect if the matchup were balanced. The test black gun did was to determine the probability that zerg wins at least as much as they do, assuming they have an expected win % of 55%. That probability turned out to be just under 0.6%, which means for the usual significance levels, the null hypothesis of 55% must be rejected, and replaced by something higher. ok but im still having trouble understanding how, if zerg is expected to win 55% of games, and they win 59% over 800 chances, theres a 0.6% chance of that happening. It would seem that it happens all the time in BW. Of course if you were to have a two sided coin and say heads is expected to come 55% of the time, it would see very improbable to get 59% over 800 times. So what would this mean to BW? That each game can not be counted as a separate event? Well, one problem I see with using all the data in TLPD and then constructing a probability distribution out of that are the map changes, and the changes in the metagame. It's not quite the same coin that we keep flipping, it changes over time. Though, if we consider that the metagame is mostly influenced by the maps (eh may or may not be true, but simplifies analysis), we could use that data to get the effect that maps have on the game. Of course, if the data were pretty consistent over the entire TLPD history, then ZvP is really balanced regardless of the maps, but I think we all know that's not true. It's one of the reasons I prefer using just this 7 month period, with more or less consistent maps. Now we are flipping the same coin, and it seems to be zerg tilted. One thing you can use though, using all the games, is to calculate the probability of zerg's win % being under 50% at any point in time, and compare that to toss's. Off the top of my head, they should be relatively close if it the chance/magnitude of map imbalance were not tilted toward zerg, but I'd have to be more careful about that before saying anything conclusive. Too many possible pitfalls.
Lots of factors would have to be limited I agree. A test would have to be done by time, and another by maps I guess. Also we have to set the standard for tv/non tv games, minor league/non minor league games, prelim/no prelim, etc.
|
motbob
United States12546 Posts
On October 21 2009 04:18 zulu_nation8 wrote:Show nested quote +On October 21 2009 04:15 motbob wrote:On October 21 2009 04:10 zulu_nation8 wrote:On October 21 2009 04:05 EtherealDeath wrote:On October 21 2009 03:55 zulu_nation8 wrote:On October 21 2009 03:53 EtherealDeath wrote:On October 21 2009 03:44 zulu_nation8 wrote:On October 21 2009 03:40 Black Gun wrote:On October 21 2009 03:34 Day[9] wrote:
i'm reading so much about standard deviation
what happened to null hypothesis tests? : [ they were performed the result: the zvp winning percentage of the last 7 months significantly exceeds 55%. so even if the historical race imbalance would be as high as 55% zerg wins, the recent trend would still be much higher than that, so that it cant be explained as a fluke. if i was to plot the data of zvp win % over every 800 games in the history of bw, and find the standard deviation. And then plug that into a z test for the current 800 game period and have the null be 55%, would that be a better test to explain if the current trend is significant? I'd imagine that taking games in a 400 game radius around each game, and plotting the win % in that range continuously would be better. That way, we have ~30k data points. right so if the current sample comes out as insignificant what would that mean? And what does the test black gun did mean over a sample of 800 games? If it were insignificant, then almost certainly the historical chance of a zerg beating a toss is pretty high, more so than you would expect if the matchup were balanced. The test black gun did was to determine the probability that zerg wins at least as much as they do, assuming they have an expected win % of 55%. That probability turned out to be just under 0.6%, which means for the usual significance levels, the null hypothesis of 55% must be rejected, and replaced by something higher. So what would this mean to BW? That each game can not be counted as a separate event? Sorry... not sure what this means it means youre ignoring that not every game is played under the same conditions like a coin flip, if the test is to become more accurate, factors such as map, time, tv/non tv games would have to be taken under consideration, for example delete all the non official games from the 855 games and do another z-test and see what comes out.
I don't see any evidence that playing on TV/ not playing on TV would have any impact to skew the results towards zerg or towards protoss. In the absence of any such evidence, there's no reason to redo the test.
|
On October 21 2009 04:22 EtherealDeath wrote:Show nested quote +On October 21 2009 04:18 zulu_nation8 wrote:On October 21 2009 04:15 motbob wrote:On October 21 2009 04:10 zulu_nation8 wrote:On October 21 2009 04:05 EtherealDeath wrote:On October 21 2009 03:55 zulu_nation8 wrote:On October 21 2009 03:53 EtherealDeath wrote:On October 21 2009 03:44 zulu_nation8 wrote:On October 21 2009 03:40 Black Gun wrote:On October 21 2009 03:34 Day[9] wrote:
i'm reading so much about standard deviation
what happened to null hypothesis tests? : [ they were performed the result: the zvp winning percentage of the last 7 months significantly exceeds 55%. so even if the historical race imbalance would be as high as 55% zerg wins, the recent trend would still be much higher than that, so that it cant be explained as a fluke. if i was to plot the data of zvp win % over every 800 games in the history of bw, and find the standard deviation. And then plug that into a z test for the current 800 game period and have the null be 55%, would that be a better test to explain if the current trend is significant? I'd imagine that taking games in a 400 game radius around each game, and plotting the win % in that range continuously would be better. That way, we have ~30k data points. right so if the current sample comes out as insignificant what would that mean? And what does the test black gun did mean over a sample of 800 games? If it were insignificant, then almost certainly the historical chance of a zerg beating a toss is pretty high, more so than you would expect if the matchup were balanced. The test black gun did was to determine the probability that zerg wins at least as much as they do, assuming they have an expected win % of 55%. That probability turned out to be just under 0.6%, which means for the usual significance levels, the null hypothesis of 55% must be rejected, and replaced by something higher. So what would this mean to BW? That each game can not be counted as a separate event? Sorry... not sure what this means it means youre ignoring that not every game is played under the same conditions like a coin flip, if the test is to become more accurate, factors such as map, time, tv/non tv games would have to be taken under consideration, for example delete all the non official games from the 855 games and do another z-test and see what comes out. We could just do say... HBR only, or Desti only and see. I don't have any stats software with me though ;/ Pretty sure the stats will come out heavily zerg favored though.
It would come out zerg favored but I highly doubt Desti would be called an "imbalanced map." Since the standard deviation for all maps with a minimum of like 30 games played would be much higher than 5% or whatever Desti is from 55%.
|
zulu_nation8 you're not adding anything to the discussion with your random comments. Read up on some basic statistics if you want to try to disprove what people have done so far.
|
On October 21 2009 04:29 zulu_nation8 wrote:Show nested quote +On October 21 2009 04:22 EtherealDeath wrote:On October 21 2009 04:18 zulu_nation8 wrote:On October 21 2009 04:15 motbob wrote:On October 21 2009 04:10 zulu_nation8 wrote:On October 21 2009 04:05 EtherealDeath wrote:On October 21 2009 03:55 zulu_nation8 wrote:On October 21 2009 03:53 EtherealDeath wrote:On October 21 2009 03:44 zulu_nation8 wrote:On October 21 2009 03:40 Black Gun wrote:[quote] they were performed the result: the zvp winning percentage of the last 7 months significantly exceeds 55%. so even if the historical race imbalance would be as high as 55% zerg wins, the recent trend would still be much higher than that, so that it cant be explained as a fluke. if i was to plot the data of zvp win % over every 800 games in the history of bw, and find the standard deviation. And then plug that into a z test for the current 800 game period and have the null be 55%, would that be a better test to explain if the current trend is significant? I'd imagine that taking games in a 400 game radius around each game, and plotting the win % in that range continuously would be better. That way, we have ~30k data points. right so if the current sample comes out as insignificant what would that mean? And what does the test black gun did mean over a sample of 800 games? If it were insignificant, then almost certainly the historical chance of a zerg beating a toss is pretty high, more so than you would expect if the matchup were balanced. The test black gun did was to determine the probability that zerg wins at least as much as they do, assuming they have an expected win % of 55%. That probability turned out to be just under 0.6%, which means for the usual significance levels, the null hypothesis of 55% must be rejected, and replaced by something higher. So what would this mean to BW? That each game can not be counted as a separate event? Sorry... not sure what this means it means youre ignoring that not every game is played under the same conditions like a coin flip, if the test is to become more accurate, factors such as map, time, tv/non tv games would have to be taken under consideration, for example delete all the non official games from the 855 games and do another z-test and see what comes out. We could just do say... HBR only, or Desti only and see. I don't have any stats software with me though ;/ Pretty sure the stats will come out heavily zerg favored though. It would come out zerg favored but I highly doubt Desti would be called an "imbalanced map." Since the standard deviation for all maps with a minimum of like 30 games played would be much higher than 5% or whatever Desti is from 55%.
Eh yeah I suppose the p-value might not be low enough to reject the 55% hypothesis using any reasonable alpha.
|
On October 21 2009 04:25 motbob wrote:Show nested quote +On October 21 2009 04:18 zulu_nation8 wrote:On October 21 2009 04:15 motbob wrote:On October 21 2009 04:10 zulu_nation8 wrote:On October 21 2009 04:05 EtherealDeath wrote:On October 21 2009 03:55 zulu_nation8 wrote:On October 21 2009 03:53 EtherealDeath wrote:On October 21 2009 03:44 zulu_nation8 wrote:On October 21 2009 03:40 Black Gun wrote:On October 21 2009 03:34 Day[9] wrote:
i'm reading so much about standard deviation
what happened to null hypothesis tests? : [ they were performed the result: the zvp winning percentage of the last 7 months significantly exceeds 55%. so even if the historical race imbalance would be as high as 55% zerg wins, the recent trend would still be much higher than that, so that it cant be explained as a fluke. if i was to plot the data of zvp win % over every 800 games in the history of bw, and find the standard deviation. And then plug that into a z test for the current 800 game period and have the null be 55%, would that be a better test to explain if the current trend is significant? I'd imagine that taking games in a 400 game radius around each game, and plotting the win % in that range continuously would be better. That way, we have ~30k data points. right so if the current sample comes out as insignificant what would that mean? And what does the test black gun did mean over a sample of 800 games? If it were insignificant, then almost certainly the historical chance of a zerg beating a toss is pretty high, more so than you would expect if the matchup were balanced. The test black gun did was to determine the probability that zerg wins at least as much as they do, assuming they have an expected win % of 55%. That probability turned out to be just under 0.6%, which means for the usual significance levels, the null hypothesis of 55% must be rejected, and replaced by something higher. So what would this mean to BW? That each game can not be counted as a separate event? Sorry... not sure what this means it means youre ignoring that not every game is played under the same conditions like a coin flip, if the test is to become more accurate, factors such as map, time, tv/non tv games would have to be taken under consideration, for example delete all the non official games from the 855 games and do another z-test and see what comes out. I don't see any evidence that playing on TV/ not playing on TV would have any impact to skew the results towards zerg or towards protoss. In the absence of any such evidence, there's no reason to redo the test.
do it with only official games and you'll see that the Z score will be much lower, theres your evidence that its a factor to be considered, actually i'll do it when I have time since youre still not getting why your SD is wrong.
|
motbob
United States12546 Posts
On October 21 2009 04:29 zulu_nation8 wrote:Show nested quote +On October 21 2009 04:22 EtherealDeath wrote:On October 21 2009 04:18 zulu_nation8 wrote:On October 21 2009 04:15 motbob wrote:On October 21 2009 04:10 zulu_nation8 wrote:On October 21 2009 04:05 EtherealDeath wrote:On October 21 2009 03:55 zulu_nation8 wrote:On October 21 2009 03:53 EtherealDeath wrote:On October 21 2009 03:44 zulu_nation8 wrote:On October 21 2009 03:40 Black Gun wrote:[quote] they were performed the result: the zvp winning percentage of the last 7 months significantly exceeds 55%. so even if the historical race imbalance would be as high as 55% zerg wins, the recent trend would still be much higher than that, so that it cant be explained as a fluke. if i was to plot the data of zvp win % over every 800 games in the history of bw, and find the standard deviation. And then plug that into a z test for the current 800 game period and have the null be 55%, would that be a better test to explain if the current trend is significant? I'd imagine that taking games in a 400 game radius around each game, and plotting the win % in that range continuously would be better. That way, we have ~30k data points. right so if the current sample comes out as insignificant what would that mean? And what does the test black gun did mean over a sample of 800 games? If it were insignificant, then almost certainly the historical chance of a zerg beating a toss is pretty high, more so than you would expect if the matchup were balanced. The test black gun did was to determine the probability that zerg wins at least as much as they do, assuming they have an expected win % of 55%. That probability turned out to be just under 0.6%, which means for the usual significance levels, the null hypothesis of 55% must be rejected, and replaced by something higher. So what would this mean to BW? That each game can not be counted as a separate event? Sorry... not sure what this means it means youre ignoring that not every game is played under the same conditions like a coin flip, if the test is to become more accurate, factors such as map, time, tv/non tv games would have to be taken under consideration, for example delete all the non official games from the 855 games and do another z-test and see what comes out. We could just do say... HBR only, or Desti only and see. I don't have any stats software with me though ;/ Pretty sure the stats will come out heavily zerg favored though. It would come out zerg favored but I highly doubt Desti would be called an "imbalanced map." Since the standard deviation for all maps with a minimum of like 30 games played would be much higher than 5% or whatever Desti is from 55%.
z-test on desti since March 1st! BRB
|
motbob
United States12546 Posts
On October 21 2009 04:30 zulu_nation8 wrote:Show nested quote +On October 21 2009 04:25 motbob wrote:On October 21 2009 04:18 zulu_nation8 wrote:On October 21 2009 04:15 motbob wrote:On October 21 2009 04:10 zulu_nation8 wrote:On October 21 2009 04:05 EtherealDeath wrote:On October 21 2009 03:55 zulu_nation8 wrote:On October 21 2009 03:53 EtherealDeath wrote:On October 21 2009 03:44 zulu_nation8 wrote:On October 21 2009 03:40 Black Gun wrote:[quote] they were performed the result: the zvp winning percentage of the last 7 months significantly exceeds 55%. so even if the historical race imbalance would be as high as 55% zerg wins, the recent trend would still be much higher than that, so that it cant be explained as a fluke. if i was to plot the data of zvp win % over every 800 games in the history of bw, and find the standard deviation. And then plug that into a z test for the current 800 game period and have the null be 55%, would that be a better test to explain if the current trend is significant? I'd imagine that taking games in a 400 game radius around each game, and plotting the win % in that range continuously would be better. That way, we have ~30k data points. right so if the current sample comes out as insignificant what would that mean? And what does the test black gun did mean over a sample of 800 games? If it were insignificant, then almost certainly the historical chance of a zerg beating a toss is pretty high, more so than you would expect if the matchup were balanced. The test black gun did was to determine the probability that zerg wins at least as much as they do, assuming they have an expected win % of 55%. That probability turned out to be just under 0.6%, which means for the usual significance levels, the null hypothesis of 55% must be rejected, and replaced by something higher. So what would this mean to BW? That each game can not be counted as a separate event? Sorry... not sure what this means it means youre ignoring that not every game is played under the same conditions like a coin flip, if the test is to become more accurate, factors such as map, time, tv/non tv games would have to be taken under consideration, for example delete all the non official games from the 855 games and do another z-test and see what comes out. I don't see any evidence that playing on TV/ not playing on TV would have any impact to skew the results towards zerg or towards protoss. In the absence of any such evidence, there's no reason to redo the test. do it with only official games and you'll see that the Z score will be much lower, theres your evidence that its a factor to be considered, actually i'll do it when I have time since youre still not getting why your SD is wrong.
I'm not getting it cause you're not doing your own test and showing me. Until you do that I'll never understand.
|
with a complete dataset from tlpd, one could perform a logistic regression with elo, maps and time as covariates. thus we could distinguish the effects of these factors. time would account for the metagame shifts.
edit: with the corresponding dataset i could easily carry out this stuff, i got access to stats software and know what im doing and so on
|
How would you use ELO exactly? Wouldn't you need to compare PvZ ELO with the other matchup ELOs to get some idea of the historic differences, otherwise a shift in the PvZ meta game will be hidden by the apparently higher ELOs of the Zerg players.
|
motbob
United States12546 Posts
I am going to do this step by step so that there's no question that I'm doing it right. Here we go! Remember, this is a test to see whether the zerg winrate on Destination since March 1st is significantly different from a) the historical zerg win rate against P or b) a hypothetical 50% winrate.
Wikipedia has a great screenshot of the formula for a z-test:
In this equation, we're looking for a z value that has an absolute value greater than 2. In that case, the p-value would be less than 0.05, which is what most statisticians take to be statistically significant.
The x with the bar over it is the mean of our data. Our data is a bunch of zeros and ones corresponding to the wins and losses of zergs against protoss. Therefore, the mean of our data will be equal to the winrate of ZvP. Let's head over to excel.
In Excel, I punch in our data points. We have one data point for each game: a 1 if the zerg wins, and a 0 if the zerg loses. That means that we're going to have 110+72=182 rows in the column we're using. 110 of those rows will contain a 1, and 72 of the rows will contain a 0. It doesn't matter what order the numbers are in.
Now I'm going to type in the formula "=average(A1:A182)" to get the average of this data. The average is 0.604395604. I have put it in cell B182.
μ0 is the hypothesized population mean. This is where we plug in the null hypothesis. Do I want to use 50% or the historical zerg winrate? I'll do a test with both, and I'll assume the historical zerg winrate to be 53%. I have put 0.5 in cell C182 and 0.53 in cell D182.
σ is the population standard deviation. Getting this is very simple now that we have our data plugged into excel. I'm going to type in the formula "=stdev(A1:A182)" to get the population standard deviation. It's 0.490329033, and it's in cell E182.
n is the number of data points, or 182. I put 182 in cell F182.
Now that we have all the varibles, we can do the test! I'm going to type in the formula at the very top of this post into excel: "=(B182-C182)/(E182/SQRT(F182))"
Excel spits out a result of 2.872303303, which gives a p-value of less than 0.05.
If we use the historical estimate of the zerg winrate, excel spits out a z-value of 2.046894038, which is still statistically significant.
If we were to use a historical zerg winrate of 55%, excel would NOT give us a statistically significant result.
|
Katowice25012 Posts
YOUR STANDARD DEVIATION CANNOT BE 49%
THAT IS NOT YOUR VARIATION
YOU ARE LOOKING AT ONE DATA POINT
THERE IS NO OTHER WAY I CAN SAY THIS SIMPLY, UNLESS I DO IT IN CAPS PLUS BOLD
|
|
|
|
|
|
EPT![[image loading]](http://upload.wikimedia.org/math/a/f/b/afbae1fd562813a91221818647285b51.png)
