|
motbob
United States12546 Posts
On October 21 2009 02:21 zulu_nation8 wrote:Show nested quote +On October 21 2009 02:11 Black Gun wrote:On October 21 2009 02:03 JWD wrote:
motbob I'm going to try to explain exactly why your standard deviation (technically it's a standard error, since standard deviation is a "true", unascertainable value and we are just estimating it) is wrong, since zulu won't do it. I think what you did is to calculate the standard error of the variable zerg win, which is a binary variable you defined (I picked the name for exposition's sake) that equals 1 if Zerg wins a ZvP and 0 if Protoss wins. You correctly calculated the standard error of this variable — we'd expect it to be near .5 because the mean of zerg win is about .5, and so each instance of zerg win is about .5 from that mean.
However, this standard error is not the standard error relevant to your test for determining whether the recent Z>P trend is significantly anomalous. The variable you are examining in that test is not zerg win, but ZvP balance over a several-month period, another variable which I'll call balance. Therefore the standard error you must use in your test is the standard error of balance—that is, the error of several-month ZvP balance from the mean several-month ZvP balance. You can NOT use the standard error of zerg win, which has no place in your calculation. u should read my last 2 posts. in the first one i conducted the correct tests. in the second one i explained in detail why the standard error i was using is the correct one. and no, it is not hard to compute the sd of "balance". once we get a certain zvp winning percentage as the "historical balance", the sd needed in our test is simply sqrt[p*(1-p)/n]. i already tried it with 55%, so if the historical zvp stats are not higher than 55%, then the outcome of the last 7 months differs significantly. im a statistics major close to graduating, so u can believe me  I'm having trouble believing that the trend is significant not because I don't trust your math but just by what I remember. I'm very confident there has been similar trends in the past over similar samples, and if we were to look at the stats of other matchups, something like 59% over 7 months really shouldnt be very surprising. Also, can you explain what 885 means in the equation? Like if the overall games are 30k+, is there a way to include the size of the sample?
885 is the number of ZvPs played since March 1st. We plug it into the z test equation for n.
|
motbob
United States12546 Posts
On October 21 2009 02:27 zulu_nation8 wrote:Show nested quote +On October 21 2009 02:19 JWD wrote:On October 21 2009 02:18 motbob wrote:On October 21 2009 02:03 JWD wrote:
motbob I'm going to try to explain exactly why your standard deviation (technically it's a standard error, since standard deviation is a "true", unascertainable value and we are just estimating it) is wrong, since zulu won't do it. I think what you did is to calculate the standard error of the variable zerg win, which is a binary variable you defined (I picked the name for exposition's sake) that equals 1 if Zerg wins a ZvP and 0 if Protoss wins. You correctly calculated the standard error of this variable — we'd expect it to be near .5 because the mean of zerg win is about .5, and so each instance of zerg win is about .5 from that mean.
However, this standard error is not the standard error relevant to your test for determining whether the recent Z>P trend is significantly anomalous. The variable you are examining in that test is not zerg win, but ZvP balance over a several-month period, another variable which I'll call balance. Therefore the standard error you must use in your test is the standard error of balance—that is, the error of several-month ZvP balance from the mean several-month ZvP balance. You can NOT use the standard error of zerg win, which has no place in your calculation. ...I don't have a choice as to which SE I use in my test. SE is SD (of my data) divided by sqrt(n). I can't change it. I can change my null hypothesis though... are you saying my null hypothesis should be the historical winrate instead of 50%? yeah I confused standard error and stdev, and just edited to fix that…no that's not what I'm saying. I'm too rusty on stats to make any further useful contributions to this thread, but I'm pretty sure I explained your problem right motbob motbob just think of it like this, how can the MEAN of zerg win% ever be 100% over similar samples? Surely theres never been a period in progaming when zerg has won every game vs toss over 800 games?
Uh yeah this is true but I dunno why it's relevant. Again, I think you're confusing the standard error involved in a statistical test with the standard deviation of a population.
|
On October 21 2009 02:26 Muirhead wrote:
It really doesn't matter how many games were played in the past Zulu_nation.
Like if you roll a dice 3 billion times you can get a good idea of how its weighted.
If you change something and then roll it a million times you can still get a good idea of whether the weighting significantly changed, even though a million is a tiny fraction of 3 billion.
ok let me relearn this stuff and I'll get back to you.
|
On October 21 2009 02:21 zulu_nation8 wrote:Show nested quote +On October 21 2009 02:11 Black Gun wrote:On October 21 2009 02:03 JWD wrote:
motbob I'm going to try to explain exactly why your standard deviation (technically it's a standard error, since standard deviation is a "true", unascertainable value and we are just estimating it) is wrong, since zulu won't do it. I think what you did is to calculate the standard error of the variable zerg win, which is a binary variable you defined (I picked the name for exposition's sake) that equals 1 if Zerg wins a ZvP and 0 if Protoss wins. You correctly calculated the standard error of this variable — we'd expect it to be near .5 because the mean of zerg win is about .5, and so each instance of zerg win is about .5 from that mean.
However, this standard error is not the standard error relevant to your test for determining whether the recent Z>P trend is significantly anomalous. The variable you are examining in that test is not zerg win, but ZvP balance over a several-month period, another variable which I'll call balance. Therefore the standard error you must use in your test is the standard error of balance—that is, the error of several-month ZvP balance from the mean several-month ZvP balance. You can NOT use the standard error of zerg win, which has no place in your calculation. u should read my last 2 posts. in the first one i conducted the correct tests. in the second one i explained in detail why the standard error i was using is the correct one. and no, it is not hard to compute the sd of "balance". once we get a certain zvp winning percentage as the "historical balance", the sd needed in our test is simply sqrt[p*(1-p)/n]. i already tried it with 55%, so if the historical zvp stats are not higher than 55%, then the outcome of the last 7 months differs significantly. im a statistics major close to graduating, so u can believe me I'm having trouble believing that the trend is significant not because I don't trust your math but just by what I remember. I'm very confident there has been similar trends in the past over similar samples, and if we were to look at the stats of other matchups, something like 59% over 7 months really shouldnt be very surprising. Also, can you explain what 885 means in the equation? Like if the overall games are 30k+, is there a way to include the size of the sample?
i was refering to the figures from the last page, which are the figures for the last 7 months, progaming only. there we had 524 zerg wins to 361 toss wins. thats 885 games played. thats the sample from which we estimate the winning percentage of the last 7 months. in the tests conducted so far, we assume that the historical balance with which we compare the last 7 months is given. if u want to include the uncertainty involved because we estimate this historical balance zvp winning ratio from data, u would have to conduct a two-sample-test. but as u said, there are like several thousands of games from which we would estimate the historical percentage, thus the uncertainty involved in this estimation is negligible.
but u are right in one point: the metagame has always been shifting, it goes up and down. comparing with the average of these up and down movements can be misleading. the decisive question is if the current shift in metagame in favour of zerg is more severe than previous shifts. in other words: is it even worse than the previous bad times for protoss, or is it comparable to them?
|
On October 21 2009 02:28 motbob wrote:Show nested quote +On October 21 2009 02:27 zulu_nation8 wrote:On October 21 2009 02:19 JWD wrote:On October 21 2009 02:18 motbob wrote:On October 21 2009 02:03 JWD wrote:
motbob I'm going to try to explain exactly why your standard deviation (technically it's a standard error, since standard deviation is a "true", unascertainable value and we are just estimating it) is wrong, since zulu won't do it. I think what you did is to calculate the standard error of the variable zerg win, which is a binary variable you defined (I picked the name for exposition's sake) that equals 1 if Zerg wins a ZvP and 0 if Protoss wins. You correctly calculated the standard error of this variable — we'd expect it to be near .5 because the mean of zerg win is about .5, and so each instance of zerg win is about .5 from that mean.
However, this standard error is not the standard error relevant to your test for determining whether the recent Z>P trend is significantly anomalous. The variable you are examining in that test is not zerg win, but ZvP balance over a several-month period, another variable which I'll call balance. Therefore the standard error you must use in your test is the standard error of balance—that is, the error of several-month ZvP balance from the mean several-month ZvP balance. You can NOT use the standard error of zerg win, which has no place in your calculation. ...I don't have a choice as to which SE I use in my test. SE is SD (of my data) divided by sqrt(n). I can't change it. I can change my null hypothesis though... are you saying my null hypothesis should be the historical winrate instead of 50%? yeah I confused standard error and stdev, and just edited to fix that…no that's not what I'm saying. I'm too rusty on stats to make any further useful contributions to this thread, but I'm pretty sure I explained your problem right motbob motbob just think of it like this, how can the MEAN of zerg win% ever be 100% over similar samples? Surely theres never been a period in progaming when zerg has won every game vs toss over 800 games? Uh yeah this is true but I dunno why it's relevant. Again, I think you're confusing the standard error involved in a statistical test with the standard deviation of a population.
no im not, like JWD says, your standard deviation is the mean of zerg winning, so of course zerg can only win or not win, and not 80% win. What we're actually calculating is the mean of the zerg win RATIO, in that case the SD would be something like how far on average does the zvp % deviate from the overall historical mean of say 55% over similar 855 game samples.
|
motbob
United States12546 Posts
On October 21 2009 02:36 zulu_nation8 wrote:Show nested quote +On October 21 2009 02:28 motbob wrote:On October 21 2009 02:27 zulu_nation8 wrote:On October 21 2009 02:19 JWD wrote:On October 21 2009 02:18 motbob wrote:On October 21 2009 02:03 JWD wrote:
motbob I'm going to try to explain exactly why your standard deviation (technically it's a standard error, since standard deviation is a "true", unascertainable value and we are just estimating it) is wrong, since zulu won't do it. I think what you did is to calculate the standard error of the variable zerg win, which is a binary variable you defined (I picked the name for exposition's sake) that equals 1 if Zerg wins a ZvP and 0 if Protoss wins. You correctly calculated the standard error of this variable — we'd expect it to be near .5 because the mean of zerg win is about .5, and so each instance of zerg win is about .5 from that mean.
However, this standard error is not the standard error relevant to your test for determining whether the recent Z>P trend is significantly anomalous. The variable you are examining in that test is not zerg win, but ZvP balance over a several-month period, another variable which I'll call balance. Therefore the standard error you must use in your test is the standard error of balance—that is, the error of several-month ZvP balance from the mean several-month ZvP balance. You can NOT use the standard error of zerg win, which has no place in your calculation. ...I don't have a choice as to which SE I use in my test. SE is SD (of my data) divided by sqrt(n). I can't change it. I can change my null hypothesis though... are you saying my null hypothesis should be the historical winrate instead of 50%? yeah I confused standard error and stdev, and just edited to fix that…no that's not what I'm saying. I'm too rusty on stats to make any further useful contributions to this thread, but I'm pretty sure I explained your problem right motbob motbob just think of it like this, how can the MEAN of zerg win% ever be 100% over similar samples? Surely theres never been a period in progaming when zerg has won every game vs toss over 800 games? Uh yeah this is true but I dunno why it's relevant. Again, I think you're confusing the standard error involved in a statistical test with the standard deviation of a population. no im not, like JWD says, your standard deviation is the mean of zerg winning
no it isn't lol
|
let me explain that, your SD is how far your data deviates from the possible outcomes, out of win or not win, for a zerg progamer when he enters a 1v1 starcraft game, whereas the rest of your equation is calculating the win ratio of a certain number of games, and basically not binary data.
|
motbob honestly i probably havent used a calculator in three years and Im also not an econ major so I understand why you wouldnt believe me but please, just use common sense, how can the standard deviation be 50% in this case? How is that possible? The SD is a mean.
|
Should just get all the TLPD data and calcaute the probability of a P's win chance being 48-52% (or some other arbitrary range) at each time period, using data from x months to the left and right of said time period given the win% in that time period, but I guess zulu is working on getting the data for that.
|
On October 21 2009 02:49 zulu_nation8 wrote:
motbob honestly i probably havent used a calculator in three years and Im also not an econ major so I understand why you wouldnt believe me but please, just use common sense, how can the standard deviation be 50% in this case? How is that possible? The SD is a mean.
On October 21 2009 01:29 Black Gun wrote:Show nested quote +On October 20 2009 13:26 motbob wrote:
OK I just found a much easier way to compile map matchup data! So when I get access to Stata, I'll have better data. I'll do this for all stats since March 1st, 2009.
Byzantium 3: 25-13
Byzantium 2: 30-11
Tears of the Moon: 1-0
New Autumn Wind: 3-1
Medusa: 34-23
Tau Cross: 7-7
Carthage 2: 2-4
Carthage: 0-1
Battle Royale: 4-5
Holy World: 4-3
Shades of Twilight: 1-3
Colosseum II: 2-4
Andromeda: 7-19 (?????)
Neo Harmony: 5-0
God's Garden: 56-44
Carthage 3: 1-0
Outsider: 41-27
Neo Medusa: 34-25
Return of the King: 47-22
Eye of the Storm: 1-1
El Niño: 1-1
Destination: 110-72 (this changed significantly since the time of the OP... EVER OSL prelims used it)
Tornado: 5-1
Outsider SE: 2-0
Moon Glaive: 2-3
Match Point: 3-4
Heartbreak Ridge: 90-64
Fighting Spirit: 6-3
Overall: 524-361, or 59.21% the variable we are discussing here is binary, hence the estimator of the mean is the proportion p = 524/(524+361) = 0.592. the sample size is large enough to use a normal approximation. if we assume a null-hypothesis of a balanced winrate of p0 = 50%, then in the corresponding test we need to use this p0 and not p in the formula for the standard deviation! the test statistic then is: Z = sqrt(n)*(p - p0)/sqrt[p0*(1-p0)] = sqrt(885)*(0.5921 - 0.5)/sqrt(0.5*(1-0.5)) = 5.479 -> highly significant. if we assume a null-hypothesis of p0 = 0.55, then we obtain a Z of 2.517 -> p-value of 0.0059, ie significant even on a confidence level of 99%. so the ZvP-winrate during that timeframe significantly exceeds 55%.
|
On October 21 2009 02:50 EtherealDeath wrote:
Should just get all the TLPD data and calcaute the probability of a P's win chance being 48-52% (or some other arbitrary range) at each time period, using data from x months to the left and right of said time period given the win% in that time period, but I guess zulu is working on getting the data for that.
Im gonna use # of games instead of time period, and the # of games would be a number roughly equivalent to the number of games in a season.
|
On October 21 2009 02:50 EtherealDeath wrote:
Should just get all the TLPD data and calcaute the probability of a P's win chance being 48-52% (or some other arbitrary range) at each time period, using data from x months to the left and right of said time period given the win% in that time period, but I guess zulu is working on getting the data for that.
if u get that data, could u plz send it to me aswell? id like to draw some graphs to see the development over time^^
|
motbob
United States12546 Posts
On October 21 2009 02:44 zulu_nation8 wrote:
let me explain that, your SD is how far your data deviates from the possible outcomes, out of win or not win, for a zerg progamer when he enters a 1v1 starcraft game, whereas the rest of your equation is calculating the win ratio of a certain number of games, and basically not binary data.
No it isn't, that's the standard error. The standard error is the SD divided by sqrt(n). That's what I've been trying to tell you.
EDIT: and the SD is not a mean, wtf.
|
On October 21 2009 02:54 Black Gun wrote:Show nested quote +On October 21 2009 02:50 EtherealDeath wrote:
Should just get all the TLPD data and calcaute the probability of a P's win chance being 48-52% (or some other arbitrary range) at each time period, using data from x months to the left and right of said time period given the win% in that time period, but I guess zulu is working on getting the data for that. if u get that data, could u plz send it to me aswell? id like to draw some graphs to see the development over time^^
Well my web programming is nonexistent, but I'd imagine that you would direct your code to http://www.teamliquid.net/tlpd/games/ and grab data from winner/loser, and then since each the data in each field happens to have race encoded, that would work. But yea, someone do it plz =)
|
|
|
|
|
i really wish i was better at math so i can explain what everyone already knows by common sense.
|
motbob
United States12546 Posts
On October 21 2009 03:09 zulu_nation8 wrote:
i really wish i was better at math so i can explain what everyone already knows by common sense.
I wish you were better at statistics so that you could realize that your efforts are misguided.
|
ok motbob you win, the standard deviation is clearly 50% from a mean of 55%.
|
This statistics talk is really over the top and silly. You only need to WATCH THE GAMES to get an idea of this topic. The point shouldn't be trying to prove things with pointless statistics but actual game analysis, we all know Zerg is doing better lately its plainly obvious but just point out the reasons why and what can change it.
|
|
|
|
|
|
EPT
