EPTrace v race statistics based on 551 "top" replays - Page 4
| Forum Index > SC2 General |
|
QueueQueue
Canada1000 Posts
| ||
|
Deadlyfish
Denmark1980 Posts
But whenever we see data like this, which actually says that ZvP is imbalanced, the same people call this data bad, and useless. And it's the exact same people who said that the "no zergs in top 20" statistic was super useful and showed a clear imbalance. It's like the roles have switched places. Not saying that this information is or isnt usefull, it's just a funny observation  | ||
|
QueueQueue
Canada1000 Posts
On September 22 2010 03:00 Deadlyfish wrote: I find it kinda silly that whenever we see statistics that TvZ is imbalanced, alot of people say that it is correct and that it proves that there is an imbalance. But whenever we see data like this, which actually says that ZvP is imbalanced, the same people call this data bad, and useless. And it's the exact same people who said that the "no zergs in top 20" statistic was super useful and showed a clear imbalance. It's like the roles have switched places. Not saying that this information is or isnt usefull, it's just a funny observation Yeah, honestly a lot of Z players are more afraid of the ZvP MU than the ZvT as of late. People are told to complain about Terran because it's "the cool thing to do" that they miss other fundamental issues. | ||
|
Sleight
2471 Posts
On September 21 2010 23:01 Sleight wrote: Hey y'all, Before this debate turns into some kind of statistical pissing match, I thought I'd link a useful post I made so we can discuss this properly: http://www.teamliquid.net/forum/viewmessage.php?topic_id=153500 I would appreciate seeing actually statistical tests for significance on any of these values. My intuition is that most of these are statistically significant, but I can't be sure without someone actually doing the math. How does this data hold up to Chi-squared analysis? I suspect that it shows almost perfect balance of the 3 race's overall win percentages. Read My Damn Statistics Thread. It's linked above. Stop bickering about useless things. I am quoting myself so we can move along. The sampling size MAY be too small. How can we find out? Run a series of parametric statistical tests and the Chi-squared analysis on average win rates. That will give us a great idea if the sample is begin enough. In any case, here's the facts: A) This data obviously cannot be used to definitively generate a conclusion to a different population. It is not a random sampling. B) This data, while non random and thus not directly applicable to other groups, still needs to be test for significance because you need to prove that it still isn't due to random chance given the population size. C) If it isn't due to random chance, we can discuss whether or not this result may warrant further examinations in other paradigms. This can be evidence to try and examine a different population, like all of Diamond by random sampling, and see if this trend continues. Stop bitching about statistics when most of you are saying irrelevant things. Look at the data for itself and conclude something about this sample, then redo the study under different conditions and see if it holds. | ||
|
Serendipicus
United States90 Posts
| ||
|
FabledIntegral
United States9232 Posts
| ||
|
Serendipicus
United States90 Posts
| ||
|
FabledIntegral
United States9232 Posts
On September 22 2010 03:26 Serendipicus wrote: Also if zerg only won about 40% of their matches, they wouldn't even be in diamond league. You obviously don't pay attention to the forums and don't know how the matchmaking system is intended to work.  | ||
|
Black Gun
Germany4482 Posts
On September 22 2010 03:16 FabledIntegral wrote: And these are NOT random, agreed, but we don't want a random sample, we want only top players. with "they are not random samples" ppl usually point out to the fact that the players tend to only upload oustanding games or wins of themselves. additionally, many observations (read replays) of this sample belong to the same 2 players playing against each other, so the outcome of these observations depends on each other in the sense of mindgames and psychological effects in a BoX game. also sometimes the style of a particular guy just doesnt fit the style of some other guy. if dimaga is at (for example, made up) 11-3 against demuslim and we only got 100 observations for tvz, then the fact that dimaga seems to dominate demuslim might have an impact on our impression of the general tvz matchup. therefore its not only about the number of single observations in our sample, its also about the variety of features which underly these observations. (for example 40 replays from the ro8 and higher of the iem, but these 40 replays were created by the games between only 8 different players. then there is dependency and less variation in our sample than the nominal sample size of 40 would suggest...) | ||
|
Sideburn
United States442 Posts
On September 22 2010 02:19 travis wrote: THIS (and honestly the sample is small, too. but that's not the primary problem) Really, can you explain why it is too small? Too small for what tests, presuming it was random data? | ||
|
Deleted User 3420
24492 Posts
with a sample size of only say, 200 replays of a matchup all it would take is 10 games that skew from the norm(very easily accomplished through variance), to take odds from being 55-45 in one races favor, to being 45-55 now in the other race's favor. but in reality, with a sample of only 200 games, the variance could be WAY BIGGER than that. could be. of course. maybe it's spot on though. but who knows... that's the point of having bigger samples. | ||
|
Mastermind
Canada7096 Posts
| ||
|
QuanticHawk
United States32028 Posts
On September 22 2010 03:00 Deadlyfish wrote: I find it kinda silly that whenever we see statistics that TvZ is imbalanced, alot of people say that it is correct and that it proves that there is an imbalance. But whenever we see data like this, which actually says that ZvP is imbalanced, the same people call this data bad, and useless. And it's the exact same people who said that the "no zergs in top 20" statistic was super useful and showed a clear imbalance. It's like the roles have switched places. Not saying that this information is or isnt usefull, it's just a funny observation Most people can't be bothered to change their views, no matter how many facts you throw at them. Per the site: "The definition of a "top" player is up to each individual site to decide. Use at your own risk. " That doesn't say anything. What's the benchmark YOU used, op?? obviously this means dick with the current sample size and he knows that, but depending on the criteria used, this could be interesting to look at next month. duh, skimmed it in the shuffle: On September 21 2010 23:00 bingobango wrote: I didn't pick replays by hand. These are replays aggregated from 5 sites over the past several weeks that I chose because they had 1) good geographic coverage 2) frequent updates 3) top players. You can see the sites I used here: www.replayspider.com/about/ Being a bit of a replay junky, I'd say the selection of replays from these 5 sites has really good coverage of the entire "top player" replay scene. If there's replays missing or a site that has replays that I am missing, I'd love to know about it. I agree with the first part, not so sure about the second part. The selection bias + small sample size makes it a bit squirrely, but it's better than nothing. "top" player in this case means whatever the maintainers of the site in question mean by "top" when they upload their replays. You can look through the replays yourself and see what qualifies. I've put the rankings (and sc2rank regional ranks) by each replay. Kind of skews the numbers. | ||
|
Sairon
47 Posts
On September 22 2010 03:14 Serendipicus wrote: Prepatch stats for all diamond players, showing all races are within 1% win ratio. http://www.sc2ranks.com/stats/league/all/1/all/ This is not the way to interpret that data. The win ratio will be rather constant as that's the whole point of the ladder system, the ladder system doesn't rank depending on race. One has to look at the race distribution across tiers, but interpreting that data is very hard as you must make certain assumptions, like for example that the distribution of good players for every race is equal. | ||
|
Serendipicus
United States90 Posts
On September 22 2010 04:46 Sairon wrote: This is not the way to interpret that data. The win ratio will be rather constant as that's the whole point of the ladder system, the ladder system doesn't rank depending on race. One has to look at the race distribution across tiers, but interpreting that data is very hard as you must make certain assumptions, like for example that the distribution of good players for every race is equal. The page on the site does all that you suggested. | ||
|
Black Gun
Germany4482 Posts
On September 22 2010 04:24 travis wrote: too small because in any game where luck is a contibuting factor, the smaller the sample size the greater the chances are you will experience variance induced by that luck factor with a sample size of only say, 200 replays in a game all it would take is 10 games that skew from the norm(very easily accomplished through variance), to take odds from being 55-45 in one races favor, to being 45-55 now in the other race's favor. . this is exactly what significance tests are testing. oO folks, plz keep in mind that first of all, statistical significance doesnt equal relevance, and secondly that significance does depend on the sample size. for example rolling a dice: even a rigged dice that gives 6 every single time cant be detected as non-regular by statistical tests if all u have is 3 rolls (which ofc turned to 3 sixes...) as a general guideline, the smaller the true statistical anomaly, the higher the sample size required to detect this anomaly. obviously its gonna be hard to reliably detect deviations in the 1-5% range if the sample size is barely above 100.... | ||
|
FabledIntegral
United States9232 Posts
On September 22 2010 04:24 travis wrote: too small because in any game where luck is a contibuting factor, the smaller the sample size the greater the chances are you will experience variance induced by that luck factor with a sample size of only say, 200 replays of a matchup all it would take is 10 games that skew from the norm(very easily accomplished through variance), to take odds from being 55-45 in one races favor, to being 45-55 now in the other race's favor. but in reality, with a sample of only 200 games, the variance could be WAY BIGGER than that. could be. of course. maybe it's spot on though. but who knows... that's the point of having bigger samples. Uh that's exactly what tests do. They see if the data is simply too far skewed for it to be random chance, or luck. How are you using that as an argument.... And if you were just talking about a normal, random sample of 200 replays, it'd be a rather large sample size, wouldn't it? | ||
|
Fitzhunt1
United States169 Posts
| ||
| ||
