EPTWhich part would you want me to elaborate on?
Race distributions, win percentages and more in GM - Page 9
| Forum Index > SC2 General |
|
buldermar
Denmark102 Posts
Which part would you want me to elaborate on? | ||
|
buldermar
Denmark102 Posts
On April 26 2011 06:41 Etrnity wrote: I completely agree here with what Pelican has added to the discussion, and his given conclusion. I also want to note that what we don't see in the raw data is also some of the omitted variable bias that can play a role in the numbers, themselves. I'm not just speaking to values we cannot see, but also values that are rather hard to quantify. Such as that of differences in the goals of individual players on ladder. For instance, some will use ladder as a means of practice, while others will use it for experimentation. Unless you can justly quantify how this would affect percentages, or control for that in the data between the races, I don't see how you could logically come up with an argument. The data also does not, to what I have seen, take into account what the players are matched up with. Such that there is a possibility that zerg mainly face protoss instead of mirror or zvt, and thus the data is biased in that the win percentage may be speaking about the zvp match-up instead of Zerg on the whole. The data would also require trying to control for the differences in the ability of the individual. I would be so bold as to say that some players are simply better than others, in areas such as mechanics, game sense, and decision making. How can you control for the differences of the individual in trying to analyze the abilities of the races? In summation, the data is too clouded, or rather, not collected enough in order to properly account the statistical relevance. If you would be able to embark upon the statistics with a team of people, and try to control for variable bias, then you may have an argument. As it currently stands, there is no such possibility. I like the idea, but the execution is a little poor. I agree with this, but I still do not see how this contradicts my desire of finding explanations. Obviously, more thorough datacollection, organization and interpretation would be preferred in terms of meassuring a confidencelevel, but this still does not contradict discussing what could possibly help terran to a dominant position in the top of GM in the current ladderform. There is a correlation between the benefits (in terms of improvement of accuracy on confidencelevel) in spending additional time on the reseach part, and how this influences the discussion part. I sincerely believe that what I've shown is sufficient to be basis for a discussion (not to be confused with balancing the game or anything like that). I'll respect if you believe otherwise, just as I expect of you to respect my standpoint. | ||
|
Silfurstar
Switzerland263 Posts
On April 26 2011 18:47 buldermar wrote: Although it is unlikely, it could be that the matchup system works so that you're facing an even amount of players from each respective race undependant on how many players that you can be matched up against are playing these respective races. Because I cannot rule this out due to lack of insight in how the matchup system works, I cannot justify accounting for it. Most of my protoss friends history point to a huge tendency to get PvP these last few weeks... so no, it doesn't balance the matchup distribution for you unfortunately. I think many people's point in this thread is that you basically took numbers and made them say whatever you wanted. The only real conclusions you can make with the informations you have are just facts like : 38% of GM players are Protoss, and in the top 10, 7 out of 10 are Terrans. Your question already implied an unproved hypothesis "terran crush the top ladder". Which isn't very objective. There's unfortunately not much to discuss, at least not seriously. | ||
|
buldermar
Denmark102 Posts
On April 26 2011 07:05 MasterKush wrote: I know in the OP you only gave figures using grandmaster/top % of players, but I really feel like you cannot come to the kind of conclusion you have solely based on these statistics. Would it not be fair to suggest that there are in fact more Terran players total than any other race currently competing in the ladder? I think it's also reasonable to say that the more players playing 1 specific race *could* lead inevitably to that race creating more top level players in the long term. I don't know whether this is actually true, but I think if it were plausible, you should find out approx the ratio of players playing with each race. Just throwing it out there, there are so many variables that you need to think about if you want to be taken seriously on this matter. I've answered this multiple times now. On April 26 2011 02:00 buldermar wrote: My first hypothesis was that more people were playing terran in GM league. That would be a logical explanation, as there will naturally be a clear correlation between the amount of players playing each individual race and the percentages of each individual race in the top rankings. However, is this not the case. Here is the race distribution by league graph taken from sc2ranks.com: http://img687.imageshack.us/i/racedistributionbyleagu.png There currently is 1638 players listed in GM. The distribution is: 2.6% random 38% protoss 30.3% terran 29% zerg How does this not answer your question, or did you simply not bother reading the entire post before answering? | ||
|
MockHamill
Sweden1798 Posts
| ||
|
Mutaahh
Netherlands859 Posts
if 40% = terran 30% = zerg 30% = protoss its not so weird to see more terrans right? | ||
|
Pelican
United States45 Posts
On April 26 2011 19:42 buldermar wrote: I am not trying to analyze the current state of balance based off of the Blizzard ladder, merely trying to understand the top of GM is overpresenated by terrans. While what you're doing works for a single sample of data, it does not look into multiple areas and their interrelation. Yes, I performed a simple example of actual statistical analysis. The example I gave was looking at a single variable. There are literally dozens of tests that can be used to "look into multiple areas and their interrelation." You have to actually use these tests. You cannot make broad statements based on simply looking at numbers and calling it data analysis. Some examples of tests that could be used for this broad analysis: Linear Discriminant Function analysis Multivariate Analysis of Variance Principal Components Analysis You should probably do it on top 100 or top 200 of GM rather than specifically on NA GM. SC2ranks.com allows for showing this data listed the same way that NA GM is listed. See below. This method does not account for the likelyhood of having multiple samples show the same trend. It solely looks at any one group of players, some variables related to them (i.e. win%, points, etc), and if the trend you're looking at is inside or out of the 95% confidence interval. (I do not know the english words, so I can only hope my point is still clear) I only did it on the NA GM because it would take forever to import all of the GM data into excel. The statistical method is only limited by the data that is put into it -- if I ran data from every GM player through an Analysis of Variance, it would make the test more statistically robust, not less. It seems like you might be confused as to what and Analysis of Variance (ANOVA) actually is, or maybe there is just a language barrier. See this: ANOVA explanation Again, I performed a simple statistical analysis on a single variable. I wasn't necessarily trying to prove you wrong with one test, I was simply trying to illustrate that you can't draw conclusions from simply looking at numbers. That is not data analysis. There are ways to analyze the multivariate data that you want, and if you are so set on drawing conclusions from these data, then you should learn how to perform the proper analyses. The difference shows a less than 5% chance of it being due to variance/coincidence. Add to that the fact that you've only made this statistical analysis on one sample, and that this sample is NA rather than the actual top 200 of GM. I'd say that this supports what I reached with my approach more so than it conflicts it. You can't just do your 95% confidence interval analysis and then 'ignore' the fact that T-P is out of that interval. The difference is only significant in the Least Significant Differences (LSD) post-hoc test. The Tukey post-hoc test shows the difference as being not significant (or marginally significant, depending on what alpha-value thresholds you set for significance). LSD post-hoc tests are generally criticized for not controlling for Type I ('false positive') errors, while Tukey tests are quite robust, given that the sample distribution is normal. I only included the LSD test to emphasize the pattern in the data (e.g., differences between T-Z, P-Z, everything-R are extremely insignificant) -- it was probably a mistake to include it, but it's logical to me -- Tukey is an HSD test, and LSD offsets that, except that its inaccurate. Strictly speaking, there is probably no significant difference between the win% of T and P in the NA GM -- LSD is an inaccurate post-hoc test, and Tukey showed the difference being marginal at best (p=0.081). If you're up for it, I'd be interested in what a similar analysis would show of only top 100 GM. Here is the link; http://www.sc2ranks.com/stats/league/all/1/100 If you have a link to a free version of the program you're using, I'll do it myself. Ooookay. Here is an Analysis of Variance on the top 100 GM. The numbers have actually changed since yesterday -- as of right now it is: T: 37 P: 34 Z: 29 R: 1 This adds up to 101 because there is a point-tie. First, I had to exclude the 1 R because it had a group size of 1 -- this would make it impossible to do any post-hoc testing. I ran a quick chi-square test to see if there was any difference in race distribution: + Show Spoiler + ![[image loading]](http://i.imgur.com/GTohx.png) Next, I actually run the ANOVA. Here is the output for the descriptive statistics. Remember that these are basically unanalyzed numbers and can't tell you much. The confidence intervals are interesting, but they haven't been analyzed/compared yet. + Show Spoiler + ![[image loading]](http://i.imgur.com/uPIkB.png) The actual analysis of variance: + Show Spoiler + ![[image loading]](http://i.imgur.com/HX8XP.png) Post-hoc test output: Link for Big! Note that 1=P, 2=T, 3=Z This time, I used the Tukey HSD test and Scheffé's test, to eliminate confusion. Both of these tests are fairly robust, and are useful in different ways. Tukey's test is more valid if the interest is in pairwise comparisons, but Scheffé's is preferred when all contrasts may be of interest. Here, we see that there aren't any statistically significant differences between T-P win% (p=0.062, 0.056). The tests suggest that there may be a marginally significant difference, but that is really in the eye of the beholder -- this is why alpha values are set beforehand. Also, what analysis method would you recommend for showing the confidence interval of the amount of terran players in the respective intervals of 10, accumulative (top10, top20, top30 etc) in relation to the total amount of terran players in GM? Is there a way around doing each group seperately? The most straightforward method here is to do an Independent 2-sample t-Test or a Paired t-Test on each group and the whole. There are other ways, but they are more complicated than necessary. | ||
|
Etrnity
United States88 Posts
Firstly, I want to point out that I'm taking a mathematical approach to my question. This is not based on personal experinces but on observations and statistics. Pelican has pointed out that what you have done is not in-depth enough of statistics. I have pointed out that even using Pelican's methods are insufficient, considering variables unaccounted for. The only available source to do statistic on at this moment is www.sc2ranks.com. The site updates the ladder standings constantly. There are other websites that examine the base statistics between the races, individual match-ups, and player statistics. These are generally replay websites, and the data is unclear as to how accurate, or useful, they may be. I decided to base my statistics on the most skilled players as the gap between the level of play by these players and what can be described as optimal play is the smallest. You ignore the differences between regions, you are making an assumption that each player in each region is of equal skill, and you ignore the reasons for laddering at the highest level (not all of which is winning each game). My first hypothesis was that more people were playing terran in GM league. That would be a logical explanation, as there will naturally be a clear correlation between the amount of players playing each individual race and the percentages of each individual race in the top rankings. However, is this not the case. Here you correctly analyze the data. You don't necessarily need to run a regression to tell that there are more of one race than another, but if you wanted to, I supposed you could see if there was a significance....... Based on the above statistics, I feel safe to make the conclusion that terran is dominating the top of GM League in every way I could think of (if you have alternative angles to observe from, let me know). I want to point out that this is not a topic about balance/imbalance. I am looking at the statistics with objective eyes in hopes of finding an ALTERNATIVE answer to that. Why does terran crush the top ladder? This is the real question here. Pelican has taken data just in the same way you have, and run p-tests in order to determine if there is any statistical significance. Having found a p-value greater than .05, we cannot reject the null hypothesis that there is no difference in the means of the data. He has answered your question, given just as much data as you have used, so what more do you want? If you want more, then you need to provide significantly more data than what you have presented. If you cannot do this, then you must accept that what you have set out to find has been looked at, and has reached conclusion. I do not blame you for your confusion, as I assume you have not taken regression based statistics, but please end your silliness, it's annoying at this point. | ||
|
oxxo
988 Posts
| ||
|
Techno
1900 Posts
On April 26 2011 05:49 Pelican wrote: This... ...and this. Let me start by saying that I don't think it's very useful to try and statistically analyze the current state of balance based off of the Blizzard ladder. I just want to make a point that if you're going to attempt using statistics, you should actually use statistical analysis and not just look at numbers and try and determine if patterns exist -- this doesn't work and this is why the field of statistics exists. I'm not a statistician, just a lowly science grad student, but I did some really, really simple stat analysis on the North American Grandmaster league statistics (it would take a really long time to gather the data from every league into Excel). If you just look at the data (like the OP) did, it seems like there are some differences here: Win% by race: P: 59.0% (15,632) T: 61.4% (10,083) Z: 59.7% (11,163) R: 55.4% (1,333) But you can't really tell if there is or not without actually analyzing the data. So I ran a one-way ANOVA on the NA GM league, comparing win% between the races. All the data were pulled from SC2Ranks. The data weren't normal, so a square root transformation was applied [automatically, by the program] to meet the assumptions of normality for ANOVA. The assumption of homogeneity of variances was also met (p=0.430). Here is the descriptive statistics table generated: ![[image loading]](http://i.imgur.com/LkdRn.png) From just this table, you can see that the 95% CIs overlap between every race. Note that the mean values are the actual values, not the sqrt transformed values (not sure why the program does this). The ANOVA table: ![[image loading]](http://i.imgur.com/DvH3v.png) This shows that there is a statistically significant (p=0.043) difference overall between the group means, but it doesn't tell us where the differences are. This is where post-hoc tests are important, as they lets us look at comparisons between multiple groups: Link for Big! Here, we have two post-hoc test results. In this table 1=P, 2=T, 3=Z, 4=R. We can see that in the Tukey test, there is a marginally significant difference between T and P win%s, and the Least Significant Difference test shows that there is a significant difference between T and P. Essentially, what we glean from this entire test is that the only real differences in win% are between Terran and Protoss, and that difference is pretty small. To visualize the actual comparisons, here is a bar graph with ERROR BARS! ![[image loading]](http://i.imgur.com/LfocY.png) TL;DR: Analysis of variance shows that the only difference between NA GM win%s is in T and P -- and it's not very big. Don't make threads about taking 'mathematical' approaches or statistical comparisons if you're not going to actually use statistics. Looking at numbers =/= statistics. Quoting Pelican's demolition of this thread for those too lazy to scroll back. | ||
|
HiredGoonThug
United States72 Posts
| ||
|
zyce
United States649 Posts
http://twitpic.com/4sdkpv note: i was threatened in a pm for making this post and im sorry. | ||
|
AutomatonOmega
United States706 Posts
On April 26 2011 02:02 udgnim wrote: they are the most resilient race to random strategies / attack timings? As well as having excellent random strategies/cheeses/timings of their own. | ||
|
AutomatonOmega
United States706 Posts
| ||
|
Cloak
United States816 Posts
| ||
| ||
