|
another funny thing about those numbers,
http://www.sc2ranks.com/stats/league/all/1/all
zerg and terran behave inversely to one another when you go up and down the leagues, zerg has a higher percentile of players in the higher leagues while terran has a higher percentile in the lower leages
if OP is correct and the game is balanced (which is a ridiculous assumption as his numbers mean nothing other than the AMM is working) then one can only conclude that noobs pick terran and pros pick zerg :D:D
overall the balanced or not discussion is pretty dumb, there's a reason blizzard holds off with the first patch after release; any balance or imbalance cannot be detected this early in the game by skill levels between bronze to mid diamond, simply because neither side is playing their race to full potential, you can only hope to notice imbalance in high level tournament play and even then you have to account for a players subjective bias and just random strokes of chance.
tl:dr
OP means nothing
|
On August 17 2010 08:08 mahnini wrote:Show nested quote +On August 17 2010 08:04 Wr3k wrote:On August 17 2010 07:53 mahnini wrote:On August 17 2010 07:51 Wr3k wrote:On August 17 2010 07:48 ejac wrote:The problem the way you're calculating the imbalances is that you're assuming that races should have different win percentages. A 1000 point terran may only be as skilled as an 800 point zerg, but both may have 50% win records at their perspective levels. It's just the terran has a racial imbalance allowing him to play competitively at 1000 points. This graph: http://www.sc2ranks.com/stats/race/us/1shows that as points goes up, terran starts to dominate the ranks more and more, and zerg gets worse and worse. Yeah, this is much more scientific "proof" that Terran is in fact OP, and Z is the worst race. I mean cmon... look at it, you would have to be blind to not see a relationship: http://www.sc2ranks.com/stats/race/us/1 that's ignoring the fact that as skill level gets higher sample size becomes smaller and you have to compensate for a margin of error (calculate significance or something?). 70:30 with a sample size of 1000000 is vastly different from 70:30 with a sample size of 10. Yes, the sample size is very small, but there are only so many 800+ diamond players to get data from. SC2 needs to be balanced at the highest level. It's the only sample size we have. A small sample size with a analysis that actually makes sense is still infinitely better than one with a large sample size that is completely and utterly flawed. I know the sc2ranks numbers are unreliable due to their size, but there is still enough players above 600/700 to show that the difference in racial distribution is significant. All the OP has shown with his numbers is that the match making system is working properly. saying there are enough player to make the imbalance significant doesn't make it so there are calculations for this but i'm terrible and don't know how to do them. though the OP itself doesn't show us anything revelating it does give us a concrete basis off which we can make assumptions such as: blizzards matchmaking works properly therefore we can look towards racial distribution at certain levels to help gauge imbalance. if there are a proportional amount of zerg in at the top levels as in the general population it means the ladder perceives the skill levels of those zergs to be high which would not be the case if a certain matchup were extremely imbalanced.
I didn't say the imbalance was significant, I said there was a significant difference between the # of terrans and # of zergs in 600+ diamond.
I agree with the rest of your post.
|
i posted this in a different thread. and there was a thread that also talked about this topic specifically but i dont remember what it was.
actually, if you disregard random as a race, the number of zerg in the top 100 is proportional with the number of zergs in the general population ~21.75% vs 22% in the top 100, protoss is underrepresented at ~39.20% vs 32%, terran is overrepresented with 39.05% vs 46%.
i haven't taken stats in a while so i don't know how to calculate the margin of error or whatever but clearly zerg isn't underrepresented.
|
Can you obviously stretch the meaning of "scientific proof" when you don't even use a legitimate peer review process? Sounds like another amateur's(which doesn't measure knowledge fyi) opinion to me.
|
but dave[9] this IS the peer review process!
we are doing science! :D
|
Very well written and comprehensible.
Although I agree with a lot of people that say this data can't give us a conclusive answer about balance, I think people need to realize that it doesn't mean this analysis is useless.
This is a very good analysis of one part of the equation.
|
So, how is your thesis different for BW?
|
Lol this ought to be good. Brb reading.
|
On August 17 2010 08:08 Jameser wrote:another funny thing about those numbers, http://www.sc2ranks.com/stats/league/all/1/allzerg and terran behave inversely to one another when you go up and down the leagues, zerg has a higher percentile of players in the higher leagues while terran has a higher percentile in the lower leages if OP is correct and the game is balanced (which is a ridiculous assumption as his numbers mean nothing other than the AMM is working) then one can only conclude that noobs pick terran and pros pick zerg :D:D overall the balanced or not discussion is pretty dumb, there's a reason blizzard holds off with the first patch after release; any balance or imbalance cannot be detected this early in the game by skill levels between bronze to mid diamond, simply because neither side is playing their race to full potential, you can only hope to notice imbalance in high level tournament play and even then you have to account for a players subjective bias and just random strokes of chance. tl:dr OP means nothing
More pros still pick Terran over Zerg. All it means is that noobs don't pick zerg.
|
Looks like all the bronze Terran are getting 6 pool'ed and proxied by pylon lol. poor bronzies.
Evolving as a player is fun and rewarding. eg: figuring out you can kite a zealot by moving and stopping etc.
|
Get back to me when you have calculated the win percentage against respective race.
User was warned for this post
No need to warn, as this man speaks the truth.
|
its not only a of results and charts... zerg its a way difficult to play than other races and bad terran and protoss players are doing so well against very good zerg players that all are getting mad about it. Its an incredible effort for zerg players to maintain good positioning and win rates that i expected to see a zerg dominance when the game finally reach a plateau about balance (two expansions to come and many patches in the future will change the game for create the ultimate RTS game that all are waiting for)
|
I think the only people who know the true state of balance from a statistics standpoint is Blizzard. In order to make an accurate analysis you would need to know info only Blizzard knows (all this stuff I hear about hidden rating...).
I'm sure everyone knows that you don't always get paired with people from your own league, and often times get put into a game higher league players. I'm just going to make up an arbitrary "true" rating for this, let's say a Terran player has a rating of 1000. You could compare the win % of that terran player when paired against someone who is +- 10 of his rank, so 990 to 1010. Then you could look to the win% against people that have a rating 200 or more. An imbalance would be obvious if the Terran had a higher win rate against higher ranked opponents than any of the other races.
Of course this is just 100% speculation, I don't know how the matchmaking system works.
|
On August 17 2010 08:27 No_eL wrote:
its not only a of results and charts... zerg its a way difficult to play than other races and bad terran and protoss players are doing so well against very good zerg players that all are getting mad about it. Its an incredible effort for zerg players to maintain good positioning and win rates that i expected to see a zerg dominance when the game finally reach a plateau about balance (two expansions to come and many patches in the future will change the game for create the ultimate RTS game that all are waiting for)
while your input may be true, frankly i think that is a cop out (and off topic!). it is near impossible to balance the difficulty of races while maintaining the diversity that starcraft does. terran was widely accepted as the most mechanically heavy race at pretty much all levels in sc1 and yet there were no large complaints about imbalance there.
|
On August 17 2010 07:18 StarcraftGuy4U wrote:
None of these stats are worthwhile because the matchmaking system does not assign people like they would in a blind study, instead it is actively adjusting the matches so that every player reaches 50%. The numbers you are pulling are worthless for this reason.
+1 this is true so statistics wont show any correct imbalances so sadly this thread was mostly pointless good effort though :D
|
On August 17 2010 08:08 Jameser wrote:another funny thing about those numbers, http://www.sc2ranks.com/stats/league/all/1/allzerg and terran behave inversely to one another when you go up and down the leagues, zerg has a higher percentile of players in the higher leagues while terran has a higher percentile in the lower leages if OP is correct and the game is balanced (which is a ridiculous assumption as his numbers mean nothing other than the AMM is working) then one can only conclude that noobs pick terran and pros pick zerg :D:D overall the balanced or not discussion is pretty dumb, there's a reason blizzard holds off with the first patch after release; any balance or imbalance cannot be detected this early in the game by skill levels between bronze to mid diamond, simply because neither side is playing their race to full potential, you can only hope to notice imbalance in high level tournament play and even then you have to account for a players subjective bias and just random strokes of chance. tl:dr OP means nothing
and thats only a part of it.
there another few billion assumptions such as
higher points = more skilled
that everyone does the same build and every game is carried out similarly
that is your treating a all-in proxy reaper 800 point terran the same as a siege cliff abuse 800 point terran. Heck they could be the same guy doing different builds at different times.
the imbalance is dependant on what the people do not win/loss.
Check out idras win loss ratio, he wins almost every game. Yet he is complaining about terran simply because terran does (in his eyes) hold an advantage. This has NOTHING to do with win and losses.
|
This scientific proof is incorrect for the reasons stated several times above. The most accurate way to find racial imbalance is:
What percentage of all Zerg players are in diamond?
What percentage of all Zergs are 600 ELO plus diamond?
The higher the number the more accurate it will be if you have a large enough sample size.
|
On August 17 2010 08:04 stochastic wrote:
OP, I really like what you’ve attempted here (as a probability and statistics major!).
Perhaps a more meaningful test would be a Chi-Square test of independence on the number of diamond level players of each race:
Find percentage of total players of each race (at ALL levels). Using this, find the expected number of diamond ranked players of each race under the assumption that league placement is independent of race. Use the appropriate Chi-Square test statistic to determine the likelihood that the observed proportion of diamond players of each race fits what would be expected under independence.
I think this will help alleviate bias caused by the matchmaking service, as all we care about is that it places people into diamond correctly.
Also, we will have accounted for the fact that more terran players on the whole means more terran players in diamond, if the skill level of players across races is equal.
A problem is that the skill level of players across races may not be equal. But I have to think that players of greater skill tend to choose the “stronger” race, so the relative strength of that race would be reflected in our results.
I have a feeling, given the large sample size, that I'd get a similar result -- there would be a strongly significant but small heterogeneity in the distribution of races as you move up a ladder.
@ Warent, to do this test I calculated the average win rate within a league, then calculated the proportion of games played in each league by each race. I multiplied the number of games played by each race by the average win rate to calculate expected win rates for each race if wins were homogenously distributed (i.e. equally likely to occur for each race within a league).
Doing the statistics is simple; you just subtract the observed result (number of won games for a given race) from the expected result, square that quantity, and divide it by the expected result. Then you sum the results for each race. This gives you a standardized score for the degree of difference between the observed and expected results that's greater when the difference is greater, and that follows a chi-square distribution. You then look up the value you get on a chi-square distribution with the appropriate degrees of freedom to get your p-value.
|
Sorry to double post, but I wanted this to stand alone:
Judging by the quality of the discussion and the number of alternate hypotheses and research suggestions I've gotten, I must've done good science after all!
If I had to write a new conclusion based on the discussion we've had here, I'd say this:
Given the data I had access to, I did the only analysis I think that was available to me. What it showed was that there is essentially no difference in the performance of the races viewed on a per-game (not per-player) basis, at least within the top four leagues. What this suggests is that regardless of race, players in each league are having a similar experience. Within a given league, no particular race is more likely to win a given game than one of the other races. Whether it's because of inherently strong balance or good matchmaking, the system is adept at providing a balanced experience for players at a given level regardless of race.
Another worthwhile conclusion, in my opinion, is that for an individual person who feels they are the victim of race imbalances, what they are actually experiencing is just trouble with a particular matchup. These data offer what in my opinion is pretty strong evidence that race alone is not a strong determinant of success at a given player level. You can't randomly pick a game played by a platinum player and predict the outcome based on the race of that player.
I think it's really important that people recognize that these data were gathered based on games played, not players playing the games. There are some important implications of that and I think some of the criticisms of the 'model' assumptions aren't as relevant when viewed in a per-game context.
In order to really answer a question about balance I acknowledge that a more rigorous analysis is needed. I'd do it, except I've already used up all the data I have access to. If anyone wants to send me data I'll happily test alternative hypotheses. I also want everyone to know that I'm not taking this too seriously, and you shouldn't either! My tongue was in my cheek a bit when I wrote this, but I acknowledge that that doesn't communicate on the internet.
|
You do realize that all statistics relating win percentage are irrelevant because of the matchmaking scheme? Matchmaking ensures that all players have roughly a 50% win percentage. The only way to test imbalance is to have a large number of games not using the matchmaking system.
|
|
|
|
|
|
EPT
