|
On September 05 2010 04:40 Scarmath wrote:I've uploaded a version of the spreadsheet I've been working on here: http://www.mediafire.com/?uta56tbvr5w8ectThe easiest way to play with it is by replacing the Raw table with other information copy and pasted from SC2Ranks.com. (Copy and paste it from Firefox. Chrome doesn't copy the Alt-test I use to identify race, and I haven't tested it in Internet Explorer). The rest of the sheets should update automatically. Other things may be messed with, but may require more extensive fiddling to work. Still working on this a few hours at a time.
ty! I was looking for this :D
|
Ok guys,CLEARLY random needs a buff.
The reason zerg is low and terran is high is because there are 3x more terran. Is random the weakest choice because its lowest in the %? No.
Also, Protoss is favored in tournys now.
|
On September 06 2010 05:57 ibreakurface wrote:
The reason zerg is low and terran is high is because there are 3x more terran. Is random the weakest choice because its lowest in the %? No.
If you read the thread, you'll see that a lot of the posts are about how the Terran point distribution differs from what you'd expect based on the number of players per race.
|
Match up win percentages would be a lot more helpful to be honest.
I am not going to get into my thoughts on balance or lack thereof, but these numbers indicate race popularity at various levels. Rule number one of trying to find a causing agent of something is this 'Correlation does not mean causation'.
|
Match up win percentages would be a lot more helpful to be honest. And are not here right now, there are some projects ongoing to see the mu win percentages but it will take some time.
|
I updated SC2Ranks to do the column style mappings instead of the stacked percentages: http://sc2ranks.com/stats/race/all/1
I'm going to try and get some data available in CVS format too so you can do the stats without having to go through some elaborate process to get the data off of the site.
|
On September 05 2010 09:38 rackdude wrote:Show nested quote +On September 05 2010 08:03 Shorack wrote:On September 05 2010 04:52 rackdude wrote:On September 04 2010 23:24 Shorack wrote:
Since you know the entire population, there is no need at all for statistics of significance.
. Absolutely not true. What could be happening in the population could be random chance. If you created a population randomly with balanced races, you won't get 33% 33% 33% (forget random to make it easier), you get something slightly off. Statistical significance tells you "if the races were balanced, there would be a P percent chance of this happening". When you get a number like .01%, you go "wow, there is almost no chance this randomly occurred". However, if you look at the data, you can make an inference from something that is random. For example, if you flipped a coin 3 times, and you saw all heads, if you looked at the graph you'd go "wow, this is definitely heads biased." Statistics would tell you though "hey dude, chill. There is a 12.5% chance of that happening randomly, so I wouldn't jump to conclusions just yet". That is why we are using statistics. You can't create random populations. The population needs to confirm to the research question. You can create random samples though. As your post is now, i disagree (assuming we can achieve perfect balance in the broad sense (appeal), which is ofc not possible, so just as a thought-experiment.) Replace population with sample in your post and i'll completely agree. You are right, but it actually depends completely upon where your model starts. For most models, experiments, scientific papers, etc, you are completely correct. The population is what is and the sample is what is measured. But that's because the ideas dealing with "random populations" are already dealt with in the mathematics. An example is like this. Participants enter a room where there is Card A and Card B. Assume there is no preference for either card. Participants pick a card and are now designated as group A or B. From this you create a field of theoretically possible populations from the different combinations of card picking that is possible. From this theoretical model, you can ask the question, "if I were to randomly pick a population, what is the chance I pick one that matches the population that I measured?". This is what I mean by "create a random population", it's like theoretically picking a card from your hand of possibilities. I probably should have said "take an arbitrary p element of the set of possible populations", and I probably shouldn't have said you won't get 33% 33% 33% because there could exist at least one population with that distribution. But I think you get the point. Good call because you cannot take a random population in any empirical science because the population is defined as what exists. But I was speaking from a mathematical standpoint that wasn't measuring what exists, but rather the probability of such a population existing given the model we have created (which is what the simplified formulas in non-upper division statistics classes give you). I guess it's a slip we make these days since with computers we actually do "create" random populations for models, though we should be saying we are taking a possible random population.
First of all, interesting post.
Second, i'll try to implement my remarks to scarmath in this same post.
Generating random populations does indeed happen. (i'm mainly thinking of the bootstrapping procedure for predictive modeling, e.g. churn models)
But there, the point is that we're interested in an unknown future population and we want to make sure the model will be robust enough for that.
Here, we're interested in the current population and in our case, it's fully known. The goal is not prediction.
Then there is the use of Scarmath's bins.
Using those bins, i see the point of chi-square, since you are comparing two different populations. (even if the distribtuion of races is the same, they're still two different populations by their definition)
This may sound like semantics, but as i understood it earlier, i believed you were putting forward a certain race distribution and then used the chi-square test to see if the diamond 600+ (or whatever+, i'm not arguing here about exact numbers, but about the method) is a different population or not. (in that case, if they weren't, that would be arguing that the actual population could very well be that proposed population, which would be nonsense.)
Just to indicate that last point, the correct formula for the standarderror in the binomial case would be root(p*(p-1)/n)*root((N-n)/(N-1)) With N=population size, n=sample size.
Since the 'sample' is the actual population, N=n and so the standarderror becomes 0.
I still have some doubts about the bin approach, but since i can't base them for myself on some statistical foundation and i don't want to be irrational and obstructing at the same time, so keeping Wittgenstein's famous saying in mind (Wovon man nicht sprechen kann, darüber muss man schweigen), i won't try getting in on that.
|
Is anyone taking into account how much each race plays? If for whatever reason people who play terran tend to play more, then it would be inflated as well. What are the number of games per race at these levels?
If you see higher numbers of terran games at that level, then it makes perfect sense that there would be more. If you see terran with about the same or less games played than the others, then it might suggest an imbalance at the higher levels.
Also, it will be interesting to see what will happen come 1.1 when tanks get reduced a bit. I'm a terran player and IM sick of seeing tanks :-p
|
It's true there's a lot of problems with this model of race selection. Nonetheless this is very cool, and I hope for Orb to come in here and declare it mathematically proven that terran is imba.
|
Correct me if I'm wrong, but let's say theres a difference in the amount of players who play each race (as it is right now)... just for example: 40% Terran, 40% Protoss and 20% Zerg. Now, if the game were perfectly balanced and every player played a nearly perfect (humanly possible) game, the top 10/top100/top1000 should also have about 40% Terran, 40% Protoss and 20% Zerg, right? So any lower/higher number should point to imbalances (at least in the metagame if not in the game itself), is that correct?
edit: and what kind of effect on the top x players distribution would it have if only one of the 3 interracial matchups was imbalanced and the others were perfectly balanced? lets say Protoss vs Terran would be 90% in favour of Terran and ZvT as well as ZvP was 50/50 either way. How would that change the distribution in an AMM ladder which (theoretically) almost perfectly assigns players according to their internal rating?
|
Just a question, and I'm sorry if I missed someone else asking, but have you guys doing statistical analysis of this data tried simply disregarding the random players? I mean, as far as I could tell from your different tests and such(I'm not particularly familiar with them) the random players only really seemed to make things a lot more complicated.
On the basis that a significant amount of random players(more than maybe 2-3) would almost certainly play as 1/3 terran, 1/3 zerg, 1/3 protoss wouldn't simply ignoring the random players completely make your tests much more revealing? You wouldn't get the weird dips because a few random players were clustered around a certain rating.
I know disregarding data is a big nono in general, but what purpose does examining random players really serve? They play random, after all?
|
heishe, the problem is that there are no samples of games that have been played perfectly, meaning that there will always be a difference from the 'ideal' stats and what really shows.
Furthermore, as I said before, you should look at the number of games each of these players plays.
Top points are (i'll go top 20):
01 - 1823 - Toss
02 - 1780 - Zerg
03 - 1750 - Toss
04 - 1749 - Terran
05 - 1706 - Toss
06 - 1684 - Terran
07 - 1677 - Terran
08 - 1666 - Toss
09 - 1665 - Toss
10 - 1660 - Terran
11 - 1654 - Random
12 - 1652 - Zerg
13 - 1652 - Toss
14 - 1651 - Toss
15 - 1646 - Toss
16 - 1644 - Terran
17 - 1644 - Terran
18 - 1633 - Terran
19 - 1633 - Terran
20 - 1628 - Terran
If you look at that, almost all the top 20 toss are ABOVE terran. The only ones not really represented about equally are zerg, which everyone agrees has problems (I think its because they lost things like the "dark swarm" which shields zerg's 'mostly' melee / close range units until they could get to a good fighting position. I think if they brought that back it would really balance thigns out.
But you can't just look at the points. you have to look at the games played too, and the win rations of them. If #1 played 200 more games than #5, then it makes sense that his points are higher.
If you take the points + ratio, pretty much the top 10 are 2 toss and the rest terran.
If you take just ratio, then there are 2 terran in the top 10 and the rest are zerg and toss.
Another thing you have to look at is what type of match up it is. were they all evenly matched? was one favored over the other? If you lose but you're not favored, then even if you lose 50 games, you're points will not go down as fast as if you lose 10 where you are favored.
This is why there are so many statistics in the world, we've been analyzing them for years, yet nobody can ever seem to predict how they will end up. It's not really valid to take out a single stat and determine if a race is op, because you have to have a global view of the overall picture. It takes more than one chart.
If you look at games played by the top 10 (based off points) (i dont feel like typing top 20 numbers) it is:
01 - 425 - Toss
02 - 819 - Zerg
03 - 1077 - Toss
04 - 499 - Terran
05 - 286 - Toss
06 - 274 - Terran
07 - 224 - Terran
08 - 594 - Toss
09 - 546 - Toss
10 - 196 - Terran
To me the first noticeable number is the zerg. He's number 2, but he had to play a lot of games to get there. Also, #3 toss is the same way. The lowest average games played while still being in the top 10 is Terran with an average of about 298 games. then the poor lonely zerg with more than 800, and then toss with an average of 585 games.
If you look at the games played, with the ratio, with the points together, i think it suggests that the Terran ARE a little better represented than others. They average a full 250-300 games less than toss, and about 500 games less than zerg, yet are remaining decent contenders with them (again looking at the top 10 because i didnt want to go down to 20, although it looks like in general 10-20 terran seem to have between 200-500 games, so it might raise the terran average to mid 300's, while toss might go down a bit with the remaining toss in the top 20.)
Once again, I think if we saw a comeback of the "swarm", the zerg wins would dramatically increase. Honestly, it seems like toss and terran are ALMOST even, maybe with the smallist imba in favor of terran. Zerg have a veerrry poor showing.
Once again that is to be a little expected considering how few people play zerg v. terran and toss, but Even with that There should be more of a zerg showing than only 2 in the top 20.
Another thing to take into consideration is that there are more toss in diamond than terran, and fewer zerg. That puts a little extra weight on the terran showing in the top 20, a little less on the toss showing, and much more weight on zerg showings. I think coming up with some sort of weighted system that takes all of this into account would be the best mechanism to actually determine how each race should be performing, based on games played (which should also heavily weight the results considering more games typically = higher points), win/loss ratios, etc...
sorry if i rambled.... there was a lot i wanted to get out there
|
|
|
|
|
|
EPT
