|
As others have stated, the OP is based on a flawed methodology. If trying to use stats to figure out which race is overpowered, 4 items need to be controlled:
1. Homogeneous skill in race choice (maybe old BW semi-pro's just gravitate towards Terran in sc2)
2. The matchmaking system instead of random opponents
3. Player MU difference (one player may, in the long run, win 60% pvt, another lose 60% pvt)
4. Player MU skill changes over time (maybe a day9 video will change pvt win stats by several bps in a single week)
To control all 4 of these I suggest mining for at least 1,000 players that:
1. Have players over 200 games
2. Played at least 30 games in the last 72 hours
3. Are in the diamond league
From this list, throw out all games involving a random player (less consistent MU performance), everything older than most recent 30 games, and and calculate the group's median win ratio using the most recent 30 games. Keep the 100 players of each race with win ratios closest to the median win ratio and throw out the other 700 players games. For instance if the 1,000 players have win ratios ranging from 35% to 90% (in most recent 30 games), with median of 55%, then pick the 100 zerg, protoss, and terran players who are closest to 55%. From the remaining 300*30 games, a simple win/loss record for each MU will be about the best possible indication of imbalance I believe data mining can come up with (short of using the same methodology with more games or tweaked ratios).
|
I think a race being OP isn't necessarily based on how many games they win or their win percentage, but more importantly HOW they win those games. Winning in itself is only the beginning marker in determining the balances of races. However, you can't prove imba based solely on win percentages: you don't have all the facts.
We need to take a deeper look into the game itself. If a Terran player beats a Protoss player 5 out of 5 times, does that mean that Terran is imba? No, it does not. But if in all of those games, the Terran player produces a build that the Protoss player CANNOT beat, no matter how good the micro or macro is, then yes, cry imba. It comes down to how you lose the game, in my opinion.
I'm a Protoss player, have been since BW. 600+ Diamond player but I struggle with Terran (as do most non-Terrans, it seems like). I don't lose games because I can't macro or I can't micro - no, I tend to lose most of my games to Terran because of cheap harass, EMPs, or tanks AI. How come Terran gets to have 2.5 units specifically made for harassing (reaper, banshee, helion - which counts as .5 because it can be used normally as well) when Protoss only gets 1, and it's Tier 3! Or how is it fair that a couple EMPs can cut the overall HP of my army in half in the first 2 seconds of a battle? Or how can seige tanks be so damn smart that they can strategically position their shots to maximize splash damage in my army? My Staulkers don't target-fire on their own.
Imbalances come down to how the game is played, not the end result. I don't cry imba when an MMM ball beats me fair and square, or when a nicely timed Thor/Tank/Helion push catches me off-guard. It's the little things that make Terran an imbalanced race. PvZ is fine just the way it is.
If EMP didn't have aoe, it would come down to microing Temps vs. Ghosts, feedback vs EMP, and that's a fight in which the better player (regardless of imba) would win. If tanks were as dumb as every other unit, it would come down to skill for them to do massive amounts of damage, not imba.
Hopefully, this doesn't seem like a whiny post, because I've taken a lot of time and thought into how I feel the game is going so far. I think the most rational of us can agree there is a problem with Terran, because there hasn't been this much outcry against any other race. Something needs to be done, whatever that may be.
|
Scientific proof that SC2 is imbalanced:
It was released less than a month ago.
Give it time people.
|
The game is imbalanced because of terran opening options, not because a bundle of useless statistics that only prove Blizzard's matchmaking system works to keep all players around the same W/L ratio's............... I am sure I am the hundredth person to mention that.
I feel bad the OP tried to be helpful with his stats and now realizes he wasted his time.
|
Sanya12364 Posts
Argh.... understanding of statistics is really really bad. Really really bad. Even academicians are terrible at understanding statistics.
You can't prove anything with statistics. A proof requires logic and proceeds by induction or deduction. All statistics can do is provide evidence that it's highly likely or unlikely to be imbalanced. It's highly circumstantial. Even then you need to clearly state out ALL your assumptions and if any assumptions fail to be tested then, the evidence will be flawed.
|
The overall winrate is useless, you need win rate from each matchup.
|
On August 17 2010 07:18 nam nam wrote:
Get back to me when you have calculated the win percentage against respective race.
User was warned for this post
He did. an average win rate in any division would assume playing each of the other races an equal amount, what that means is either zerg wins against protoss way more then they should or terran is losing equally against all races.
|
On August 19 2010 04:59 texmix wrote:
As others have stated, the OP is based on a flawed methodology. If trying to use stats to figure out which race is overpowered, 4 items need to be controlled:
1. Homogeneous skill in race choice (maybe old BW semi-pro's just gravitate towards Terran in sc2)
2. The matchmaking system instead of random opponents
3. Player MU difference (one player may, in the long run, win 60% pvt, another lose 60% pvt)
4. Player MU skill changes over time (maybe a day9 video will change pvt win stats by several bps in a single week)
To control all 4 of these I suggest mining for at least 1,000 players that:
1. Have players over 200 games
2. Played at least 30 games in the last 72 hours
3. Are in the diamond league
From this list, throw out all games involving a random player (less consistent MU performance), everything older than most recent 30 games, and and calculate the group's median win ratio using the most recent 30 games. Keep the 100 players of each race with win ratios closest to the median win ratio and throw out the other 700 players games. For instance if the 1,000 players have win ratios ranging from 35% to 90% (in most recent 30 games), with median of 55%, then pick the 100 zerg, protoss, and terran players who are closest to 55%. From the remaining 300*30 games, a simple win/loss record for each MU will be about the best possible indication of imbalance I believe data mining can come up with (short of using the same methodology with more games or tweaked ratios).
I wanted to say this, but feared the backlash of "NO We has psience we is wright". The methodology of the observational study is flawed in a few ways. For one, I don't think you are considering any confounding variables such as how the ranking system comes into effect. If one race is overpowered then it's simple to assume that it will be overrepresented in relation to its total population within the top of diamond rank only. But this could also be confounded by how people think Terran is the strongest race, so the more serious players switch over to that race thinking this is true. In addition, the sample sizes here are very small.
I would like to hear what a statistician, or Blizzard statistician has to say about the data.
edit: Oh I see, the OP understands that the matchmaking systems kind of voids his analysis. I'm sorry if I sounded harsh. Great effort put into this.
|
Sanya12364 Posts
On August 19 2010 06:13 Parodoxx wrote:
He did. an average win rate in any division would assume playing each of the other races an equal amount, what that means is either zerg wins against protoss way more then they should or terran is losing equally against all races.
This is the first of many untested assumptions. There are about 4 or 5 other assumptions being made.
On August 19 2010 06:18 Hidden_MotiveS wrote:
In addition, the sample sizes here are very small.
Sample size is large enough - provided we are looking for something that is really imba (+/-5%) rather than just a couple percentage points.
|
Dude watch the IEM tournament replays.. yeesh!
|
hi, given the number of games you've studied, are the differences in % actually significant? are they more than any statistical errors, or other errors you might be able to think of? If you split the sample in two (let's say by date) do you see any difference in results? do you build that difference into the final error?
ffs, if you quote differences in percentage, it's time you say whether it actually means anything, or whether it's chance, or a static or dynamic situation.
damn lies and statistics.
|
so you pounded data for win / loss for different levels, good job but i really don't think that you can call anything imbalanced based on who has a lower win percentage, (which zerg appears to have the lowest by a slight margin) but race mechanics / diversity and zerg is leading both of those with the lowest diversity and most micro intensive mechanics between all of the races zergs errors build up much faster than the other two races
|
OP I appreciate the work and thought that went into this, but was all dis-proven within the first couple of posts stating that all you have shown is that the matchmaking system is working. You cannot take balance information from a system designed to always give you a 50/50 win percentage.
And with that, can we close this thread?
|
Sanya12364 Posts
Best evidence to look at is the performance of random players and watch out for differences in the performance with the three races.
1. No selection biases (assumes blizzard's randomizer is good - check distribution of races when selecting random).
2. The same body of players playing all three races.
3. No biases in opponent selection (assumes blizzard's matchmaker doesn't skew - should check this.)
Analysis should look at the 6 possible match ups separately rather than aggregate into ZvAll, etc.
All this analysis will only capture the current state of the SCII metagame since the current body of players haven't fleshed out all strategies and tactics. There is also significant amount of copycat strategies going on.
All in all, it's easier to logically deduce imbalances based on build order advantages and flexibility rather than observe it in the statistical data. It'd be interesting exercise to try though.
|
The problem with your 'science' is that you have a forgone conclusion in your head about what reality is and you're trying to twist facts to suit reality. As others have said, you're drawing conclusions about a causal relationship from an imperfect data set with tons of degrees of freedom. Furthermore you have no basis to claim that win percentages indicate any sort of racial imbalance; there is a one hundred percent correlation between being born and dying but that doesn't mean one directly causes the other. You have little to no evidence to support any claim of causality. I'm all for trying to bring quantitative reasoning to Starcraft, but this pretty wildly off mark. Sorry to sound so harsh, but isnt that what peer review is about
|
The win rate does not prove imbalance.
Would a "perfectly balanced" Starcraft 2 have perfect 50% win rates? No.
|
It's hard to take numerical data to prove statistical imbalances in this game... mostly because there are too many variables.. a simplified version of "too many variables..."
IntoTheRainbow > Idra > Tester > IntoTheRainbow in a tournament... now multiply this concept by 1 billion since there are infinite possibilities and relationships between the several thousand diamonds world wide...
|
The data the OP used doesn't accurately reflect what it's trying to show. Imbalances are seen through data between the different combinations of match ups possible (PvT, TvZ, etc.) at different time points in the game. This is obviously difficult data to gather, and so far all we got are win/lose data of races (in different leagues) without variables like time and all the different match ups. This complex problem could be better shown with data on different match ups with games organized by length (i.e. group 1 is <17 mins of game time, group 2 is >17 mins of game time). This isn't a polished idea, but with more and more tests, you could definitely see at what intervals of time in the particular match up there seems to be a certain win/loss ratio and try to figure out the reason.
Even if you're a good player and you can tell that you have a certain disadvantage at some time interval, you still need to persuade the masses. With that point, good job OP for your work to support your idea even though it's a bit rough.
|
If you could just show how many players there are in each group, that would be great. If there are similar amount of players for each race in each league, I seriously don't see the problem, unless you can argue diamond league is too big and should be split into diamond and S+ or something.
And as mentioned before, T > Z > P > T does make matches unbalanced, but not these stats. But I'd first concentrate on the general strength of a race.
|
Statistics are cool and interresting, BUT if you want to "prove" than the game is imbalanced, look at the game itself not at some shiny statistics. Imo what really matters in the end is how the game is actually played at top level and if there are imbalances you should spot them by looking/playing games (at top level).
Plus if you spot an imbalance by actually playing the game, you can fix the matchup in a correct way. Imo statistics are a bonus, not a "proof" for imbalances.
|
|
|
|
|
|
EPT
