EPTAbstract: A simple statistical analysis revealed that, you guessed it, SC2 is IMBALANCED (technically). However, the degree of imbalance is so infintessimally small that you’d literally have to play millions of games to ever know it. Moreover, my initial analysis suggests that the balance is very consistent across all player skill levels, and I think Blizzard deserves a lot of credit for the hard work they’ve put into balancing Starcraft 2.
Introduction: Ever since the SC2 Beta was released, people have been arguing about whether and which races are imbalanced, and what should be done to fix the balance problem in Starcraft 2. There have been a plague of articles on the subject, some of which I’ve read as I trolled around the TeamLiquid forums. I decided that the best way to settle the issue once and for all was to take a scientific approach, using statistical analysis to test the hypothesis that the races in Starcraft 2 are inbalanced. I'm starting with the only data I have available, which is a table of win/loss results for each game that's been played so far. My hypothesis is that if SC2 is imbalanced, then some races will have better win/loss ratings than others.
Methods: The data I used were found in the ‘stats’ section at sc2ranks.com in the morning of August 16th. Data are reported as win percentages for each race at each league level, and in parentheses is a number that I assumed were the total number of wins in each category. To answer the question of whether Starcraft 2 races have different win/loss ratings, I used a Chi squared analysis in Excel, which let me compare the observed win rate with the rate that would be expected if all races were balanced (i.e. wins were randomly distributed among races within each league). I used available win percentages to estimate overal win rates for the 50 million games of SC2 that have so far been played. I also did a somewhat wonky power analysis to explore how many games you’d have to play before noticing the imbalance. Spoiler: it’s a lot.
Results: OK, Starcraft 2 appears to be technically slightly imbalanced, but before I discuss that I want to raise some observations. First, the league system seems to work fairly well at giving players of diverse skill sets a similarly frustrating online experience. While Bronze players are the whipping-boys of the online SC2 community, with a 42% overall win rate, things are more even in the other leagues, with win rates of 50% for Silver players, rising gradually through the leagues to 56% for the average Diamond player.
![[image loading]](http://i667.photobucket.com/albums/vv32/GagnarTheUnruly/winperc.jpg)
![[image loading]](http://i667.photobucket.com/albums/vv32/GagnarTheUnruly/percdiff.jpg)
Blue shading indicates the race does better than expected, red indicates the race does worse, according to the results from my chi-square analysis (p < 0.001 where significant, n ~ 50 million, 3 d.f.). The light values indicate that differences are only detectable at sample sizes of about 1 million games played.
Regarding the imbalance issue, with the roughly 50 million games that have been uploaded into the sc2ranks.com database, it seems that Starcraft 2 is ever so slightly imbalanced (see Table 1). One thing that really jumps out is that the game is least balanced for bronze players, where Terran players have a distinct disadvantage, winning about 2% fewer games than the other races, and Protoss players have a slight advantage, winning about 1% more games than average.
Among the other leagues, however, the story is quite different. The win rate disparities, although statistically significant, are small, never exceeding a third of a percent in any direction, and averaging a disparity of about 1.5 tenths of a percent. In general, the game appears to be very well balanced from silver through diamond leagues. Within those leagues, Terran has a slight advantage (see, Terran is IMBA!), meaning you’ll win about 2 games in a thousand more often than you should. With Zerg, a thousand games would yield about 1.5 more losses than average (see Zerg is weak and IdrA is the MAN!). Protoss and random aren’t much different from average.
For Diamond-level players, terran is the strongest race with a 56.1% win percentage and random is the worst (55.53%). A Diamond Solo Zen Master would have to play 1801 games to win as random and 1800 to win as zerg, but only 1794 to win as Protoss and 1784 to win as Terran. So if you want Terran mastery, you’ll get it in 17 fewer games than a random player! That’s enough time to take your significant other out to dinner, so if relationships are important to you, I recommend Terran. It also could help win you $2000. For sharing this knowledge with you, I expect a portion of the proceeds.
![[image loading]](http://i667.photobucket.com/albums/vv32/GagnarTheUnruly/solozen.jpg)
OK, so I’ve said that the races are imbalanced, but the imbalances are pretty darn small. Because Chi-square tests become extremely sensitive at very large sample sizes, I decided to do a jury rigged power analysis to determine just how many games you’d need to play to detect an imbalance. I basically just ran the Chi-square analysis using simulated data sets of various sample sizes to see how many games you'd have to play before it was obvious that some races won more games than other races.
So, let’s say you play ten thousand games using each race equally (2500 games with each race). If you’re a Silver through Diamond player, you wouldn’t be able to detect a statistically significant difference in your win rate. A Bronze player will notice terran has a disadvantage.
Now let’s say you keep playing until you get to a hundred thousand games. A Bronze player will notice that Protoss is a little extra strong, but players in the other leagues will still think the game is perfectly balanced. In fact, you’d have to play about a million games before you started to notice that the races were imbalanced in the diamond league.
To show the level of balance graphically, I used a sample size of about 100,000 games, played according to the racial proclivities of the player base (so, not all races used equally), because this was the largest sample size for which I could accurately estimate numbers of games won for each race. You can see on the log-transformed graph below that for silver through platinum players, the races are very well balanced, and that racial imbalances are only apparent in bronze level play.
![[image loading]](http://i667.photobucket.com/albums/vv32/GagnarTheUnruly/balancegraph.jpg)
The y-axis just represents how divergent the race is from average (0 on the graph). Higher values indicate that the race does better than expected, and lower means the race does worse. The colored regions at the top and bottom of the graph are the regions of statistical significance (p = 0.005). Mostly the differences between the races aren't significant.
Discussion: My data show that, within a league, each of the races has a rougly equal chance of winning a randomly selected game. This indicates that the balance of SC2 is probably pretty good. People have pointed out that matchmaking would cause this to happen, because it strives to set each player's win rate at 50%. That in turn would cause the win rate of each race to trend towards 50%. That being the case, poor balance would tend to result in 'weak' races getting pushed into the lower tiers of play. Because we don't see that happening either within or among leagues (data not shown), my data suggest both that the matchmaking system works well and that SC2 is inherrently pretty well balanced.
Technically speaking, my results suggest that Starcraft 2 is NOT perfectly balanced, but that the degree of imbalance is so small that it is functionally imperceptible until literally hundreds of thousands of games are played. Also, the imbalance is clearly most extreme and noticeable in the Bronze league. Perhaps this should come as no surprise, because less experienced players are probably much less predicable than more experienced players, because the weakest players are likely to pick the more familiar Terran race, and because the overall level of play in each league is generally lower (in my opinion) than during the Beta, meaning that Blizzard probably didn’t get a strong chance to balance the game for players playing at the current level of Bronze play.
In short, it's my opinion that the designers of SC2 did an amazingly good job of balancing the game, and across a very wide spectrum of player skill levels. So, unless you’ve played a few hundred thousand games of Starcraft 2, rest assured that any sense you have that the game is imbalanced is probably illusory. Hopefully, this analysis will put to rest the question of balance in SC2, at least in many people’s minds.
As a possible next step I could do the same analysis for SC1, and test the hypothesis that SC1 is more balanced than SC2. Based mostly on the suggestions of other people in this thread, I also think it would be a good idea to try to do an analysis where I test to see whether there are matchup imbalances, and whether race use is really consistent among leagues.
Finally, I ask that you not take these results too seriously. This was just a little project that I did for fun. I believe that my results suggest that SC2 is pretty well balanced in general, but I had a pretty limited data set and there are a lot of factors I couldn't account for. If I'm able to get better data I'll do a more involved analysis (unless my experience with this thread burns me out utterly), but until then this is the best that I can offer! Enjoy it for what it's worth!
