• Log InLog In
  • Register
Liquid`
Team Liquid Liquipedia
EST 04:33
CET 10:33
KST 18:33
  • Home
  • Forum
  • Calendar
  • Streams
  • Liquipedia
  • Features
  • Store
  • EPT
  • TL+
  • StarCraft 2
  • Brood War
  • Smash
  • Heroes
  • Counter-Strike
  • Overwatch
  • Liquibet
  • Fantasy StarCraft
  • TLPD
  • StarCraft 2
  • Brood War
  • Blogs
Forum Sidebar
Events/Features
News
Featured News
[ASL22] Finals Preview: Sweet Victory!12[ASL22] Ro4 Preview: Mirror Mirror6[ASL22] Ro8 Preview: Within Reach5[ASL22] Ro8 Preview: In A Tizzy11[ASL22] Ro16 Preview: Holy Diver5
Community News
Weekly Cups (Nov 9-15): Clem and herO take over0Dark completes military service, returns to SC212[BSL23] RO16 Group Stage20Weekly Cups (Nov 2-8): herO back on top?2Weekly Cups (Oct 26 - Nov 1): MaxPax, herO, Solar win0
StarCraft 2
General
Weekly Cups (Nov 9-15): Clem and herO take over No more SC2 streamers on team liquid? Dark completes military service, returns to SC2 Serral's getting trashed by New Catsle's Mind Weekly Cups (Nov 2-8): herO back on top?
Tourneys
RSL goes to London! 2026 Offline Finals Nov 21-22 Sparkling Tuna Cup - Weekly Open Tournament Sea Duckling Open (Global, Bronze-Diamond) Dark returns! GSL CK #7 4v4 Team Battle (Nov 19) soO & FanTaSy return! SC2 Kevin Scrim 4v4 (Nov 16)
Strategy
[H] ZvP Mid-Late Game: Stalkers Collossi HT
Custom Maps
[G] How to playtest arcade maps Starcraft 2 Advance [D] the control group Simple Questions/Answers
External Content
Mutation # 552 Secret Storm The PondCast: SC2 News & Results Mutation # 551 Robotic Revival Mutation # 550 Graveyard Shift
Brood War
General
NaDa's Body List of top players active on ShieldBattery BW General Discussion Shield Battery Server New Patch BGH Auto Balance -> http://bghmmr.eu/
Tourneys
[IPSL] Winter 26-27 Round of 16! [Megathread] Daily Proleagues 2v2v2v2 Tournament Stage 2: Players [$1250] International ShieldBattery League
Strategy
Pluto's TvP 5Marine 2Medic Push Simple Questions, Simple Answers Infested Terran strategies in 2026 Nuke strategy worked in K-League 2026-11-09
Other Games
General Games
Stormgate/Frost Giant Megathread Free Space RTS Game, optimised f. Phone/Tablet etc Nintendo Switch Thread List of Game Threads The PlayStation 5
Dota 2
League of Legends
[TL LoL EUW IHs] Teemo shall perish
Heroes of the Storm
Heroes of the Storm 2.0
Hearthstone
Deck construction bug
TL Mafia
TL Mafia Community Thread
Community
General
Russo-Ukrainian War Thread US Politics Mega-thread Artificial Intelligence Thread Things Aren’t Peaceful in Palestine Trading/Investing Thread
Fan Clubs
Fan Club Resource Thread Reynor Fan Club The Flash Fan Club
Media & Entertainment
Movie Discussion! Anime Discussion Thread [Manga] One Piece
Sports
Football (Soccer) Thread TeamLiquid Health and Fitness Initiative For 2023 MLB/Baseball 2023
World Cup 2022
Tech Support
Computer Build, Upgrade & Buying Resource Thread
TL Community
The Automated Ban List Recent Gifted Posts
Blogs
The time I organised my…
AssyrianKing
Why I built Repalyzer (1.1.…
BlueStar
Innovate or Game Over?
TrAiDoS
ShieldBattery on Mac
Kraekkling
Can Bots Beat Pros?? Starcr…
namkraft
Customize Sidebar...

Website Feedback

Closed Threads



Active: 13285 users

Mirror Match-ups and Balance Metrics

Blogs > ChristianS
Post a Reply
ChristianS
Profile Blog Joined March 2011
United States3308 Posts
May 30 2013 15:12 GMT
#1
A Closer Look at How we Measure Balance


Starcraft players love discussing balance. And when they do, there's a variety of statistics they like to cite to support their claims. Some will cite win rates from Liquipedia or Aligulac, others will cite racial representation in Grandmaster League or some tournament, and others still will use metrics like total GSL champions for each race. Whatever the metric, the assumption is that with a sufficiently large player base, differences in skill between players will average out. What is left should be nothing more than a reflection of the absolute winningness of each race, or in a word, balance.

The underlying assumptions here are not always accurate. For example, the last metric listed is obviously quite poor. Even if it is assumed that racial balance is the only factor in determining GSL champions, a tally of GSL champions for each race still wouldn't reflect current SC2 balance; it would only reflect the accumulation of SC2 balance over its history since WoL release. For purposes of assessing the current state of the game, that is obviously useless, and I think most people on Team Liquid know that such a metric isn't good for much more than painting a broad, fuzzy picture of what WoL balance looked like.

But what about win rates or racial representation at the highest level as metrics? Most prefer to cite win rates because they are more statistically rigorous and have a larger sample size; some people prefer to cite racial representation at upper levels because it involves fewer arbitrary decisions about which games, regions, tournaments, etc. should be included in win rate data. But almost everyone will acknowledge that one or the other is pretty much an accurate representation of racial balance, aside from some statistical noise.

But there could be more at play here than statistical noise, specifically because of essential issues with the fact the Starcraft 2 is not one game, but six. Any given player does not just play Starcraft 2; they specifically play three different match-ups, each of which is radically different. Win rate analyses will generally respond to this issue by partitioning the data into particular matchups, and then analyzing balance in each matchup separately. This is probably the best way to salvage data taken from tournament games, but an essential sampling error is introduced when such a technique is employed. And this same error can interfere with racial representation in the higher levels of tournaments, poisoning that, too, as a metric for balance.

I'll explain this sampling error by a hypothetical. I do this partly because it's simpler and easier to understand in the abstract. But I also do this because balance is a notoriously heated issue, and it isn't my intention to argue that any particular match-up is or isn't imbalanced.

The Hypothetical

Imagine a hypothetical 2-race competitive game, with races A and B. The AvB match-up is highly skill-based, and quite balanced. The AvA and BvB match-ups are obviously balanced; but the AvA matchup is highly skill-based, while BvB wins are much more random, with highly skilled B players often losing to much worse B players simply because the matchup is much more chance-based. We'll also assume, for simplicity's sake, that a player's skill is the same in the AvB match-up as in their mirror match-up, even though in reality some players are obviously more skilled at one match-up than another.

A tournament is organized, with a large player pool of both A and B players, and the skill distribution appears to be quite even between the two races. In the first round, players are randomly paired and the winner advances. It doesn't much matter whether the format is single elimination, double elimination, or any other tournament format; all that matters is that each player is either playing the AvB match-up or their respective mirror match, and losers are eventually eliminated.

As stated previously, when the tournament begins the player pool had roughly equal skill of A players and B players. But by the end of the first round, a highly skilled B player is more likely to have been eliminated in a mirror matchup than a highly skilled A player, since the BvB matchup is so chaotic. Each round this pattern continues, until by the later stages of the tournament, the skill level of the B player pool is much lower than that of the A player pool. Obviously this quickly results in lower B representation in the upper levels of the tournament, since the now-weakened B player pool is more likely to fall in AvB matches.

How does this play out in the post-tournament balance statistics? Obviously anyone who looks at racial representation in the top 4, top 8, top 16, etc. will find see the A race dominating. More specifically, the racial representation will start out even, and then steadily drop for the B race as the tournament proceeds. Win rates in the AvB match-up will display a similar pattern; in the first round they should be perfectly balanced, but by the end of the tournament they will have shifted toward the A race. The effect is that an overall win rate of the AvB match-up will favor the A race, even though in a data set of games in which players are randomly paired and keep on playing win or lose, the match-up might appear to be balanced.

Conclusions

So what does that mean for Starcraft 2? Inevitably some balance warrior will use this argument to claim that Terran is actually more under-powered than balance statistics suggest, since the TvT match-up is often thought to be more skill-based than PvP or ZvZ. This is not the intention of this blog, and I believe that line of reasoning to be highly suspect. Starcraft 2 is not a hypothetical 2-race game; it is a real, complex 3-race game, and as such there are more considerations than just the randomness of the mirror match-up. For instance, if TvZ, TvP, and TvT all require quite different skills so very few Terrans are exceptionally good at all three, while all the Protoss match-ups require similar skills so being an all-around player is much more common, then both tournament representation and TvZ win rates would shift in favor of the Zerg race because talented Zergs are less likely to hit an unlucky match-up and get knocked out. There are so many confounding factors, I don't think much can be achieved by attempting to solve for and compensate for whatever error is introduced by this effect.

Ultimately, the only conclusion I wanted to draw is this: our current techniques for measuring balance have essential sampling errors in them. Because the winners in a tournament play more games than the losers, the win rate of any particular match-up is inevitably affected by the state of a race's other match-ups. Even if the other match-ups appear to be balanced, they could still throw off statistics by a disparity in the randomness with which wins are awarded.

*****
"Never attribute to malice that which is adequately explained by stupidity." -Robert J. Hanlon
Targe
Profile Blog Joined February 2012
United Kingdom14103 Posts
May 30 2013 15:52 GMT
#2
This is kind of the reason people say not to nerf things so quickly and it's also the reason some of us don't complain about the lack of toss players in GSL, there are just far more really good Terran and Zerg players.
11/5/14 CATACLYSM | The South West's worst Falco main
Pursuit_
Profile Blog Joined June 2012
United States1330 Posts
May 30 2013 17:40 GMT
#3
Wow, very interesting way of looking at game balance, 5/5. I feel almost silly for not having thought of this myself.
In Somnis Veritas
Antylamon
Profile Joined March 2011
United States1981 Posts
May 30 2013 19:01 GMT
#4
Finally, someone who puts my thoughts into words.

Great writeup, 5/5.
ChristianS
Profile Blog Joined March 2011
United States3308 Posts
May 30 2013 20:50 GMT
#5
On May 31 2013 00:52 Targe wrote:
This is kind of the reason people say not to nerf things so quickly and it's also the reason some of us don't complain about the lack of toss players in GSL, there are just far more really good Terran and Zerg players.

Wait, what? Did you read that in there somehow? Or am I misunderstanding you? My post wasn't about how many good players there are of each race, it was about how the characteristics of one match-up can throw off another match-up's statistics. + Show Spoiler +
Or at least, I think it was. I was pretty tired when I wrote it.
"Never attribute to malice that which is adequately explained by stupidity." -Robert J. Hanlon
Salivanth
Profile Blog Joined October 2010
Australia1071 Posts
Last Edited: 2013-05-31 03:35:57
May 31 2013 03:35 GMT
#6
Every post like this that I read reinforces one message for me.

I do not understand statistics. I must not attempt to make any argument that relies on statistics for the majority of my evidence, ever.

Very nice post Thanks for writing it!
<@Wikt> so you are one of those nega-fans <@Wikt> that hates the company that makes a game and everything they stand for <@Wikt> but still plays the game <@Wikt> (like roughly 30% of blizzard's player base, maybe much more...)
Targe
Profile Blog Joined February 2012
United Kingdom14103 Posts
Last Edited: 2013-05-31 08:05:12
May 31 2013 07:10 GMT
#7
On May 31 2013 05:50 ChristianS wrote:
Show nested quote +
On May 31 2013 00:52 Targe wrote:
This is kind of the reason people say not to nerf things so quickly and it's also the reason some of us don't complain about the lack of toss players in GSL, there are just far more really good Terran and Zerg players.

Wait, what? Did you read that in there somehow? Or am I misunderstanding you? My post wasn't about how many good players there are of each race, it was about how the characteristics of one match-up can throw off another match-up's statistics. + Show Spoiler +
Or at least, I think it was. I was pretty tired when I wrote it.


I'm saying it's a similar idea, many people cite Protoss' lack of presence in the GSL as a reason for imbalance when actually there are less good toss players.

Edit: I do get the post, I was just relating it to toss' lack of dominance and why that shouldn't be used for balance.
11/5/14 CATACLYSM | The South West's worst Falco main
Please log in or register to reply.
Live Events Refresh
Next event in 2h 27m
[ Submit Event ]
Live Streams
Refresh
StarCraft 2
SortOf 162
Lowko105
StarCraft: Brood War
GuemChi 1957
Zeus 1341
Horang2 881
Bisu 648
Yoon 434
Larva 236
Leta 215
actioN 126
sorry 89
Killer 83
[ Show more ]
Light 63
Stork 55
Sacsri 48
Mini 37
ToSsGirL 28
NotJumperer 24
Rush 22
yabsab 18
ajuk12(nOOB) 14
Sharp 14
Movie 11
Terrorterran 10
League of Legends
JimRising 428
Reynor69
Counter-Strike
shoxiejesuss515
olofmeister489
Heroes of the Storm
Trikslyr23
Other Games
summit1g12879
ceh9601
Happy208
Mew2King101
Fuzer 17
Pyrionflax16
Organizations
Other Games
gamesdonequick561
StarCraft: Brood War
lovetv 5
[ Show 9 non-featured ]
StarCraft 2
• AfreecaTV YouTube
• intothetv
• Kozan
• IndyKCrew
• LUISG 0
• Migwel
StarCraft: Brood War
• BSLYoutube
• STPLYoutube
• ZZZeroYoutube
Upcoming Events
LiuLi Cup
2h 27m
Maru vs Rogue
Clem vs herO
Monday Night Weeklies
7h 27m
PiGosaur Cup
15h 27m
The PondCast
1d
LiuLi Cup
1d 2h
OSC
1d 6h
Replay Cast
1d 13h
GSL
2 days
Maru vs Dark
Patches Events
2 days
OSC
2 days
[ Show More ]
Replay Cast
2 days
KCM Race Survival
3 days
RSL Revival
4 days
BSL
4 days
eOnzErG vs StRyKeR
Cross vs JDConan
RSL Revival
5 days
IPSL
5 days
Ret vs ggnore13
nOmaD vs Aether
BSL
5 days
Bonyth vs Doodle
Kyrie vs Jimin
Replay Cast
5 days
WardiTV Weekly
6 days
Kung Fu Cup
6 days
Liquipedia Results

Completed

Proleague 2026-11-16
Blizzard Classic Cup 2026
HCC Season 3

Ongoing

Acropolis #5
KCM Race Survival 2026 Season 4
International ShieldBattery League
BSL Season 23
IPSL Winter 2026-27
Calamity Cup Div. A Season 8
Underdog Cup 2026 Div 2-3
Stake Ranked Episode 6
BLAST Rivals Fall 2026
IEM Beijing 2026
PGL Masters Bucharest 2026
Stake Ranked Episode 5
1win Private Club #2
Thunderpick World Champ. '26
ESL Pro League Season 24
Stake Ranked Episode 4
1win Private Club #1
Logitech G Play Connect 2026
SL StarSeries Fall 2026
FISSURE Playground #3
BLAST Open Fall 2026

Upcoming

Sector 591
YSL S4
SC4ALL II: Brood War
NBSL 2026 Xiamen LAN
BSL 23: Non-Korean Championship
HSC XXX
Stellar Fest 2: Lunar Cup
StarCraft2 Weekly Open Tournament 2026 Grand Finals
SC4ALL II: StarCraft II
Kung Fu Cup 2026 Grand Finals
RSL Offline Finals
IEM Kraków 2027
PGL Bucharest 2027
BLAST Bounty Winter 2027
PGL Major Singapore 2026
TLPD

1. ByuN
2. TY
3. Dark
4. Solar
5. Stats
6. Nerchio
7. sOs
8. soO
9. INnoVation
10. Elazer
1. Rain
2. Flash
3. EffOrt
4. Last
5. Bisu
6. Soulkey
7. Mini
8. Sharp
Sidebar Settings...

Advertising | Privacy Policy | Terms Of Use | Contact Us

Original banner artwork: Jim Warren
The contents of this webpage are copyright © 2026 TLnet. All Rights Reserved.