• Log InLog In
  • Register
Liquid`
Team Liquid Liquipedia
EDT 14:50
CEST 20:50
KST 03:50
  • Home
  • Forum
  • Calendar
  • Streams
  • Liquipedia
  • Features
  • Store
  • EPT
  • TL+
  • StarCraft 2
  • Brood War
  • Smash
  • Heroes
  • Counter-Strike
  • Overwatch
  • Liquibet
  • Fantasy StarCraft
  • TLPD
  • StarCraft 2
  • Brood War
  • Blogs
Forum Sidebar
Events/Features
News
Featured News
[ASL19] Finals Recap: Standing Tall10HomeStory Cup 27 - Info & Preview18Classic wins Code S Season 2 (2025)16Code S RO4 & Finals Preview: herO, Rogue, Classic, GuMiho0TL Team Map Contest #5: Presented by Monster Energy6
Community News
Firefly given lifetime ban by ESIC following match-fixing investigation9$25,000 Streamerzone StarCraft Pro Series announced4Weekly Cups (June 30 - July 6): Classic Doubles6[BSL20] Non-Korean Championship 4x BSL + 4x China9Flash Announces Hiatus From ASL66
StarCraft 2
General
Firefly given lifetime ban by ESIC following match-fixing investigation TL Team Map Contest #4: Winners Weekly Cups (June 30 - July 6): Classic Doubles The SCII GOAT: A statistical Evaluation The GOAT ranking of GOAT rankings
Tourneys
FEL Cracov 2025 (July 27) - $8000 live event $25,000 Streamerzone StarCraft Pro Series announced Sparkling Tuna Cup - Weekly Open Tournament RSL: Revival, a new crowdfunded tournament series WardiTV Mondays
Strategy
How did i lose this ZvP, whats the proper response Simple Questions Simple Answers
Custom Maps
[UMS] Zillion Zerglings
External Content
Mutation # 481 Fear and Lava Mutation # 480 Moths to the Flame Mutation # 479 Worn Out Welcome Mutation # 478 Instant Karma
Brood War
General
BGH Auto Balance -> http://bghmmr.eu/ ASL20 Preliminary Maps [G] Progamer Settings [ASL19] Finals Recap: Standing Tall SC uni coach streams logging into betting site
Tourneys
[Megathread] Daily Proleagues [BSL20] Non-Korean Championship 4x BSL + 4x China [BSL20] Grand Finals - Sunday 20:00 CET CSL Xiamen International Invitational
Strategy
Simple Questions, Simple Answers I am doing this better than progamers do.
Other Games
General Games
Nintendo Switch Thread Stormgate/Frost Giant Megathread Path of Exile What do you want from future RTS games? Beyond All Reason
Dota 2
Official 'what is Dota anymore' discussion
League of Legends
Positive Thoughts on Setting Up a Dual-Caliber FX
Heroes of the Storm
Simple Questions, Simple Answers Heroes of the Storm 2.0
Hearthstone
Heroes of StarCraft mini-set
TL Mafia
TL Mafia Community Thread Vanilla Mini Mafia
Community
General
US Politics Mega-thread Things Aren’t Peaceful in Palestine Russo-Ukrainian War Thread Summer Games Done Quick 2025! Stop Killing Games - European Citizens Initiative
Fan Clubs
SKT1 Classic Fan Club! Maru Fan Club
Media & Entertainment
Anime Discussion Thread [Manga] One Piece [\m/] Heavy Metal Thread
Sports
Formula 1 Discussion 2024 - 2025 Football Thread NBA General Discussion TeamLiquid Health and Fitness Initiative For 2023 NHL Playoffs 2024
World Cup 2022
Tech Support
Computer Build, Upgrade & Buying Resource Thread
TL Community
The Automated Ban List
Blogs
momentary artworks from des…
tankgirl
Culture Clash in Video Games…
TrAiDoS
from making sc maps to makin…
Husyelt
StarCraft improvement
iopq
Customize Sidebar...

Website Feedback

Closed Threads



Active: 683 users

Mirror Match-ups and Balance Metrics

Blogs > ChristianS
Post a Reply
ChristianS
Profile Blog Joined March 2011
United States3188 Posts
May 30 2013 15:12 GMT
#1
A Closer Look at How we Measure Balance


Starcraft players love discussing balance. And when they do, there's a variety of statistics they like to cite to support their claims. Some will cite win rates from Liquipedia or Aligulac, others will cite racial representation in Grandmaster League or some tournament, and others still will use metrics like total GSL champions for each race. Whatever the metric, the assumption is that with a sufficiently large player base, differences in skill between players will average out. What is left should be nothing more than a reflection of the absolute winningness of each race, or in a word, balance.

The underlying assumptions here are not always accurate. For example, the last metric listed is obviously quite poor. Even if it is assumed that racial balance is the only factor in determining GSL champions, a tally of GSL champions for each race still wouldn't reflect current SC2 balance; it would only reflect the accumulation of SC2 balance over its history since WoL release. For purposes of assessing the current state of the game, that is obviously useless, and I think most people on Team Liquid know that such a metric isn't good for much more than painting a broad, fuzzy picture of what WoL balance looked like.

But what about win rates or racial representation at the highest level as metrics? Most prefer to cite win rates because they are more statistically rigorous and have a larger sample size; some people prefer to cite racial representation at upper levels because it involves fewer arbitrary decisions about which games, regions, tournaments, etc. should be included in win rate data. But almost everyone will acknowledge that one or the other is pretty much an accurate representation of racial balance, aside from some statistical noise.

But there could be more at play here than statistical noise, specifically because of essential issues with the fact the Starcraft 2 is not one game, but six. Any given player does not just play Starcraft 2; they specifically play three different match-ups, each of which is radically different. Win rate analyses will generally respond to this issue by partitioning the data into particular matchups, and then analyzing balance in each matchup separately. This is probably the best way to salvage data taken from tournament games, but an essential sampling error is introduced when such a technique is employed. And this same error can interfere with racial representation in the higher levels of tournaments, poisoning that, too, as a metric for balance.

I'll explain this sampling error by a hypothetical. I do this partly because it's simpler and easier to understand in the abstract. But I also do this because balance is a notoriously heated issue, and it isn't my intention to argue that any particular match-up is or isn't imbalanced.

The Hypothetical

Imagine a hypothetical 2-race competitive game, with races A and B. The AvB match-up is highly skill-based, and quite balanced. The AvA and BvB match-ups are obviously balanced; but the AvA matchup is highly skill-based, while BvB wins are much more random, with highly skilled B players often losing to much worse B players simply because the matchup is much more chance-based. We'll also assume, for simplicity's sake, that a player's skill is the same in the AvB match-up as in their mirror match-up, even though in reality some players are obviously more skilled at one match-up than another.

A tournament is organized, with a large player pool of both A and B players, and the skill distribution appears to be quite even between the two races. In the first round, players are randomly paired and the winner advances. It doesn't much matter whether the format is single elimination, double elimination, or any other tournament format; all that matters is that each player is either playing the AvB match-up or their respective mirror match, and losers are eventually eliminated.

As stated previously, when the tournament begins the player pool had roughly equal skill of A players and B players. But by the end of the first round, a highly skilled B player is more likely to have been eliminated in a mirror matchup than a highly skilled A player, since the BvB matchup is so chaotic. Each round this pattern continues, until by the later stages of the tournament, the skill level of the B player pool is much lower than that of the A player pool. Obviously this quickly results in lower B representation in the upper levels of the tournament, since the now-weakened B player pool is more likely to fall in AvB matches.

How does this play out in the post-tournament balance statistics? Obviously anyone who looks at racial representation in the top 4, top 8, top 16, etc. will find see the A race dominating. More specifically, the racial representation will start out even, and then steadily drop for the B race as the tournament proceeds. Win rates in the AvB match-up will display a similar pattern; in the first round they should be perfectly balanced, but by the end of the tournament they will have shifted toward the A race. The effect is that an overall win rate of the AvB match-up will favor the A race, even though in a data set of games in which players are randomly paired and keep on playing win or lose, the match-up might appear to be balanced.

Conclusions

So what does that mean for Starcraft 2? Inevitably some balance warrior will use this argument to claim that Terran is actually more under-powered than balance statistics suggest, since the TvT match-up is often thought to be more skill-based than PvP or ZvZ. This is not the intention of this blog, and I believe that line of reasoning to be highly suspect. Starcraft 2 is not a hypothetical 2-race game; it is a real, complex 3-race game, and as such there are more considerations than just the randomness of the mirror match-up. For instance, if TvZ, TvP, and TvT all require quite different skills so very few Terrans are exceptionally good at all three, while all the Protoss match-ups require similar skills so being an all-around player is much more common, then both tournament representation and TvZ win rates would shift in favor of the Zerg race because talented Zergs are less likely to hit an unlucky match-up and get knocked out. There are so many confounding factors, I don't think much can be achieved by attempting to solve for and compensate for whatever error is introduced by this effect.

Ultimately, the only conclusion I wanted to draw is this: our current techniques for measuring balance have essential sampling errors in them. Because the winners in a tournament play more games than the losers, the win rate of any particular match-up is inevitably affected by the state of a race's other match-ups. Even if the other match-ups appear to be balanced, they could still throw off statistics by a disparity in the randomness with which wins are awarded.

*****
"Never attribute to malice that which is adequately explained by stupidity." -Robert J. Hanlon
Targe
Profile Blog Joined February 2012
United Kingdom14103 Posts
May 30 2013 15:52 GMT
#2
This is kind of the reason people say not to nerf things so quickly and it's also the reason some of us don't complain about the lack of toss players in GSL, there are just far more really good Terran and Zerg players.
11/5/14 CATACLYSM | The South West's worst Falco main
Pursuit_
Profile Blog Joined June 2012
United States1330 Posts
May 30 2013 17:40 GMT
#3
Wow, very interesting way of looking at game balance, 5/5. I feel almost silly for not having thought of this myself.
In Somnis Veritas
Antylamon
Profile Joined March 2011
United States1981 Posts
May 30 2013 19:01 GMT
#4
Finally, someone who puts my thoughts into words.

Great writeup, 5/5.
ChristianS
Profile Blog Joined March 2011
United States3188 Posts
May 30 2013 20:50 GMT
#5
On May 31 2013 00:52 Targe wrote:
This is kind of the reason people say not to nerf things so quickly and it's also the reason some of us don't complain about the lack of toss players in GSL, there are just far more really good Terran and Zerg players.

Wait, what? Did you read that in there somehow? Or am I misunderstanding you? My post wasn't about how many good players there are of each race, it was about how the characteristics of one match-up can throw off another match-up's statistics. + Show Spoiler +
Or at least, I think it was. I was pretty tired when I wrote it.
"Never attribute to malice that which is adequately explained by stupidity." -Robert J. Hanlon
Salivanth
Profile Blog Joined October 2010
Australia1071 Posts
Last Edited: 2013-05-31 03:35:57
May 31 2013 03:35 GMT
#6
Every post like this that I read reinforces one message for me.

I do not understand statistics. I must not attempt to make any argument that relies on statistics for the majority of my evidence, ever.

Very nice post Thanks for writing it!
<@Wikt> so you are one of those nega-fans <@Wikt> that hates the company that makes a game and everything they stand for <@Wikt> but still plays the game <@Wikt> (like roughly 30% of blizzard's player base, maybe much more...)
Targe
Profile Blog Joined February 2012
United Kingdom14103 Posts
Last Edited: 2013-05-31 08:05:12
May 31 2013 07:10 GMT
#7
On May 31 2013 05:50 ChristianS wrote:
Show nested quote +
On May 31 2013 00:52 Targe wrote:
This is kind of the reason people say not to nerf things so quickly and it's also the reason some of us don't complain about the lack of toss players in GSL, there are just far more really good Terran and Zerg players.

Wait, what? Did you read that in there somehow? Or am I misunderstanding you? My post wasn't about how many good players there are of each race, it was about how the characteristics of one match-up can throw off another match-up's statistics. + Show Spoiler +
Or at least, I think it was. I was pretty tired when I wrote it.


I'm saying it's a similar idea, many people cite Protoss' lack of presence in the GSL as a reason for imbalance when actually there are less good toss players.

Edit: I do get the post, I was just relating it to toss' lack of dominance and why that shouldn't be used for balance.
11/5/14 CATACLYSM | The South West's worst Falco main
Please log in or register to reply.
Live Events Refresh
uThermal 2v2 Circuit
16:00
Warmup Cup 2
uThermal528
TKL 258
IndyStarCraft 228
SteadfastSC112
Liquipedia
[ Submit Event ]
Live Streams
Refresh
StarCraft 2
uThermal 528
mouzHeroMarine 492
TKL 258
IndyStarCraft 228
SteadfastSC 112
UpATreeSC 100
BRAT_OK 79
ProTech55
MindelVK 23
StarCraft: Brood War
Bisu 1468
EffOrt 880
Shuttle 877
Mini 305
firebathero 279
Dewaltoss 123
TY 94
soO 49
sas.Sziky 40
IntoTheRainbow 8
[ Show more ]
LuMiX 1
Dota 2
Gorgc11319
qojqva2078
League of Legends
Grubby2036
Dendi1036
Counter-Strike
Fnx 1326
fl0m1193
flusha347
Stewie2K201
Foxcn178
sgares97
Heroes of the Storm
Liquid`Hasu499
Khaldor152
Other Games
summit1g2559
FrodaN2130
Beastyqt769
C9.Mang0238
mouzStarbuck188
KnowMe133
oskar124
Trikslyr70
QueenE46
Sick45
ArmadaUGS22
Organizations
Other Games
gamesdonequick45299
StarCraft 2
angryscii 18
Other Games
BasetradeTV16
StarCraft 2
Blizzard YouTube
StarCraft: Brood War
BSLTrovo
sctven
[ Show 20 non-featured ]
StarCraft 2
• Reevou 10
• Kozan
• sooper7s
• Migwel
• AfreecaTV YouTube
• LaughNgamezSOOP
• IndyKCrew
• intothetv
StarCraft: Brood War
• FirePhoenix5
• STPLYoutube
• ZZZeroYoutube
• BSLYoutube
Dota 2
• C_a_k_e 3413
• masondota21009
• Ler67
League of Legends
• Nemesis5587
• Jankos1443
• TFBlade1081
Other Games
• imaqtpie1335
• Shiphtur633
Upcoming Events
Replay Cast
5h 11m
RSL Revival
15h 11m
ByuN vs SHIN
Clem vs Reynor
OSC
18h 11m
Replay Cast
1d 5h
RSL Revival
1d 15h
Classic vs Cure
FEL
1d 21h
OSC
2 days
RSL Revival
2 days
FEL
2 days
FEL
2 days
[ Show More ]
CSO Cup
2 days
BSL20 Non-Korean Champi…
2 days
Bonyth vs QiaoGege
Dewalt vs Fengzi
Hawk vs Zhanhun
Sziky vs Mihu
Mihu vs QiaoGege
Zhanhun vs Sziky
Fengzi vs Hawk
Sparkling Tuna Cup
3 days
RSL Revival
3 days
FEL
3 days
BSL20 Non-Korean Champi…
3 days
Bonyth vs Dewalt
QiaoGege vs Dewalt
Hawk vs Bonyth
Sziky vs Fengzi
Mihu vs Zhanhun
QiaoGege vs Zhanhun
Fengzi vs Mihu
Replay Cast
5 days
Liquipedia Results

Completed

Proleague 2025-07-07
HSC XXVII
Heroes 10 EU

Ongoing

JPL Season 2
BSL 2v2 Season 3
Acropolis #3
KCM Race Survival 2025 Season 2
CSL 17: 2025 SUMMER
Copa Latinoamericana 4
Jiahua Invitational
Championship of Russia 2025
RSL Revival: Season 1
Murky Cup #2
BLAST.tv Austin Major 2025
ESL Impact League Season 7
IEM Dallas 2025
PGL Astana 2025
Asian Champions League '25
BLAST Rivals Spring 2025
MESA Nomadic Masters
CCT Season 2 Global Finals
IEM Melbourne 2025

Upcoming

2025 ACS Season 2: Qualifier
CSLPRO Last Chance 2025
CSL Xiamen Invitational
CSL Xiamen Invitational: ShowMatche
2025 ACS Season 2
CSLPRO Chat StarLAN 3
K-Championship
uThermal 2v2 Main Event
SEL Season 2 Championship
FEL Cracov 2025
Esports World Cup 2025
Underdog Cup #2
StarSeries Fall 2025
FISSURE Playground #2
BLAST Open Fall 2025
BLAST Open Fall Qual
Esports World Cup 2025
BLAST Bounty Fall 2025
BLAST Bounty Fall Qual
IEM Cologne 2025
FISSURE Playground #1
TLPD

1. ByuN
2. TY
3. Dark
4. Solar
5. Stats
6. Nerchio
7. sOs
8. soO
9. INnoVation
10. Elazer
1. Rain
2. Flash
3. EffOrt
4. Last
5. Bisu
6. Soulkey
7. Mini
8. Sharp
Sidebar Settings...

Advertising | Privacy Policy | Terms Of Use | Contact Us

Original banner artwork: Jim Warren
The contents of this webpage are copyright © 2025 TLnet. All Rights Reserved.