Playing With ELO

Musoeun

United States4324 Posts

February 03 2011 03:55 GMT

Some Additional Thoughts:

Building off the numbers collected below, I ran a "normalizing" algorithm to attempt to discover how raw number of players reflected actual chances. I added up the matchup ELO of all "good" players in a race, and divided it by the total number of "good" players of all races. I then obtained a standard figure by averaging the vs [race] ELO of all "good" players and dividing that by 3. Higher is better. Results:

X vs Protoss
Zerg: 761.59
Protoss: 750.18
[All: 717.65]
Terran: 641.18

Conclusions: Zerg have a small advantage in the matchup (taking the ZvP number compared to the mirror) but not much. Terran are at a significant disadvantage against Protoss. Also, PvP is a matchup in which it is possible to be very good.

X vs Terran:
Terran: 973.85
[All: 721.32]
Protoss: 652.80
Zerg: 537.30

Conclusions: As noted in the OP, this seems skewed due to the larger number of good TvT players compared to PvT and ZvTers. My guess was that this was due to a larger number of Terrans overall. I would also venture that TvT is the most disparate mirror: you're either good or bad with little middle ground. Other than that, Protoss do significantly better than Zerg against Terran.

X vs Zerg:
Terran: 894.00
[All: 723.41]
Zerg: 638.88
Protoss: 637.35

Conclusions: As before, ZvP seems to correspond well with ZvZ, the mirror, indicating a well-balanced matchup. Terran have a significant advantage over Zerg. I find it interesting that the average good ZvZer doesn't have that great an advantage; this may also indicate a little Z>P.

I also ran similar tests comparing all "good" players by general ELO, using the same "good" >= 2100 definition. When looking at overall ELO, this is probably not quite accurate: 2120 might be a better cut-off. However, I was unwilling to change the definition. This gave 11 Protoss, 11 Terran, and 11 Zerg. Results:

X vs X
Terran: 726.03
[All: 723.84]
Protoss: 723.30
Zerg: 722.18

Conclusion: going by ELO, with the particular sample I took, Terran has a small overall advantage, and Zerg a miniscule overall disadvantage. MSL semifinals notwithstanding. (I'm not sure whether that indicates a flaw in the method or not: Stork vs ZerO was an unfortunate for Stork but in better condition he might have won; Snow vs Jaedong could have gone either way; in the great vs Kal match the clearly better player choked hard.)

+ Show Spoiler [OP] +

Somebody in the PR was talking about HiyA's TvP, which used to be excellent, and how was it now? I thought I'd see, so I took a look at the ELO table (answer: not bad). Then I got carried away into balance issues.

I realize nobody's debating BW balance. We all know it's pretty good, but not quite perfect. I'm going to merely illustrate that, so:

In looking at the ELO tables, I've estimated that a player who's "good" in a matchup right now corresponds to an ELO at or over 2100. Over 2200 we can call "very good". Over 2300 is matchup-bonjwa level. (Hi Flash.) I'm theorizing that a balanced matchup should have an equal number of "good" players of each race.

Numbers:

X vs Protoss: 17 "good" players.
6 Protoss: Bisu, Horang2, Shuttle, Kal, free, Stork
5 Terran: Flash, fantasy, Sea, Bogus, HiyA
6 Zerg: Jaedong, Calm, ZerO, Hydra, Hyuk, RorO

(Very Good: Flash, Jaedong, fantasy)
Balance Grade: A

X vs Terran: 20 players.
6 Protoss: Stork, BeSt, Bisu, Snow, free, Kal
9 Terran: Flash, fantasy, sKyHigh, Sea, Leta, Iris, Mind, Light, BaBy
5 Zerg: Jaedong, ZerO, Hydra, Shine, RorO

(Very Good: Flash, Jaedong, Stork, fantasy)
Balance Grade: ...Hell, I don't know. Why so many extra Terran?

X vs Zerg: 17 "good" players.
5 Protoss: Bisu, Stork, Stats, Kal, Movie
7 Terran: Flash, Light, Midas, fantasy, Sea, TurN, BaBy
5 Zerg: Jaedong, Shine, Hydra, great, Modesty

(Very Good: Flash, Jaedong, Bisu, Light)
Balance Grade: B

(Very Good in 3 Matchups: Flash, Jaedong
Very Good in 2 Matchups: fantasy
Balance Grade: B-)

Mostly irrelevant. Mostly balanced but imperfect. I get that we know this. But I like to confirm things we know, and I'd like some opinions about the XvT weirdness.

+ Show Spoiler [SC2 fun] +

If I take a wild guess that we can apply the same 2100 = Good criterion to SC2 (I suspect BW ranking has a little inflation, so this might actually be 2050-2080 for SC2 but I'm not going to try to peg it exactly) we have some interesting results.

KOREA

X vs Protoss: 7 players. 1 Protoss, 5 Terran, 1 Zerg.
Balance Grade: F

X vs Terran: 6 players. 1 Protoss, 4 Terran, 1 Zerg.
Balance Grade: F

X vs Zerg: 4 players. 1 Protoss, 2 Terran, 1 Zerg
Balance Grade: B+

NOT KOREA

X vs Protoss: 19 players. 6 Protoss, 8 Terran, 5 Zerg.
Balance Grade: C

X vs Terran: 23 players. 7 Protoss, 13 Terran, 3 Zerg.
Balance Grade: F

X vs Zerg: 17 players. 5 Protoss, 10 Terran, 2 Zerg.
Balance Grade: WTF why do Zerg suck vs Zerg.

Disclaimer: Although I think there's clear illustration of current imbalance in SC2, I firmly believe that at this point we have to attribute this (mainly) to lack of a fully developed metagame. (Compare Broodwar Terran before Slayers"Dropship"Boxer.)

What I find most intriguing here is that in both BW and SC2 we currently have more Terrans good at TvT than Protoss good at PvT or Zerg good at ZvT. I'd take a wild guess (in fact I'm pretty sure it's very true in SC2) that this is due to Terran being the highest percentage of players.

emperorchampion

Canada9496 Posts

February 03 2011 04:06 GMT

Well, if you look at Terran, the majority of the better players is from Terran (TvT match up). I imagine that has something to do with it, although I'm not really sure why. Maybe it's because of the nature of TvT, and the way the games tend to be long (and boring xD)- favoring a different skill set. Or maybe it's the way the maps work with TvT, making winning more random in some fashion? Or the fact that certain players understand TvT in a way others don't, kinda like JvZ.

I really don't know, just some ideas.

Goragoth

New Zealand1065 Posts

February 03 2011 04:47 GMT

Unfortunately it seems that if you dare to suggest that BW is anything but perfectly balanced you tend to get shouted down or even banned on TL. While there is an argument that maps make a huge difference in balance, top players frequently seem to overcome the maps (e.g. SaviOr, Jaedong on Katrina). Besides, if the imbalances are due to maps or inherent in the game doesn't really matter.

I'm more interested in the historical balance (at the very top level of play) than what it is right now, so let's run by some numbers:
Peak Elo - vT (top 10)
Z 5
T 3
P 2

Peak Elo - vZ (top 10)
Z 2
T 7
P 1

Peak Elo - vP (top 10)
Z 3
T 4
P 3

Total:
Z 10
T 14
P 6

Looks very lopsided to me.
How about gold medals won (combined OSL + MSL):
Z 19 (including this MSL)
T 23
P 12

Terran and Zerg are pretty close but Protoss once again is getting shafted. Virtually the entire history of pro BW is like this too, usually we have a Terran that dominates, a Zerg rival who is also very good but still second best and then Protoss players that are only known because they managed to not suck as bad as the rest of their race, not because they were particularly dominant. There have been periods where things looked different (SaviOr era, Protoss Golden Age, Swarm Season) but it certainly has been the dominant narrative.

saltywet

Hong Kong1316 Posts

February 03 2011 07:21 GMT

fantasy is not very good in 2 matchups, just saying.

still flash (bonjwa status in 3 matchups) and jaedong (very good in 3 matchups)

Comeh

United States18919 Posts

February 03 2011 07:36 GMT

On February 03 2011 13:47 Goragoth wrote:
Unfortunately it seems that if you dare to suggest that BW is anything but perfectly balanced you tend to get shouted down or even banned on TL. While there is an argument that maps make a huge difference in balance, top players frequently seem to overcome the maps (e.g. SaviOr, Jaedong on Katrina). Besides, if the imbalances are due to maps or inherent in the game doesn't really matter.

I'm more interested in the historical balance (at the very top level of play) than what it is right now, so let's run by some numbers:
Peak Elo - vT (top 10)
Z 5
T 3
P 2

Peak Elo - vZ (top 10)
Z 2
T 7
P 1

Peak Elo - vP (top 10)
Z 3
T 4
P 3

Total:
Z 10
T 14
P 6

Looks very lopsided to me.
How about gold medals won (combined OSL + MSL):
Z 19 (including this MSL)
T 23
P 12

Terran and Zerg are pretty close but Protoss once again is getting shafted. Virtually the entire history of pro BW is like this too, usually we have a Terran that dominates, a Zerg rival who is also very good but still second best and then Protoss players that are only known because they managed to not suck as bad as the rest of their race, not because they were particularly dominant. There have been periods where things looked different (SaviOr era, Protoss Golden Age, Swarm Season) but it certainly has been the dominant narrative.

'
I have to disclaim the fact that historical relveance is important - after all, there are periods of maps that certainly favor one race, or certain trends in the game that favor one race over another (mech TvZ was a problem for zerg for awhile).

Also, pretty much PvZ prior to the bisu build is pretty irrelevant - the matchup was pretty inarguably balanced for zerg heavily - even now we are seeing a resurgence of quality PvZ play by protoss players.

xxpack09

United States2160 Posts

February 03 2011 08:03 GMT

On February 03 2011 16:36 Comeh wrote:

Show nested quote +

I always wonder (even though I know this isn't possible) what the lifetime PvZ winrate is in games WITH Forge FE and in games WITHOUT Forge FE. I do have to say though playing without Forge FE is damn hard. 2gate is pretty cheesy even if you do 2gate -> expo and most 1gate techs can just get crushed so hard by hydralisks...

Trap

United States395 Posts

February 03 2011 08:27 GMT

TvT is such a different beast from the other matchups. Less volatile in the early game than ZvZ and PvP, and dynamic in the mid and late game where a good TvTer has lots of opportunity to outposition and outthink his opponent; it's not that surprising that some Terrans are really good at TvT. The abnormal number of skilled TvT players is ultimately bad news if you are a Terran progamer.

Also there are some problems with the statistics sample you've chosen. According to your cutoffs there is only 1 more "good" TvZ player than expected, which isn't very convincing of imbalance. Midas's TvZ is kind of inflated by past achievements (7-7 vs Z this season) and Movie hasn't been relevant vs Zerg since his OSL title a year ago. Also a number of (forced) retirements knocked out some people who may or may not have remained relevant to zvt like Yarnc, Effort, and Hwasin.

Musoeun

United States4324 Posts

February 03 2011 14:33 GMT

On February 03 2011 17:03 xxpack09 wrote:

Show nested quote +

This is a bit wild, but here's one way to guesstimate. Stork's been playing since 2004. Before Bisu > sAviOr, he was 31-35 PvZ. This was almost exactly 2 years and 6 months. In the next 2 years and 6 months, Stork was 64-40 in PvZ. Partly this is due to his maturing as a player, partly to the expansion of Proleague. Since then, he's not dramatically improved, going 37-22 PvZ in the last year and a half.

Interestingly - and why this is not a good measure - free, who was known for a while as a top-tier PvZ player, has an opposite problem. From his debut in 2005 through Bisu > sAviOr, his winrate was 70% PvZ, 29-12. In the next year and a bit his win-rate fell off dramatically, to a mere 54% and a 23-19 record. He's improved slightly since then - over the last two and a half year's he's gone 47-36 PvZ. His initial PvZ problems did coincide with a definite improvement in PvT and PvP, so he's not a great test case there as it looks like his focus changed.

So really I'm not sure what to say.

Please or register to reply.

Playing With ELO

Completed

Ongoing

Upcoming