|
Katowice25012 Posts
With no class to attend I've had a lot of free time recently. A typical way of dealing might be to catch up on sleep, spend time with family, or find a new hobby. Maybe use creative energy towards making a masterpiece. I've mostly been bored.
Deciding to use the time wisely, I opted to start 2011 off right and make some graphs, you know, play with some numbers. I've barely loaded up excel since SC2's release, and the last time I wrote too many words about a simple issue was in beta which is a damn shame.
This time its about positions in maps. At one point I considered piecing together a grand scheme involving length of game in relation to the maps and location, ideally to supply the map-hate brigade with ammunition but then Gom announced new maps and Odoakar timed games. Instead this will strip extraneous info and look at spawn locations.
The maps looked at were Lost Temple and Metalopolis, both 4 player maps with similar layouts and have been in GSL forever. We're looking at GSL only, from the start of the Open Season 1 until Jan 11, an arbitrary cutoff date set by when I last felt like updating the pictures.
![[image loading]](http://i.imgur.com/7rlr9.jpg) I had trouble visualizing my own description below so I made this professional looking diagram
The idea is easy: record where people spawned and see which won the most. To make it more fun I also categorized the distance between the two main base locations. Cross map (12-6 and 9-3 which split the map), close by air (12-3 and 9-6, the two mains are next to eachother but not through land) and close by walking (6-3 and 12-9, universally disliked by zerg). The numbers following are LT and Metalopolis combined. While the two have their own defining features that change game outcomes (Metal has wide open nats, cliffs on LT etc) they are a nice close enough here.
First look at this pretty graphic of which map quartile wins most often.
![[image loading]](http://i.imgur.com/mVmYY.png) Clock positions and their winrates for Metal and LT put together.
Probably nothing significant going on here, but the way it shakes up is pretty funny. 6 o'clock wins a solid 15% less than other locations. The split was pretty even between Metal and LT so it doesn't look like any specific feature, maybe it is mysteriously harder to play (conspiracy). Looking further is more interesting though.
The LT style architecture has a neat property in that a game will play out vastly different depending on starting location. This is true to some extent for all 4 player maps, but the effect is considerably more subtle on those with corner bases (Delta, Shakuras) where it might not be relevant until well into midgame.
To see how it effects each race this is split into each matchup and by distance. The P T columns show the PvT record, Protoss won 7 games to Terran's 6 when the players spawned cross map (12-6 or 9-3).
![[image loading]](http://i.imgur.com/JHMCv.png) Walk is 12-9 & 6-3, Air is 12-3 & 9-6, Cross is 12-6 & 9-3
When Idra spawned at 6 to MVP's 3 on Metalopolis to decide his Open Season 3 fate, Zelniq immediately typed "gg Idra nice try". He was right, MVP ran at him with SCVs and marines for a quick submission, an unsurprising result looking at past performance.
A few intersting things are going on here. It points towards confirming a lot of the greivances our community has, Terran wins a lot when the spawns are 12-9 or 6-3. PvT has trouble with shorter distances, but does okay cross map. Presumably this has to do with the past consensus that Protoss is weaker early game (maybe outdated after MC's win). ZvT changes drastically based on size between bases.
![[image loading]](http://i.imgur.com/n4O0o.png) I, too, dream in color and in rhyme.
Same data, but combines each race's matchups (mirrors need not apply). Shows the same basic ideas, in a way that is more convinient.
Notice how skewed the walk distance numbers are, Terran at nearly three quarters and Zerg a hair below a third. Then that winrate shrinks rapidly, nearly halving in cross map situations while Zerg's does the opposite. Overall, this supports the 'small map bad, big map good' thought process that has been particularly popular in the last week.
For me, its particularly satisfying to see opinion/intuition reflected in hard data this way. However, keep in mind that this spans the entire GSL lifetime, in addition to being a limited dataset. The PvT numbers specifically are least reliable, it seems like no one truly knew how to play that matchup until MC took a crown.
At Blizzcon they claimed to not keep position statistics internally, feeling it was slicing the data too thin to be useful. For strict balance concerns this is probably true. These charts suggest that maybe its not a totally useless metric, however, it lends to credability to the idea that the maps themselves need balancing or stronger playtesting.
At the very least its a fresh way to look at some commonly discussed issues. Go forth, TL denziens, be inspired to find new ways to look at common talking points.
Edit: Addendum
I made pie graphs that show what percentage each race's position was when winning. This is an aggregate total, I tallied up the wins then divided them up by what the spawn relationship was.
A nice way to show the data, most of Terran's wins come from close spawns which isn't surprising. The Protoss air portion is something I didn't see when doing the charts earlier though and is kind of fun!
|
I feel like these kind of GSL statistics are just kind of useless because the sample size is tiny. The game is new and the players are crap, this isn't enough to base any meaningful conclusions on.
|
Wow, great work sir!
Nice to see numbers give a basement to feelings, and gives some interesting perspective on what to come once GSL changes mappool maybe?
I guess tendencies get caught but nothing extremely worrying
+ Show Spoiler +Why can't every OP look like this? 
|
Canada5565 Posts
On January 13 2011 23:27 ChickenLips wrote:
I feel like these kind of GSL statistics are just kind of useless because the sample size is tiny. The game is new and the players are crap, this isn't enough to base any meaningful conclusions on.
75% win rate for Terran in close positions is pretty telling.
|
On January 13 2011 23:27 ChickenLips wrote:
I feel like these kind of GSL statistics are just kind of useless because the sample size is tiny. The game is new and the players are crap, this isn't enough to base any meaningful conclusions on.
You'd be surprised, but if you have data that is conclusive enough, you can conduct statistical analysis that will lead to an answer (ie. "Positions have no influence has a likelihood lower than 5%") with ~30 items.
Now, we haven't really reached that amount of data yet, but many games are playedat GSL, so this will sort out quite quickly.
|
Looks like most Terrans will be pretty screwed when GSL brings in some of the huge maps. I think everyone already had a hunch about that.
It's hard to say though, since there are a ton of Terrans in the GSL that, when compared to the very good ones such as Jinro, MVP, Ensnare, Clide, etc. are pretty bad and seem resigned to all-in every game, which short walk distance lends itself to. Not sure how the good Terrans will do on the super huge maps that are being looked at.
|
Yeah, I'm definitely starting to feel like they need to change the ladder map pool before they balance the game too specifically for shorter maps. I mean, Terran's win rate is low in far positions to compensate for it being so high in close positions, but if the maps all become large then Terran is gonna be way underpowered.
|
On January 13 2011 23:29 Xxio wrote:Show nested quote +On January 13 2011 23:27 ChickenLips wrote:
I feel like these kind of GSL statistics are just kind of useless because the sample size is tiny. The game is new and the players are crap, this isn't enough to base any meaningful conclusions on. 75% win rate for Terran in close positions is pretty telling.
They two rax a lot in the GSL. The close positions should help to pull that off. Might be why that percentage is so high.
|
And Blizzard said they werent concerned with the map pool at all...
|
Well statistics shows that in close air or cross positions the win/loss ration is pretty even but in close ground is really bad. That means that match ups would be much more fair if it would be impossible to spawn by close positions on LT and Meta, just like on shakuras. But well blizzard even took out shakuras lol.
|
Canada5565 Posts
On January 13 2011 23:36 Klelith wrote:Show nested quote +On January 13 2011 23:29 Xxio wrote:On January 13 2011 23:27 ChickenLips wrote:
I feel like these kind of GSL statistics are just kind of useless because the sample size is tiny. The game is new and the players are crap, this isn't enough to base any meaningful conclusions on. 75% win rate for Terran in close positions is pretty telling. They two rax a lot in the GSL. The close positions should help to pull that off. Might be why that percentage is so high.
Yea, exactly...
|
On January 13 2011 23:34 TestSubject893 wrote:
Yeah, I'm definitely starting to feel like they need to change the ladder map pool before they balance the game too specifically for shorter maps. I mean, Terran's win rate is low in far positions to compensate for it being so high in close positions, but if the maps all become large then Terran is gonna be way underpowered.
I disagree that Terran is underpowered in large maps. The reason Terran wins so much in close positions is because of how they're played right now. Right now, all the "good" players are doing super aggressive all in strategies and those do not work well in cross positions and large distances. The solution is not about balancing the races. It is learning to play differently and developing new strategies. Really good Terran players are fine on large maps and in cross positions because they know how to play with that play-style successfully.
TL;DR Large maps = different strategies.
|
Katowice25012 Posts
I imagine Metal would be a lot better if it spawned only cross map. It would be cool to add in game length stats to the position stuff, seems likely that the games with cross map would be a lot longer (and probably qualitatively better for spectating).
|
Interesting, im definitly not a fan of statistics interpretation, because I find that analysing the games themselves is far better. But well, it confirms some impressions we always had.
|
GSL's small sample size and vast range of skill (we see this disparity between code A and code S games) gives this data less merit, but I think the public's consensus is correct. Most people want larger maps because of zerg's mechanical advantage in those situations, and it leads to less all-in strategies. I'm sure Blizzard has the data for positions and maps, but chooses not to look at it for balancing (why should they, the forums will QQ enough for them to get the picture).
Hopefully bigger maps make it into the ladder pool soon, or we will see a lot more complaining and evidence being dug up about positional imbalance on maps. Blizzard can only turn a blind eye to this for so long.
Edit: Additionally, it may be that many of the past balances (Bunker time, Tank Damage, no Flux Veins, Roach range, etc.) were not really necessary in longer, cross-position situations. Positions actually affect the effectiveness of the individual units drastically, as most players understand. A 7 Roach Rush is drastically less effective the longer it takes to get the 7 Roaches to the opponent's base, so in a sense, the roach is less effective the longer the spawning positions when using that strategy. A buff like +1 roach range makes that strategy even MORE effective with close positions, because of the timing and the resources necessary to defend it. Of course, the roach buff is still in effect all game long, but I think slight variations in unit stats will make positions in some MUs much more volatile. When you spawn close to your opponent, it's a completely different game with a completely different set of viable strategies.
|
Props for the statistics. This only says we really need more tech&expo friendly maps to understand the balance better.
All the 2player maps in the pool are so small... I find myself in starvation mode a lot more often than in BW.
|
i loooove statistics like that, thanks for all the effort you put into that. good job!
|
Katowice25012 Posts
I added a new graph at the bottom. Its pie! yay!
When reading anything about numbers, keep in mind that the numbers themselves never prove anything one way or another and they're not meant to. It is one more piece of information to use when looking at an issue, weigh it appropriately as you would when judging something a commentator says during the game. People are good at finding the bias in the data, but bad at using the data itself to help form an opinion based on a variety of other sources at the same time (players opinion, personal experience). They're never entirely useless in the same way they can never be used to base a definitive conclusion on.
When you spawn close to your opponent, it's a completely different game with a completely different set of viable strategies.
It is, which makes it really cool that matchup numbers show some difference when looking at spawns. I've had a good laugh at a few GSL games since making those graphs though, its fun to watch live with knowledge of what past performance looked like.
|
On January 13 2011 23:27 ChickenLips wrote:
I feel like these kind of GSL statistics are just kind of useless because the sample size is tiny. The game is new and the players are crap, this isn't enough to base any meaningful conclusions on.
It's very telling
|
Why not do the same on Master's league ladder stats. The sample size here is too small for me to care, zvt anecdotes aside.
|
On January 13 2011 23:29 Xxio wrote:Show nested quote +On January 13 2011 23:27 ChickenLips wrote:
I feel like these kind of GSL statistics are just kind of useless because the sample size is tiny. The game is new and the players are crap, this isn't enough to base any meaningful conclusions on. 75% win rate for Terran in close positions is pretty telling.
People who are pulling "statistics knowledge" out of nowhere, even though there arne't that many items we can tell with certainty that there is corellation, just due to the 75 percent.
|
A sample of 10 is too small, but the samples that the TC was posting are in the 70-80s, which will start to have accurate representations.
|
Vancouver14381 Posts
Hmm, I do like the idea of making the games longer by only generating cross positions, but then a lot of the 4-player maps will become 2-player maps and scouting becomes less important early in the game. Maybe that can be a good thing too, since it decreases the chances of a player losing due to scouting the opponent's base last on Meta and LT.
|
I think 4 player maps should a) not be like DQ with awkward "symetry", b) should prevent spawning in closest locations by land. Spawning close by air is fine imo if walk distance is long.
Btw i really like Shakuras plateau because of it being big enough to actually play on how it is supposed to be played (zerg reactions etc.) and i think it is one and only completly balanced map so far. This might lead to a thought that 2v2 maps should maybe be used in 1v1 matches sometimes, maybe with some minor changes.
|
Excellent work!
I think part of the terran close dominance is the prevalence of 2 rax play in the GSL.
|
Iceland4053 Posts
On January 16 2011 11:42 Sanguinarius wrote:
Excellent work!
I think part of the terran close dominance is the prevalence of 2 rax play in the GSL.
Tbh what exactly is wrong with 2rax even cross position? It doesn't delay the cc that much and you can be pretty safe FEing. The only thing you loose is tech, where rine is pretty good very far into the game anyway it doesn't really matter for defending purpose, but you loose the chance to harras via hellions/banshees, which kinda defeats the "terran are playing wrong, they just need to macro more"-view.
Question is, is the random roll(if it were in a bit lesser degree) a bad thing? I mean that's for blizzard to decide, but it keeps the games a bit more... diverse. Obviously such huge differences are bad, but if it was say +/- 10%, would that be a bad thing?
If it isn't, how to combat it? Zerg ofcourse say make all spots cross position, toss lands somewhere in middle(long for PvT, short for PvZ) and terran close. I'd guess we're looking to approach the 50% mark a bit more, so cross seems to be to far and close to short.
So if blizzard feels the random roll is bad, I'd say either they needed to make 3player maps or limit spawning location to correct distance.
|
will they use these maps next week?
|
What i find most interesting is the lower win rate on the 6 o clock. The other statistics are probably explained by strategies that are strong for certain races on certain positions, so the win rate is determined by race factors which are triggered by relative starting positions. I think its called 'mediation' in statistics, or 'moderation'.., its been a while 
But a starting position which has a lower win rate across the board just shouldn't exist. The winrate of a starting position is (or should be) independent from relative positions (air/walk/cross) and/or race. Maybe its the shape of the starting base, or perhaps distance to gas/minerals. Or maybe the numbers are just insignificant due to a small'ish sample size. If there is someone here with expertise on statistics he/she could prove or disprove the significance of the numbers found.
|
On January 13 2011 23:55 heyoka wrote:
I imagine Metal would be a lot better if it spawned only cross map. It would be cool to add in game length stats to the position stuff, seems likely that the games with cross map would be a lot longer (and probably qualitatively better for spectating).
I've just finished collecting game length data for all games played in GSL 2, GSL 3 and GSL 4, and this seems like a good next step to take. I might go through all the games again quickly, and just note whether the spawn positions are WALK, AIR or CROSS and then do some nice pivot tables to represent the data
|
On January 16 2011 22:30 Elp wrote:What i find most interesting is the lower win rate on the 6 o clock. The other statistics are probably explained by strategies that are strong for certain races on certain positions, so the win rate is determined by race factors which are triggered by relative starting positions. I think its called 'mediation' in statistics, or 'moderation'.., its been a while
i'd call it explanation. 
|
people are pretty lame here. its not about sample size but its about confidence intervals. if you have a high percentage of something happening you don't need a huge sample size.
in this instance you might need slightly a few more samples to get a good p value.
however, just because something isn't within a confidence interval doesn't mean that the phenomenon isn't happening.
|
Sample size is reasonable (p<<1). Calling player skill a factor is by definition subjective. The title itself limits the data to [GSL] (not ladders).
"David" Kim of Blizz stated publicly on the first day of the Jan 2011 [GSL] that Blizz intended "balance" to relate to the "highest level of competition." If there was any doubt about his meaning, note that he was standing on the set at the competition. Blizz sent him there to deliver that message (my interpretation - not a fact).
I can't see anyway this work by the OP could be more on point. Excellent meta-analysis. I hope you keep it up and you might think about doing this sort of thing for a living.
The real question this points to is: What role, if any, does play other than international competition have in "David's" "balance scheme?" Ladder players could use as much support from Blizz on this as [GSL] players. Work such as the OP may get Blizz to help us there with some ladder data.
Note: I am taking no position regarding the definition of "balance." I am a 92 rank Bronze. I don't care what Blizz does. I just want to know the landscape.
|
I did some chi-squared tests and given the small sample size the results aren't significantly different from random -- but that doesn't mean that these results aren't meaningful. They coincide with anectodal impressions people have from logging thousands of games and also make rational sense. While it's still hard to know what Terran's 'actual' win probability is when spawning at close-by-ground positions, it's pretty crazy that they are winning 3/4 of their games in the GSL in that situation.
|
those stats are pretty much expected
Hoping this will be change once the new maps are implemented
|
I need an advice - should I only check stats for Lost Temple and Metalopolis since they have similar layout, or should I try to include DQ and Shakuras, even though I'm not sure one can break down the spawn points to fit the cross, air, walk scenario:/
|
Katowice25012 Posts
I looked at Shakuras when collecting the numbers and it wasn't very interesting, since it can only spawn cross or opposite side you end up looking at "is right or left side better". The game doesn't play out all that differently depending on spawns either, sometimes you get wacky shenanigans with breaking the rocks and doing backdoor stuff but even that has become less common. DQ has potential but the dynamic is very subtle, it effects things like where your third is and if you break the rocks and take that base or the front natural. By comparison, LT/metal it can directly change your build order (ie you're basically never opening void rays if you don't scout them with close flying distance).
For both of those maps it might be worth looking at, but the effects don't show in as many games so you would need a more clever way to slice it than what I did here.
|
The thing I want is maps where the distance does not matter that much in the early game. Different positions should make different games, but not that early as it is now, where you cant know anything before you will scout and then it is to late.
|
|
|
Thanks for doing this. I have high hopes that Blizzard will release bigger maps one of these days :o
|
|
|
|
|
|
EPT![[image loading]](http://i.imgur.com/2GG5S.png)
![[image loading]](http://i.imgur.com/Z6VI4.png)
