|
On October 08 2010 19:22 Klive5ive wrote:Show nested quote +On October 08 2010 18:52 Mearis wrote:
** For example, 4 gate pushes are an easy to execute strategy that is very hard to defend effectively until mid-high diamond. That's why a Protoss ranked 107 on KR won 7 games in a row on Gisado with the same 4gate all-in. Wait what?!
Gisado proves that Protoss is OP:
On August 17 2010 10:06 hwanikani wrote:
Win Streak Rankings (over 10 win streaks):
1. oGsTheStC(T) 22 win streaks (8-11)
2. Hansin(T) 19 win streaks (8-13)
3. oGsTheStC(T) 15 win streaks (8-17)
3. Hyperdub(T) 15 win streaks (8-16)
5. 허준과침제니스(T) 14 win streaks (8-8)
6. oGsTheStC(T) 13 win streaks (8-1)
7. MioWeRRa(T) 11 win streaks (7-30)
7. LenaPark(P) 11 win streaks (8-8)
7. oGsZenio(Z) 11 win streaks (8-10)
|
Given the MMR it is indeed bullshit to infer anything form the win stats.
If there is an imbalance eg a 1850 true rating zerg couuld be at ~1775 and winning 50% vs a 1775 terran when he should be winning 55%.
On top of that you have the racial distribution imbalance, if zerg is barely over 20% of the pop then T/P will play zerg less often and be less practised compared to their T/P matchups, as well as taking into account that those zerg players are playing despite the possible balance issues - ie a more hardcore playerbase for that race.
Pulling winrates off ladder - and using them to justify anything except to demonstrate there are games played is pretty pointless.
|
I guess looking for win% in mirror can be a good estimate of a player skill, for example.
Second they expect people to believe them without explaining the mathematics they use this one is obvious, those maths I think includes a lot of variables, a lot of complicated algorythm, they just can't release it saying "Voilà !! Now just take a 10 year statistic course to understand this post."
|
Wasn't it Mark Twain who said, "there are three types of lies: lies, damned lies, and statistics."?
Tangential jab aside, I agree with the OP about the odd use of win % by Blizzard and claiming "accounting for player skill." Since player skill can only be assessed via their performance, comparing players of equivalent MMR on the basis of win % still seems flawed to me. If we presume that the game is racially imbalanced, then the hidden MMR is inherently flawed, because the match making system works to "force" 50% win ratios. The more balanced the game, the more valid/reliable the MMR metric is.
Therefore, if Blizzard is looking at the MMR to determine whether or not racial imbalances exist....the MMR is only valid if imbalances DO NOT exist. Am I wrong here? I could be grossly misunderstanding how this metric works.
|
To OP: excellent post, finally some real analysis that brings up the hard questions, and not some political bullshit.
|
Has it occurred to anyone that if Terran is indeed so OP at diamond levels of play that TvZ is actually "55% T favoured" between "Equal true skill players"...
How do you account for the higher PvZ W/L ratio? Applying the same reasoning that the OP did, does that make it what, 60% P favoured, then?
I think it's a far safer stretch to assume that TvZ is actually balanced at diamond level of play (Not high diamond, mind you - just average diamond), or close to it?
And don't forget that the reason for half the patch changes so far (Reaper speed at factory, supply depot before barracks) is because of team games. I wish there was a :vomit: smiley.
|
On October 09 2010 02:42 Nightfall.589 wrote:
And don't forget that the reason for half the patch changes so far (Reaper speed at factory, supply depot before barracks) is because of team games. I wish there was a :vomit: smiley.
is that official ?
|
One point that I think can be agreed upon is that this is a measure of the strength of strategies as much as anything. It is not controversial that Protoss has some very strong, relatively easily executed strategies- cannon rushing, 4 gate and fast void rays- that can enable it to excel at most of the lower levels. Similar strategies like 6 pool our weak and even reaper/hellions are hit and miss.
I think Blizzard is cognizant of this somewhat as they have not yet announced any Protoss nerfs. 4 gate has already been nerfed. In any event, it would behoove them to consider the effect of these strategies.
In a sense, it's difficult to make a balancing decision based on these numbers because you cannot directly infer a specific reason for the variation.
|
On October 09 2010 02:43 MrCon wrote:Show nested quote +On October 09 2010 02:42 Nightfall.589 wrote:
And don't forget that the reason for half the patch changes so far (Reaper speed at factory, supply depot before barracks) is because of team games. I wish there was a :vomit: smiley.
is that official ?
The reaper speed upgrade will require the Factory, which is meant to weaken a lot of the early terran reaper attacks that dominate so many matches, especially in team games.
And I can't see the barracks change being aimed at anything other then fast reaper. 9/8/7rax reaper in particular - which is much more effective when combined with a 10/8/6 pool or 10/10/10 proxy gate in 2v2 than it is in 1v1.
|
Thanx, missed that quote (from a blue post, right ?)
|
On October 09 2010 02:01 Cloak wrote:Show nested quote +On October 08 2010 19:22 Klive5ive wrote:On October 08 2010 18:52 Mearis wrote:
** For example, 4 gate pushes are an easy to execute strategy that is very hard to defend effectively until mid-high diamond. That's why a Protoss ranked 107 on KR won 7 games in a row on Gisado with the same 4gate all-in. Wait what?! Gisado proves that Protoss is OP: Show nested quote +On August 17 2010 10:06 hwanikani wrote:
Win Streak Rankings (over 10 win streaks):
1. oGsTheStC(T) 22 win streaks (8-11)
2. Hansin(T)19 win streaks (8-13)
3. oGsTheStC(T) 15 win streaks (8-17)
3. Hyperdub(T) 15 win streaks (8-16)
5. 허준과침제니스(T) 14 win streaks (8-8)
6. oGsTheStC(T) 13 win streaks (8-1)
7. MioWeRRa(T) 11 win streaks (7-30)
7. LenaPark(P) 11 win streaks (8-8)
7. oGsZenio(Z) 11 win streaks (8-10)
Lol those statistics as you call them are innacurate, Cause StC is a godlike entity, held back by the military. He stoped that 4gate toss, and crushes the very streakes of any1 who gets in his way! Even on the day of the Zerg(FruitDealer) Hero's glory, StC abused him on this stream 2-0. His games are a work of art, marine spreads and unit control so well its almost trademark.
Edit: Misssing one letter ftl
|
APM is not that bad a skill measure. If it is possible to apply cheese or one-build-all-in with APM 25 to APM 80 opponent frequemtly, there is something wrong with balance. However i think this mostly applies to APM <100. In the upper range 150..300 i think skill does not correlate that linear with skill
|
I'm going to write a simulation program tonight to Prove that the numbers they gave mean absolutely nothing. I might end up proving them right, and if i do, i'll let you know.
|
Kent Brockman: Mr. Simpson, how do you respond to the charges that petty vandalism such as graffiti is down eighty percent, while heavy sack beatings are up a shocking nine hundred percent?
Homer Simpson: Aw, you can come up with statistics to prove anything, Kent. Forty percent of all people know that.
xD
I think there is no point in loosing your mind much over it, i just a statistics, and they have probed to be wrong in the past many many times (not only for blizzard, but in many many areas)
|
Possible solution:
Determine skill based on win percentage against random opponents in mirror matches only.
Then, with this skill "descriptor," look how players of equal skill fared against eachother when their race was different.
|
United States12274 Posts
On October 08 2010 18:52 Mearis wrote:This is a topic that's been buzzing around my head for a while, so I wanted to get some discussion going on this topic. SC2 Ladder Analysis: What YOU Need to KnowSC2 Ladder Analysis: Part 2So far, so good - it is a lot more complicated than it has to be, but there is a reason for it. Blizzard didn't want to force players to be confronted with the fact that they are most likely terrible at this game, so instead of letting people know that they are player 973,412 out of 2 milion, they created leagues and divisions. According to Blizzard then, two players with the same rating have an equal chance of winning a game - that's the definition of rating. Here comes the tricky part. Blizzard recently posted some statistics about relative win rates within divisions, and used those numbers to draw some conclusions about class balance. Due to their skill-rating system, and due to the match-making system that aims at placing similarly rated players against each other, on average, the win % should always be 50%, and any deviations from that are due to statistical noise. Note, however, that so far we didn't discuss skill at all. For example, assuming that zerg was underpowered against terran prior to 1.1, the average zerg versus terran match up would still have a 50% relative win rate, simply because the average zerg at a rating of X would be more skilled than the average terran at the same rating. The effect of imbalance would be incredibly difficult to notice through match win %, however, what you should see, is a shifting of the curve at the very high end, where the top tier terrans should have a very high win % against the top tier zerg - which, incidentally is what you saw examining the top 100 or 200 players. Someone made this argument to Blizzard, and a blue poster replied with: Show nested quote +
"The system does account for this to an extent. After analyzing thousands of games and compiling trends based on player skill and race balance, our metrics will attempt to predict who the winner in a given match should be based on player skill and what it thinks is the favored race. These calculations will "correct" themselves over time as trends change.
We do welcome feedback about the announced changes, but there are a considerable number of people outright disregarding the win/loss percentages we provided. Saying we conjured up these numbers to support our own ends, or that our numbers must be incredibly flawed, doesn't really add much to the discussion. The math was developed to disregard the way in which the matchmaking system attempts to keep players as close to a 50% win/loss ratio as possible. As stated in the blog, it's certainly not the only way we analyze balance, but it's very useful to us.
This, to me, seems very hard to understand. Player skill and race balance are not something that can be cleaved from each other, especially if you judge player skill by performance. You can get some insight into player skill and race balance but it requires a lot of assumptions - for example, assuming that the skill distribution is evenly spread throughout the races (*) and the match-up imbalances are constant at all levels of skill (**) then you can end up getting some information about race imbalances by calculating the shift in the rating gaussian from an expected distribution, but you can only really work with the tails of the distribution, and the real skill distribution in a game like SC2 is something that has a very very long tail. Basically, as long as you are working with bayesian inference, you are limited in what you can do by what assumptions you make about the prioris, and it seems to me that a whole set of assumptions are incredibly unsafe to make. Are there any experts on bayesian inference that would know an analytical way of calculating the expected shift? I was playing around with some paper & pencil and the math gets pretty brutal rather quickly, I think it might be easire to do some kind of simulation with a high number of agents and measure the shift. Even if you did measure the shift though, I am not sure how you'd backpropagate the shift to the PSR system or the MMR system. I doubt the shift is uniform. * Terrans are more likely than not to have more newbie players due to the campaign mode, and the fact that terran and protoss have by far the most intuitive and least punishing game mechanics. ** For example, 4 gate pushes are an easy to execute strategy that is very hard to defend effectively until mid-high diamond.
We can't really figure this out either. I suppose one method would be that they separate every race and break it down into race and map matchups by MMR differences, so say if a trend emerged where 2000-MMR Zs were losing regularly to 1800-MMR Ts on Kulas, that would make it an imbalanced map for Z which would contribute to T beating Z X% more than they should be overall because Kulas is played in TvZ Y% of the time. Maybe another possibility is that they create a separate matchup-specific rating, say TvZ only, that sidesteps any effect on rating that could have been a result of being good or poor at TvP and TvT. Each of these are open to problems, though, so I'm not really sure what metric they're using. I've reached out to Blizzard for more information but I'm not expecting much more detail.
EDIT:
On October 09 2010 03:54 mati wrote:
Kent Brockman: Mr. Simpson, how do you respond to the charges that petty vandalism such as graffiti is down eighty percent, while heavy sack beatings are up a shocking nine hundred percent?
Homer Simpson: Aw, you can come up with statistics to prove anything, Kent. Forty percent of all people know that.
It was "forfty" percent :V
|
On October 09 2010 03:54 mati wrote:
Kent Brockman: Mr. Simpson, how do you respond to the charges that petty vandalism such as graffiti is down eighty percent, while heavy sack beatings are up a shocking nine hundred percent?
Homer Simpson: Aw, you can come up with statistics to prove anything, Kent. Forty percent of all people know that.
xD
I think there is no point in loosing your mind much over it, i just a statistics, and they have probed to be wrong in the past many many times (not only for blizzard, but in many many areas)
Well, the problem is that i've studied these algorithms that they use. I have a degree in freaking mathematics which i don't even use for work (I am in computer science now) and I know that unless they've come out with a breakthrough in probability, that the information they posted is absolutely meaningless. It is impossible to separate player skill from performance when you play a single race. There simply is not a way of doing it. The algorithm for calculating rating itself is based on on a probabilities, but the probabilities themselves are based on ratings. It is circular. It has no meaning because the conditions are not the same. If players were forced to play random 100% of the time then yes! But they are not.
I want to define something.
Performance is a measurement of how well you did in a game.
Rating is a measurement of how well you did in a set of games under a single condition.
Skill is a measurement of how well you did in a large set of games under all conditions equally.
Unless blizzard is redefining skill, which they probably are, this simply does not exist.
They could be making wild assumptions like "players with similar ratings are similarly skilled" or "players are equally skilled at all of the different matchups ", but these by definition are not acceptable assumptions to make unless you want to "have something to show your boss".
There is absolutely no way that they could extract this information.
I think i can prove my point simply by stating that they put the percentages in the format 59.3% without including a +/- why? Because there is no such formula to calculate it.
|
On October 09 2010 02:18 MrCon wrote:I guess looking for win% in mirror can be a good estimate of a player skill, for example. Show nested quote +Second they expect people to believe them without explaining the mathematics they use this one is obvious, those maths I think includes a lot of variables, a lot of complicated algorythm, they just can't release it saying "Voilà !! Now just take a 10 year statistic course to understand this post."
The problem is that the matchmaking system makes sure you have a 50% win rate against all races.
|
On October 09 2010 04:19 darmousseh wrote:Show nested quote +On October 09 2010 02:18 MrCon wrote:I guess looking for win% in mirror can be a good estimate of a player skill, for example. Second they expect people to believe them without explaining the mathematics they use this one is obvious, those maths I think includes a lot of variables, a lot of complicated algorythm, they just can't release it saying "Voilà !! Now just take a 10 year statistic course to understand this post." The problem is that the matchmaking system makes sure you have a 50% win rate against all races.
Do you have a source for that? I'm under the impression that it gets you a 50% win against all opponents.
And in that case, if ZvP is 48/52, ZvT is ~50/50, and TvP is ~50/50... The logical conclusion is not that T > Z.
The OP is suggesting the following:
Let's assume that two players, a T with a True Skill of 1800 and a Z with a True Skill of 2000 play.
Those 2 people playing have 50/50 odds. Blizzard thinks the Z player is equally skilled to the T player.
The OP forgets the following:
Let's assume that a T with a True Skill of 1800 and a P with a True Skill of <Less or equal to 1800> play.
Those 2 people also have 50/50 odds. Blizzard thinks that they are equally skilled.
Now, we have the P player who blizzard thinks is equal in skill to the T and Z player, play PvZ.
P wins 52% of the time.
If anything, the imbalance is in PvZ - the True skill of the P player is at most that of the T player, and probably even lower - yet he still has a 52% W/L ratio against a more skilled Zerg.
The only way that the conclusion I made above can be false is if Blizzard considers a T with a True Skill of 1800 to be equivalent to a P player with a True Skill of <More then 1800>. With PvT and PvZ both being 50.5% and 52% win rates, I can't see that happening.
|
On October 08 2010 18:59 haffy wrote:
I'm just going to repost one of my replys from another thread because it kind of fits here too. I just think people are going to have to accept Terran may actually be harder to learn to take advantage of and stop with the ignorant, oh yeah all Terran are just new players from campaign. I bet almost everyone just picked their favourite looking race, I know I did.
this man speaks truth.
Blizzard should not pull numbers for race balance across all the ladders, idc if protoss had a 99% win rate in bronze, bronze ppl dont know how to play so it makes since that simply attacking would win you a lot of games.
What is with people finding crazy ideas as to what these statistics mean. I'm really supsrised no one went for the obvious explanation, it just kind of doesn't make sense untill you think about it abit. This is the way I'm reading the statistics, my opinion is actually differant as I think Terran are easier to play. Also I'm assuming the sample size is big and it's not biased in anyway because that changes everything.
I'm only going to do TvP because it'll just get too complicated for a simple concept even if it doesn't feel right.
The simple answer is Protoss is easier to play at equal skill levels when the skill level is low. But when the skill level evens out to a certain degree at higher levels apparently, Terran start becoming over powered over Protoss. Now think about how close the stats are in PvT at dimond, but how many Terran win tornmanets? Well it makes sense if Terran are worse for less skillful players to play and more of an advantage for more skillfull players to play. This advantage should be widening the higher up you go making Terran easier for very top level play. Also with the statistics if you go into the top percentile it should show more Terran in the top 100 or what ever and if you look at the ranks it does.
Look at it this way too. Terran in my opinion have an advantage early game against Protoss and Protoss have the advantage late game.Lower level players don't understand the importance of early pressure so less of them will do it or if they do not as effective, which could be one of the factors that make Protoss easier to play for lower levels.
|
|
|
|
|
|
EPT
