|
On July 11 2012 03:29 Mendelfist wrote:Show nested quote +On July 11 2012 03:17 Shiori wrote:On July 11 2012 03:16 Mendelfist wrote:On July 11 2012 03:12 Shiori wrote:On July 11 2012 03:09 Mendelfist wrote:On July 11 2012 02:58 skeldark wrote:On July 11 2012 02:51 sevencck wrote:On July 11 2012 02:40 Mendelfist wrote:
I don't follow you here. The point is that new players choose terran because of the campaign, and some of them later in their career switch race. The switch is important, because that WILL cause an over-representation of terrans at lower MMR. It has been known for a long time, for example by looking at sc2ranks statistics. There is no easy way to remove these types of biases from any data that we have. We also don't know how far up the leagues this bias persists. Is it only in bronze/silver? I have no idea. The bias you're addressing is (presumably) independent of skill. The over representation of Terran at lower MMR has no bearing on an analysis correlating skill by race with MMR. He's not taking a snapshot of race at all skill levels and saying the Terran average MMR is lower (that would be easy to do, and wouldn't prove anything since the ladder is biased for 50% anyway), he's analyzing winrates between races but factoring in the "hidden" MMR rating. In other words, a 50% TvZ winrate may at face value appear balanced, but if the average MMR of the Terrans in that sample is statistically significantly higher than that of the Zergs, it suggests imbalance. OP please correct me if I'm mistaken. No. thats basic it. Mendelfist i dont throw this coins. What i do is: take the avg weight of the 70% head coins and the avg weight of the the 30% tail coins . And than i say : the coins that show head are 10g heavier than the coins that show tail. I don't care how many of each are on the table. No, but the ladder throws coins. I don't understand how you can't see this. Imagine that when you buy the game you don't even have a choice. You are forced to choose terran. Then when you have played for a month the other choices open up. It's pretty clear then that low MMR's will be over-represented by terrans, right? Except there's absolutely no evidence that this is occurring. Do you really mean that I need evidence for that new players who just bought the game often choose terran as their first race? You need evidence for new players who just bought the game and choose Terran as their race only to switch a few months into playing and contrast it with evidence showing that players who pick P/Z after buying switch less or not at all. There is no need for P/Z switching less for this phenomenon to occur. Try the experiment with the coins. If there is an initial bias for whatever reason, and people then randomly switch race for whatever reason, it's unlikely that the switches will preserve the initial bias. This will cause a change of race distribution over time. That's all it takes.
Which only matters if some of the coins are more valuable than others and only the more valuable ones are showing up, so to speak.
|
United States4991 Posts
It's interesting data, but is it possible to get the same results of people over (say) 2200 MMR? That's a bit above the master league cutoff IIRC (would exclude "diamond / low master"), and it would probably help address a lot of the complaints that lower leagues may be biasing it / are irrelevant to the question of balance.
|
On July 11 2012 03:33 Shiori wrote:Show nested quote +On July 11 2012 03:29 Mendelfist wrote:On July 11 2012 03:17 Shiori wrote:On July 11 2012 03:16 Mendelfist wrote:On July 11 2012 03:12 Shiori wrote:On July 11 2012 03:09 Mendelfist wrote:On July 11 2012 02:58 skeldark wrote:On July 11 2012 02:51 sevencck wrote:On July 11 2012 02:40 Mendelfist wrote:
I don't follow you here. The point is that new players choose terran because of the campaign, and some of them later in their career switch race. The switch is important, because that WILL cause an over-representation of terrans at lower MMR. It has been known for a long time, for example by looking at sc2ranks statistics. There is no easy way to remove these types of biases from any data that we have. We also don't know how far up the leagues this bias persists. Is it only in bronze/silver? I have no idea. The bias you're addressing is (presumably) independent of skill. The over representation of Terran at lower MMR has no bearing on an analysis correlating skill by race with MMR. He's not taking a snapshot of race at all skill levels and saying the Terran average MMR is lower (that would be easy to do, and wouldn't prove anything since the ladder is biased for 50% anyway), he's analyzing winrates between races but factoring in the "hidden" MMR rating. In other words, a 50% TvZ winrate may at face value appear balanced, but if the average MMR of the Terrans in that sample is statistically significantly higher than that of the Zergs, it suggests imbalance. OP please correct me if I'm mistaken. No. thats basic it. Mendelfist i dont throw this coins. What i do is: take the avg weight of the 70% head coins and the avg weight of the the 30% tail coins . And than i say : the coins that show head are 10g heavier than the coins that show tail. I don't care how many of each are on the table. No, but the ladder throws coins. I don't understand how you can't see this. Imagine that when you buy the game you don't even have a choice. You are forced to choose terran. Then when you have played for a month the other choices open up. It's pretty clear then that low MMR's will be over-represented by terrans, right? Except there's absolutely no evidence that this is occurring. Do you really mean that I need evidence for that new players who just bought the game often choose terran as their first race? You need evidence for new players who just bought the game and choose Terran as their race only to switch a few months into playing and contrast it with evidence showing that players who pick P/Z after buying switch less or not at all. There is no need for P/Z switching less for this phenomenon to occur. Try the experiment with the coins. If there is an initial bias for whatever reason, and people then randomly switch race for whatever reason, it's unlikely that the switches will preserve the initial bias. This will cause a change of race distribution over time. That's all it takes. Which only matters if some of the coins are more valuable than others and only the more valuable ones are showing up, so to speak.
Er, what? Now you lost me again. The point is that it's impossible to tell the change of race distribution due to random race switching from the change of race distribution due to some races having it easier to move up the ladder.
|
I am not a statistics guru but...
I think the question that really needs to be answered with this data, is as follows:
Does Race X on average win against Race Y even when Race Y has a higher MMR.
In other words the Null Hypothesis is that Race does not matter. A higher skilled player should be a lower skilled player regardless of the race.
This would show that certain races win over others even if the other player is more skilled. I don't think a flat average or races really is going to say anything.
|
On July 11 2012 03:41 Smancer wrote:
I am not a statistics guru but...
I think the question that really needs to be answered with this data, is as follows:
Does Race X on average win against Race Y even when Race Y has a higher MMR.
In other words the Null Hypothesis is that Race does not matter. A higher skilled player should be a lower skilled player regardless of the race.
This would show that certain races win over others even if the other player is more skilled. I don't think a flat average or races really is going to say anything.
Better skilled is the key word. Who is better skilled? the guy with higher mmr.?
The point is the imbalance is already in the mmr. Because you play a strong race you have more mmr than your (real skill) but we dont know what your realskill is, all we know is your mmr. A method to detect your realskill in a balanced game!
So i detect imbalanced by searching for difference of the avg race mmr by assuming that all races should have equal skill.
Some people say this assumption is wrong, but if its wrong the game is per defaulted imbalanced over the playerbase.
Also there is no objective point that disagree with it.
an example:
You are race X underpowered. Your ingameskill ( mmr) is under your "realskill" because of this.
You face now a player of race Y that is overpowered = you both have the same ingame skill = MMR and trade win/looses.
Hes realskill is under your realskill but the imbalance let you both have the same ingameskill ( mmr) .
You would not detect the problem with your method. The system will always show you a perfect world.
There is one other method that you can use to show trends:
you look at the change of mmr of an race over time!
Do players of race Z loose mmr? do players of race X win mmr? this will happen after an patch. But perhaps its not imbalance perhaps it correct the imbalance that was there from the beginning.
The method i use is the only one that can give you an balance indicator.
People also look on tournement results. But tournement brackets are just another MMR system.
The winnner playes vs the winner = the better players are matched against each other.
In an tournament = everybody start by 0 mmr and everytime you advance you raise in "mmr".
So looking at races placements in tournements is exactly what i do here on ladder. The problem is there are way more laddergames than tournament games and the ladder mmr system is way more accurate than the tournament (min-max) system.
|
On July 11 2012 03:41 Smancer wrote:
I am not a statistics guru but...
I think the question that really needs to be answered with this data, is as follows:
Does Race X on average win against Race Y even when Race Y has a higher MMR.
In other words the Null Hypothesis is that Race does not matter. A higher skilled player should be a lower skilled player regardless of the race.
This would show that certain races win over others even if the other player is more skilled. I don't think a flat average or races really is going to say anything.
I think that would mainly show the instability in a race's play. Not the "imbalance", because the player with the higher mmr is already benefiting from the imbalance of his race.
|
I'm not sure what you are trying to show me. There is a clear over-representation of terrans in bronze and silver compared to the other leagues.
|
I find it sad that a community member must do these calculations and post them. Then Dustin B. just says in an interview that everything in every ladder and server is 50-50 and in winrates in every matchup early game late game what ever still 50-50. Then he says they are monitoring a situation where last month there was a 0,5% imbalance. And everything without zero facts.
I hope you get more data (masters+). Presenting clear facts based on data is never wrong. Everybody can then make up their own mind about what's the cause and balance and so on. Keep doing what you're doing.
|
It's impressive work, however there are some flaws with it.
Your calculation doesn't take practice time into account. In general the foreign scene plays less time and in a less disciplined way then the koreans. If protoss and zerg can show better results with less time-input than terran, does this mean T is underpowered? In my opinion, no it doesn't (although maybe the race needs some changes), the only balance that really matters is the balance when players of every race are putting as much time and effort into the game as possible i.e. the professional players.
Additonally the metagame follows the pro-scene, and there are hardly any foreign terran community figureheads to lead the scene. I know there a lot of korean terrans, but a lot of foreign players seem to be disinterested or biased against the korean players for whatever reasons (maybe cos gsl is on at odd times for some people).
|
On July 11 2012 04:58 Zrana wrote:
Your calculation doesn't take practice time into account. In general the foreign scene plays less time and in a less disciplined way then the koreans. If protoss and zerg can show better results with less time-input than terran, does this mean T is underpowered? In my opinion, no it doesn't (although maybe the race needs some changes), the only balance that really matters is the balance when players of every race are putting as much time and effort into the game as possible i.e. the professional players.
You nailed it. Worse macro (effective multitasking) due to lower time spent practicing, i.e., consistently showing higher avg min unspent/min shouldn't land players at the same MMR just based on different race choice.
le: completely anecdotical, most of my income/min // unspent/min in tvp is somewhere around 2200/1200 vs 2200/2400, in nailbitingly close games.
|
On July 11 2012 04:44 Mendelfist wrote:I'm not sure what you are trying to show me. There is a clear over-representation of terrans in bronze and silver compared to the other leagues.
I think the point was that this game isn't new any more. There aren't that many new playes being added to the ladder, relative to release day. So a much smaller proportion of this "over-representation" could be made up of the bad/new players. Further, there is no way to say why there are more Terran at Bronze and proceed to drop off as you go up in leagues. At time progresses, the "noob" bias becomes much less significant. It's definitely still there, only OP with leagues breakdowns can tell us.
|
On July 11 2012 05:39 slane04 wrote:Show nested quote +On July 11 2012 04:44 Mendelfist wrote:I'm not sure what you are trying to show me. There is a clear over-representation of terrans in bronze and silver compared to the other leagues. I think the point was that this game isn't new any more. There aren't that many new playes being added to the ladder, relative to release day. So a much smaller proportion of this "over-representation" could be made up of the bad/new players. Further, there is no way to say why there are more Terran at Bronze and proceed to drop off as you go up in leagues. At time progresses, the "noob" bias becomes much less significant. It's definitely still there, only OP with leagues breakdowns can tell us.
you have way better data than league breakdown. you have mmr....
if i go for leagues this data would be so inaccurate i could do nothing with it.
I found a mistake. I put many opponent more than a single time in the file.
This is race independent mistake and should not affect the result.
However i will correct it and run the calculation. Could take several hours until he is done....
|
On July 11 2012 05:43 skeldark wrote:Show nested quote +On July 11 2012 05:39 slane04 wrote:On July 11 2012 04:44 Mendelfist wrote:I'm not sure what you are trying to show me. There is a clear over-representation of terrans in bronze and silver compared to the other leagues. I think the point was that this game isn't new any more. There aren't that many new playes being added to the ladder, relative to release day. So a much smaller proportion of this "over-representation" could be made up of the bad/new players. Further, there is no way to say why there are more Terran at Bronze and proceed to drop off as you go up in leagues. At time progresses, the "noob" bias becomes much less significant. It's definitely still there, only OP with leagues breakdowns can tell us. you have way better data than league breakdown. you have mmr.... if i go for leagues this data would be so inaccurate i could do nothing with it. I found a mistake. I put many opponent more than a single time in the file. This is race independent mistake and should not affect the result. However i will correct it and run the calculation. Could take several hours until he is done....
Yep, sorry that's what I meant. I meant the percentage off MMR's which would place you in Bronze that was included in the basket of MMR's that you used to calculate imbalance.
|
|
|
can look this up in the file.
but i know my datapool so i can already tell you bronze and silver is forgettable small
Its accounts not games and i dont have race data for most of them so i cant use them.
Also like i said i have and mistake and added many accounts double in it,
Correct it at the moment.
the gamefile is in the other thread and includes 100.000 games.
and 0 of this games are from me personal!
And the random runs have nothing to do with the result.
I dont know what you guys are talking about but i you are on the wrong road...
|
Well, nothing we didn't know, but damn you seems like a smart guy. I'm impressed by your work for this and previous ones.
|
um... your random run and the terran so called "statistically significant" difference... Um, can I get some t-statistics to check for the random error? I mean.. I don't see any in your post, so i'm wondering where you're deriving the "significance" from. Also, the notion of skill independent of balance is a tricky matter in terms of interpretting data...but that's outside the scope of the post I believe anyway...
|
On July 11 2012 06:17 Aletheia27 wrote:
um... your random run and the terran so called "statistically significant" difference... Um, can I get some t-statistics to check for the random error? I mean.. I don't see any in your post, so i'm wondering where you're deriving the "significance" from. Also, the notion of skill independent of balance is a tricky matter in terms of interpretting data...but that's outside the scope of the post I believe anyway...
um...calculate ... um .... yourself.
its all there. i thought its enough to show its above 99.9% but feel free to get the exact number..
|
Nice job. But balance at (average) diamond isn't that interesting. Even high master/gm at NA/EU/SEA almost don't matter, but it would nice to do it there.
|
On July 11 2012 06:23 Tuczniak wrote:
Nice job. But balance at (average) diamond isn't that interesting. Even high master/gm at NA/EU/SEA almost don't matter, but it would nice to do it there.
you play sc2? you are korean grandmaster?
if yes and no.
Would you like it to have EU grandmaster skill and play against bronce guys because you play the wrong race?
I extrapolate to show your, your extrapolation.
This game is not for pro player! this game is for everyone!
If this game would be only for pro players than , i would not be here 
Btw most people ignore how close the numbers actually are.
|
|
|
|
|
|
EPT
