EPTLadder-Balance-Data - Page 7
| Forum Index > SC2 General |
|
AndAgain
United States2621 Posts
| ||
|
sCCrooked
Korea (South)1306 Posts
He's just releasing the info, not saying anything about balance. Talk about reading into what's not there... its got to be at least half the moronic posts already in this thread. | ||
BluemoonSC
SoCal8907 Posts
On July 11 2012 09:19 sCCrooked wrote: Where the hell are you guys getting the idea that this is somehow a formula for balance? Its a simple tool that was used for calculating the hidden MMR trait that we've been trying to find out how to see for years. He's just releasing the info, not saying anything about balance. Talk about reading into what's not there... its got to be at least half the moronic posts already in this thread. then he shouldnt use words like "balance" "underpowered" and "overpowered" if he's not talking about balance ;; | ||
|
NATO
United States459 Posts
| ||
|
Niazger
Germany41 Posts
On July 11 2012 08:31 BluemoonSC wrote: you had me until you claimed that MMR has something to do with balance - MMR is an indication of skill, not balance. i could cheese my way to a better MMR, but does that mean that the game is balanced? uncertain. i could use one build to get a better MMR, but does that mean that the game is balanced? uncertain. you take the average MMR of players across gold-GM (if i read correctly), yet somehow this is an indication of balance? that is absurd. balance can only be determined under ideal conditions at the highest level of play and even then, humans are all capable of mistakes. these are just a few of the problems i have with anyone using MMR (calculated skill) as a means of attempting to prove balance. watch some high level gameplay if you want to figure out whether or not the game is balanced bc, in my opinion, this study is completely irrelevant. Dude, you dont understand what OP wrote or didnt read it. People that cheese/do one build also don't matter since they should be equally represented in all three races. You suggest to watch some highlevel play. Well, thats what I did and it supports what the OP has concluded from his statistics. | ||
|
lolcanoe
United States57 Posts
But im still to 99,31 % secure Terran is underpowered -30.09 Where the hell are you getting these numbers for? I'm assuming this 99.3 comes from some application of the normal distribution - have you been thorough established normality in MMR? Based on my understanding of the MMR system, you would not expect normality... As a stats junky, I'm very suspicious of the statistics here. | ||
|
MrF
United States320 Posts
| ||
|
lazyitachi
1043 Posts
1) Can you show the breakdown by race for some MMR intervals to know the distribution of data used to derived these results? 2) Is it possible to further separate into submitter vs opponent data (I believe the MMR is calculated for both players in a replay?) for breakdown in (1)? 3) If you can identify submitter, then should we look at w/l ratio? At least to be certain that they submit at least a representative sample of their games and not ONLY their wins? 4) What does deviation mean?? Just means the deviation of MMR of the players within the replay? Seems to be some distortion going on just by looking at random either through submission bias or imbalanced distribution of "MMR". Timeline : Season 8 , start - 10/07 Deviation : 1000 times 4 random groups: 100% are in the area -+ 33.81: 99.15% are in the area +-25 96.6% are in the area +-20 88.88% are in the area +-15 70.3% are in the area +-10 39.7% are in the area +-5 Race groups: Terran: -44.83 Protoss: +19.83 Zerg: + 38.53 Random: -187.70 | ||
|
canikizu
4860 Posts
On July 11 2012 08:57 BluemoonSC wrote: you can throw as much math as you want at the problem, it doesn't change the fact that you're looking at data that is dependent on player skill and doesn't look what actually happens inside each individual game. MMR only looks at wins and losses. Every stocks you buy and sell, every money exchange international companies make, or even every items you buy, are independent transactions. There're many reasons you do it, whether it's impulse, invest, take care of tax, secret's company motive, ...v.v.v, but if you collect the data(which of course is dependent on participant's data) over the course of one month, one year, you will have a pretty good idea of which company stock is good, how strong an economy is, or how popular an item is. That's how statistic and chart is supposed to do. No matter what an individual does, as long as the huge majority have 1 common goal (invest to make money, buy items for self-satisfaction), you should be able to produce a data good enough to have an idea how it is going. It works the same way in here. As long as the huge majority players have one common goal which is to win the game, the data should give you an idea (a trend) of how the races are doing. MMR = player skill + race features. So if average Terran's MMR is lower than Zerg's, it's either Terran players are dumber than Zerg players, or their race features that they can take advantage of is lower than Zerg's. It's that simple. Now, like any other data, this data will not give you exactly which Zerg unit is overpowered, or which Terran unit is underpowered. That's not its job. So if you looking for answers for the questions such as : "On average, what units Terran make in order to win?", "On average, how long the game is for Terran to win the most?", "On average, what units Terran need to make to counter BL/infestors that prove the most effective?", you won't find it here. This only answers: With this data pool, on average, Terran is xx MMR lower than he's supposed to be, he has xx win less than he's supposed to be; or on average, Zerg is yy MMR higher than he's supposed to be, he has yy win higher than he's supposed to be. Does that indicate imbalance? maybe, maybe not, you decide. | ||
|
Plansix
United States60190 Posts
On July 11 2012 09:20 BluemoonSC wrote: then he shouldnt use words like "balance" "underpowered" and "overpowered" if he's not talking about balance ;; Yes, the word underpowered and balance infer things that the OP is not addressing. You cannot take into account a player like "BadHabit" who made it into GM by only six pooling(on a bet, but he still did it). There are to many other factors that effect MMR to say anything beyond "players of X skill level with Y race seem the a dip in MMR at Z point". But this data does not prove that terran is harder than protoss or zerg, like some posters are saying. | ||
|
IshinShishi
Japan6156 Posts
On July 11 2012 10:25 Plansix wrote: Yes, the word underpowered and balance infer things that the OP is not addressing. You cannot take into account a player like "BadHabit" who made it into GM by only six pooling(on a bet, but he still did it). There are to many other factors that effect MMR to say anything beyond "players of X skill level with Y race seem the a dip in MMR at Z point". But this data does not prove that terran is harder than protoss or zerg, like some posters are saying. 1 player is statistically irrelevant, on average players will play standard games, that's why we need a big sample which is what we have here, people making very bad points to deny this without thinking them through made this thread really annoying to read.The other matter has long since been settled, regardless of data(you should just try them out),you had better give up by now. | ||
|
Buddhist
United States658 Posts
MMR was used in WoW, and was visible. The idea that "warriors have higher MMR on average, therefore they are more skilled" would have been a laughable statement. There's no difference in SC2, except that MMR is mystified here because Blizzard hid it. Furthermore, you can't go the other way either, to say that higher MMR on average means less skill, because of: A) Metagame. Players don't use the best possible strategies available to their race (in the future, newer better strats will be developed, meaning that worse ones are being used in the present) B) Unequal distribution of race among skill levels (for example, very skilled players don't play random as often, because it's not viable for competition, so you'll get a lower average MMR for random necessarily as a result) | ||
|
Plansix
United States60190 Posts
On July 11 2012 10:34 IshinShishi wrote: 1 player is statistically irrelevant, on average players will play standard games, that's why we need a big sample which is what we have here, people making very bad points to deny this without thinking them through made this thread really annoying to read.The other matter has long since been settled, regardless of data(you should just try them out),you had better give up by now. Well you showed me with that clever word play right there. You mean to tell me that a sample size of "1" does is to small to provide and real evidence. I guess my example is invalid. Wait, what is the definiation of example again? Example - one of a number of things, or a part of something, taken to show the character of the whole So one could say that the one player I named in my previous post was just one of many players who are playing strangely on the ladder for any number of reasons. The data does not account for people who just play weird or "wrong". Like players who always go mech, regardless of the match up. | ||
|
cskalias.pbe
United States293 Posts
On July 11 2012 02:17 Chill wrote: Agreed. The whole basis for this project is defeated by one realistic (in my eyes) claim that is dismissed. Also, why do we care about average balance? If Zerg is easier than Terran from Bronze - Masters, does it really matter to the members of this forum? This could go the other way too. The most skilled players could choose what they perceive to be the most powerful race which, if it were Terran, as it has been for many seasons, could offset the results (although not really because there are naturally fewer of the more skilled than the lesser). That being said, with enough samples at dia+ i think that effect of lower skilled players picking terran as not really applying as much. I think members of this forum will obsess about the relative difficulty of one race to another at all levels, and I think OP chose a great way to attack the problem as opposed fickle ranting. (avg: 3 victorys behind) OP: Could you explain this please? If I played a 100 games as Terran v Zerg with same MMR, about what would be my win%? | ||
|
sevencck
Canada698 Posts
On July 11 2012 08:57 BluemoonSC wrote: you can throw as much math as you want at the problem, it doesn't change the fact that you're looking at data that is dependent on player skill and doesn't look what actually happens inside each individual game. MMR only looks at wins and losses. In a perfectly balanced game, if you averaged MMR data by race, you would expect the average MMR of each race to be about the same, allowing for small random variation. As I understand it, OP's MMR data shows that the game is on average basically rating Terrans as being less skilled than Protoss or Zerg, and his results show this difference to be much greater than what would be expected from random variation. Assuming that in reality all three races are equally skilled (a reasonable assumption), how do you rationalize this MMR disparity? It suggests that Terran might indeed be UP, or at the very least is really struggling due to the metagame? Since any imbalance gets inherently factored into the MMR, by extension we can infer certain things. Put simply, if all three races are equally skilled in reality (an assumption), and the game is on average rating Terrans as less skilled than Zergs (through MMR), then it implies that on average a Terran at equal MMR to a Zerg is in reality at least equally skilled, and quite probably more skilled than that Zerg. Thus, hypothetically, if this "averaged" T and Z at equal MMR ratings trade wins at a 50% rate, it in no way demonstrates that the game is balanced, despite a 50% winrate at equal MMR, since the imbalance is already factored into the MMR. I believe OP's point is that the way they're guaging "balance" may inherently be flawed. Not really completely sure about any of it though, I'm still thinking about it all. | ||
|
Buddhist
United States658 Posts
On July 11 2012 10:54 sevencck wrote: In a perfectly balanced game, if you averaged MMR data by race, you would expect the average MMR of each race to be about the same, allowing for small random variation. As I understand it, OP's MMR data shows that the game is on average basically rating Terrans as being less skilled than Protoss or Zerg, and his results show this difference to be much greater than what would be expected from random variation. Assuming that in reality all three races are equally skilled (a reasonable assumption), how do you rationalize this MMR disparity? It suggests that Terran might indeed be UP, or at the very least is really struggling due to the metagame? Since any imbalance gets inherently factored into the MMR, by extension we can infer certain things. Put simply, if all three races are equally skilled in reality (an assumption), and the game is on average rating Terrans as less skilled than Zergs (through MMR), then it implies that on average a Terran at equal MMR to a Zerg is in reality at least equally skilled, and quite probably more skilled than that Zerg. Thus, hypothetically, if this "averaged" T and Z at equal MMR ratings trade wins at a 50% rate, it in no way demonstrates that the game is balanced, despite a 50% winrate at equal MMR, since the imbalance is already factored into the MMR. I believe OP's point is that the way they're guaging "balance" may inherently be flawed. Not really completely sure about any of it though, I'm still thinking about it all. There's no reason to believe that the races have equally skilled players on average at any level of play. | ||
|
skeldark
Germany2223 Posts
I know it must sound cokey but i cant bring myself to answer most of the post because its so obvious the writer will not understand it anyway. If someone have a valid point or a specific understanding problem i will argue over my methods and explain it. But i will not explain fundamental basics and argue what skill is or why skillsystem dont show skill numbers. If you have a better system to measure skill, sell it to blizzard and all the others out there. you will get rich i promise! I was sceptical to publish it and i can not say im surprised by the reactions but many people here should really sit back and think a little bit before they post. If you did not understand the basic perhaps its no good idea to criticise points because most likely you dont know what you are talking about ... Sorry but had to say that. Now to some critical post that have specific points: On July 11 2012 10:15 lazyitachi wrote: Not sure what type of stats is this overgeneralisation 1) Can you show the breakdown by race for some MMR intervals to know the distribution of data used to derived these results? 2) Is it possible to further separate into submitter vs opponent data (I believe the MMR is calculated for both players in a replay?) for breakdown in (1)? 3) If you can identify submitter, then should we look at w/l ratio? At least to be certain that they submit at least a representative sample of their games and not ONLY their wins? 4) What does deviation mean?? Just means the deviation of MMR of the players within the replay? Seems to be some distortion going on just by looking at random either through submission bias or imbalanced distribution of "MMR". 1) there is a datafile in the op but it had some bugs. I will create a new one with more data. 2) it is 3) quantity dont care. Only quality. One good game is enough to calculate MMR. Also i explained why problems of the calculation dont effect the result 4) i create random groups to proof the data is not biased in any way towards something else than race. and to measure the % of chance that my results are wrong On July 11 2012 09:34 lolcanoe wrote: I'm confused here - why are there 4 random groups? Why not just lump them together? Where the hell are you getting these numbers for? I'm assuming this 99.3 comes from some application of the normal distribution - have you been thorough established normality in MMR? Based on my understanding of the MMR system, you would not expect normality... As a stats junky, I'm very suspicious of the statistics here. 4 because i had 4 races ( with random ) 2 run had only 3 . Dont have be this number only to get kind of close to the amount of accounts of the races. MMR is normal by definition ( if nothing goes wrong) If this data is normal i can not tell i did not test yet Its just the % of the chance that random data would create such an result. depending on tests. Im not 100% sure if you can do it this way but i dont see why not. Im no statistic guru in any way. I am not terrible in it but its many years ago i worked with it. Basic i just produce random data to show that the data is not biased and how "stable" it is. I think there are better ways to show this but this is the fastest with an reasonable result. On July 11 2012 10:48 cskalias.pbe wrote: (avg: 3 victorys behind) OP: Could you explain this please? If I played a 100 games as Terran v Zerg with same MMR, about what would be my win%? you get in average 16 mmr for a win. I just pointed that out to show you the number in context For every run my comuter calculates 3 h and it will get way more with more data. I have to optimise the datastruckture sooner or later. But its not easy to just change a little part and post the result... If i have more data i will put more afford in outputing a good source file so people who want can calculate them-self. At the moment the hole calculation is in the memory and its not that easy to output between steps. | ||
|
VIPlol
United States43 Posts
On July 11 2012 02:01 Malaz wrote: Ok time for me as a random player to start crying about balance. Buff random already Blizzard!! Yes, buff it's chance to play each race. 80% Protoss, 19% Zerg, 1% Terran  | ||
|
sevencck
Canada698 Posts
On July 11 2012 10:59 Buddhist wrote: There's no reason to believe that the races have equally skilled players on average at any level of play. Then there was no reason to believe Terran needed to be nerfed a year ago based on the GSL results, Terran might well have had the more skilled players. Basically, there is a very good reason to believe that the races have equally skilled players on average. It's the basis for any attempt to balance the game. | ||
|
zEnVy
United States446 Posts
On July 11 2012 10:25 Plansix wrote: Yes, the word underpowered and balance infer things that the OP is not addressing. You cannot take into account a player like "BadHabit" who made it into GM by only six pooling(on a bet, but he still did it). There are to many other factors that effect MMR to say anything beyond "players of X skill level with Y race seem the a dip in MMR at Z point". But this data does not prove that terran is harder than protoss or zerg, like some posters are saying. Sure it does. Your example of a player like BadHabit are an incredibly small part of an overall sample. The vast majority of players do not have a one trick pony that would normally remove race balance from the game. I don't see how people can look at such a staggering set of data like this and still draw the conclusion that it doesn't prove anything. I guess people can stare at overwhelming evidence and still disagree with it because it contrasts with their own personal opinion. I'm a Protoss player, and I would argue Terran is noticeably weaker than the other 2 races. In tournaments Zerg seems to be doing the best, but in my opinion PvZ is slight favored for Protoss. | ||
| ||
