EPTScientific proof that SC2 is imbalanced (sorta) - Page 8
| Forum Index > SC2 General |
|
Flyingdutchman
Netherlands858 Posts
| ||
|
GagnarTheUnruly
United States655 Posts
| ||
|
x7i
United Kingdom122 Posts
if the races are imbalanced, AMM will have the effect of changing the distribution of the races among the leagues now, will it? logically it should, but considering whole system is designed to create illusion of balance and equality there is no reason to believe it does, it as well might be maintaining equal race distribution on visible level while analysis of hidden matchmaking ratios would paint a very different pictureand of course youre right, but we believe what we choose to believe anyhow good academic work... and i do find the win ratio graph interesting and meaningful actually ;-) | ||
|
Keitzer
United States2509 Posts
| ||
|
Ganondorf
Italy600 Posts
| ||
|
edahl
Norway483 Posts
On August 17 2010 07:35 neobowman wrote: Isn't this math and not science? o_O User was warned for this post | ||
|
hdkhang
Australia183 Posts
On August 17 2010 18:45 GagnarTheUnruly wrote: Thanks. As I've said, if the races are imbalanced, AMM will have the effect of changing the distribution of the races among the leagues. Because this doesn't happen, my results still provide imformation suggesting that overall balance is good. The problem is not in your methodology as much as it is in the assumptions you are making about the system. What do we know about the system? Anyone who wishes to play on the ladder is accepted * You can have zero knowledge of the game and yet you will be placed in Bronze (in sports, that makes you 3rd place!), the only prerequisites for acceptance into a league is to just show up 5 times (you can even disconnect 5 times once the game starts and still get in). * This is an automatic invalidation of any results emerging from the Bronze league as the range of skill present is astronomical! You only play matches against people in your region * Comparisons across different battle.net servers is worthless. The hidden MatchMakingRating number is based only on wins/losses with respect to the current MMR of yourself and your opponent * What was required to score that win is not considered at all, nor should it be. Unfortunately, this places the onus on making a game as balanced as possible all the more important or else the MMR is worthless. * This point also explains why the data is "practically worthless". The AMM will attempt to pair you up with a person with a similar MMR * Note that I say similar MMR and not similar skill/ability. * If a race imbalance existed, the MMR would not reveal it since it would simply consider the person using the weaker race as a "poorer" player hence a lower MMR and the person using an OP race as a better player rewarding them with a higher MMR. * Thus if you were to compare even between players of similar MMR but across the three races, it would reveal nothing of significance since the reason they were given that MMR is due to their win/loss performance against the same people they are being compared against. Not everyone will play the same number of games * You may say "well duh" but it bears repeating. * I think in the original top 200 list one of the players that made it had only played a handful of games, I think it was 7 all up, yet 7 games was enough for the system to determine their MMR to be one of the top 200 on the server. * I honestly believe that a more stringent pre-requisite for diamond league is needed, e.g. 100 games played. There are many more other points to make, but let's just start with the above for now. | ||
|
icezar
Germany240 Posts
Even better are the your coments and response to all other. I am courious what do you make of the race distribution? There you can see a clear difference. For example in Diamond League across al regions only 24% plays zerg as opposed to 35% playing Protoss. | ||
|
Dagon
Romania264 Posts
Because of the match making system it may be true that random players from all leagues experience no imbalances but the question should be if all the mach-ups are balanced at pro level not that the game îs balanced all around.. I think.. Please correct me if i am wrong about the rock-paper-scissors thing because i really want to understand how this works. | ||
|
escapeArtist
Norway2 Posts
After reading many of the comments to this great post I have set up some graphs in Excel to see how races are represented across the leages, and how they evolve as skill level evolves. I work with analysis as a living (I'm not claiming to the best at it), but I'm not a native english so please exscuse me for not using the correct terms. Source: http://www.sc2ranks.com/stats/race/all/1 Date: 18.08.2010 The source of the data may me skewed, but I feel that it's enough players to give a reasonable representation of the general trends. Total number of players / 1v1 teams: 574 314 First off I had a look at the general representation of all the races: Race Distribution ![[image loading]](http://i1035.photobucket.com/albums/a431/escapeArtist80/racerepresentation-1.png?t=1282146816) As we can clearly see from this cake diagram Protoss is the most represented race if you look at all the brackets and players. And to noones surprise Zerg is the least represented. For a look at the general representation of players I also sorted the players by leages Player Distribution ![[image loading]](http://i1035.photobucket.com/albums/a431/escapeArtist80/playerrepresentation.png?t=1282146816) No big surprises here either I think. So if you filter by popularity of each race by points / leagues then the following line diagram comes up: Race development by points and leagues ![[image loading]](http://i1035.photobucket.com/albums/a431/escapeArtist80/rankingrepresentation.png?t=1282146816) First of all I must comment on the numbers and criterias used in this graph. I removed the best of the best in each league to prevent huge leaps and irregulareties to the graph since the there was so few players. (You can't compare 30 people to 100 000). Second I added a moving average(5) to even out the peaks to give a better visual representation of the development for each race. Not Surprisingly Terran starts out very strong, but as you can clearly see Terran looses popularuty as leagues move up and picks ups slightly again at the upper diamond. This is most likely players moving from random races, but again every race gets immigration from the random race. Protoss is going relativly strong from lower bronze to upper diamond and Zerg is starting out weak while getting more and more players as it moves up the ladders. As you can also see from this then all the races gets players from the random race in the diamond leage (probably as players have to focus their time on 1 race to stay competative). The strongest development from bronze to diamond is withouth a doubt Zerg, while Terran loses the most players. With this I guess you can say that Terran is the typical "noob" race and Zerg is the typical "Pro" race if you judge only by development. Now there are many things to consider when judging imbalance and i feel that the OP made a VERY string point in pointing out that the Matchmaking system works VERY well. If you keep in mind that every player above bronze wins about 55% of his matches, meaning that comparing percentage development of each race is a reasonable estimation of balance at the players current place at the ladder. (Sorry if that got complicated as I'm not a native english). Moving on the my last graph for the evening, the total percentage represented from each race in each league: Racial representation for each league ![[image loading]](http://i1035.photobucket.com/albums/a431/escapeArtist80/racerankingrepresentation.png?t=1282146816) This means that 9,1% of Zerg players are in diamond league compared to 6,7% of the terrans are in diamond league. This shows how many percent of each race is in each leage. Keep in mind that every player wins roughly 55% of their matches. Here we can see that Zerg and Random players are way ahead of the puny Terran and the lowly Protoss. You can say that Zerg and Random does better the higher up in the ladder you get, while Terran abd Protoss does worse the higher you get. Does this mean that Terran is underpowered while Zerg is opverpowered in the higher tier of play? Most certanly not! But it does show that every race can stay competative at all levels of play. I generally don't like to speculate, but I will anyway. Zerg takes more APM to do well, and that is a typical trademark of a good player. Terran doesn't take as high APM so many newer players pick Terran or Protoss. This may or may not be correct as I have no way of proving this, but it's a personal experience from playing Zerg. (I switched to Terran as I am not good enought to micro everything I have to when Zerging) I'm not going to conclude that there is perfect balance here, BUT I am going to conclude that if you judge by the numbers then I certanly can't find any proof of huge balance issues. Even saying that, fact of the matter is that Zerg and Random has the highest representation in the higher level of play compared to their relative playerbase. This does not make Random overpowered lol, so we can't really conculde that it makes Zerg overpowered either. The only curious point here is that the higher up the ladder the more Zerg there is... Ofcourse a valid argument is that on the top 0,06% of the players (diamond 1001 +) they may feel that there is balance issues, however this affects such as small amount of the playerbase that if you are going to balance from personal opinions from these players you would most defenetly screw up the general game balance. Ofcourse if at these levels there are such big differences that tournaments would be Terran & Protoss only then measures must be taken, but so far it looks reasonable. Time will tell tho, but even if Protoss keeps winning turnaments it does'nt necceracy means that Protoss is overpowered. All in all if there is inbalance then 99.94% of us woulnd't notice the difference anyway. | ||
|
travy
United States14 Posts
http://www.teamliquid.net/forum/viewmessage.php?topic_id=145325 | ||
|
MamiyaOtaru
United States1687 Posts
On August 19 2010 01:53 escapeArtist wrote: Because noobs don't play zerg (because you don't play them in the campaign, or because they are harder?) Or because they are harder, only higher level players stick with them? No clue, but I have a hard time drawing any conclusions from those results.The only curious point here is that the higher up the ladder the more Zerg there is... | ||
|
silencesc
United States464 Posts
| ||
|
eivind
111 Posts
On August 19 2010 01:53 escapeArtist wrote: The only curious point here is that the higher up the ladder the more Zerg there is... All in all if there is inbalance then 99.94% of us woulnd't notice the difference anyway. People experienced with SC play Zerg and the race is harder to start up with. So I dont think it is that curious that there are more Zerg players there. Your only proof is that skill can compensate somewhat for balance issues. I can beat 2-3 players at the same time with any race, this doesnt mean that my race is overpowered. If I gave 1 of them a really overpowered race which compensated for their lack of skill, then the matchmaking system would match us up if we were even players! If we assume that the races are imbalanced then the matchmaking system matches Zerg players with lower skilled Terran players. The only way of noticing imbalance (by only looking at the ladder) this is to look at the top players where there would be a less Zerg players. This data can only be used if the top players are somewhat evenly skilled. | ||
|
kataa
United Kingdom384 Posts
Imbalance only really comes into the play in the nosebleeds of any reasonably crafted game. Social issues surround playstyle, attitude ect. can always explain other small variations in win rate. If balancing a game was as simple as getting the win rates to be similar then creating RTS games would be easy as hell. We're not really going to know how bad or good the balance is for some time. Though, I'll certainly say that players like Masterasia and Sheth have made some very good points that go much deeper into SC2's design that simply 'OMFG nerf reaperz plz.' and it will be interesting to see how blizzard responds to this. All these statistic prove, is that the match making system is pretty effective. | ||
|
vesicular
United States1310 Posts
| ||
|
silencesc
United States464 Posts
I must stress that this is based on the data provided, which means that if the match making system really does try to make people have 50% (which I'm starting to doubt), then it is doing it's job, and keeping the game fair. This isn't to say that in the top 1% (pros), that there isn't inherent imbalance, but that evidence is anecdotal based on a small sample size of games played by pro players, I can run tests on the pro's based on race and W/L based on the reported total games from them on sc2ranking, but without a sample of at least 50 from each player, it won't really matter. I can tell you, however, that people like IdrA or HuK that play on ladder have huge ratios, IdrA is at like 80% right now, obviously significantly higher than everyone else, but with only about 100 games played by him, I can't say for certain if that's saying things about the caliber of random diamond players, or a real trend. | ||
|
chuninexam
Canada56 Posts
Do you realise how much time and trial and error is involved in balancing a game like this that has so many variables? | ||
|
Lalgee
United Kingdom65 Posts
The Problem Suppose a new 4th race is added, which is slightly stronger than the current 3 races. What happens? Well, the players who are playing this race and beating people of equal skill to them, however the rankings system is unable to determine a users skill level. The rankings system is only able to identify how often they are winning. Therefore, the players who play as the new 4th race will be higher in the rankings than they should be based on their skill. The fact that they are now higher in the rankings will result in them being matched with players who are better than them, but not using the imbalanced race. They will still win ~50% of their games, however they are higher in the rankings than people of equal skill, and the rankings have had to input them at a higher skill level in order to balance the imbalance of the race, so to speak. Further thoughts The fact that this is how the rankings works makes it nearly impossible to determine whether or not a race is imbalanced, apart from by listening to the people who are in the upper upper ranks. The fact that people like IdrA, White-Ra and MorroW are all able to win tournaments with their respective race means that the races are certainly balanced enough for people of equal skill lower down the rankings to have fair and even matches, where the difference in skill, or frequency/severity of errors is more likely to decide a game than the race they have selected. An Alternate Approach Suppose then that there was a way to have players of a certain skill level facing each of the three races, while playing as each of the three races and maintaining their current skill level on the rankings. Fortunately, random players fill this exact mould. Therefore I would suggest that the only way to find out whether or not a race is balanced/imbalanced is to look at the win % for random players while playing as each of the three races. If one race is significantly higher than the others, or one lower than the others, then it could be suggested that that race is imbalanced, one way or another. | ||
|
Blabla13
Comoros5 Posts
Thank you  Bob. | ||
| ||
