EPTBlizzard's "skill-adjusted-win-percentages" - Page 7
| Forum Index > SC2 General |
|
hummingbird23
Norway359 Posts
| ||
|
Ryps
Romania2740 Posts
On September 30 2011 18:47 hummingbird23 wrote: MMR is also a pretty bad indicator of balance, simply because unless Blizzard is using matchup specific MMR, one bad matchup wrecks the reliability of MMR for every other non-mirror matchup. Say TvP is screwed in favor of Terran, this would make PvZ look in favor of Protoss simply because the Protoss player is playing less skilled zerg to force the win percentage towards 50%. Hence, zerg players on the ladder would have the perception that ZvP is Protoss favoured, except that that would be because they're playing opponents of a higher skill. Who said they are balacing based on MMR, they said they look at specific match ups percentages. I see a lot of people bashing Blizzard for what system they are using but no one is giving suggestions on whats better. | ||
|
Silidons
United States2813 Posts
On September 30 2011 18:57 Drey wrote: Who said they are balacing based on MMR, they said they look at specific match ups percentages. I see a lot of people bashing Blizzard for what system they are using but no one is giving suggestions on whats better. He is saying that win-percentages are not a good indicator of things like balance because of how they are actually measuring the win-percentages (since who you play is based on MMR) This is what he means: Let's say Toss has a 40% win rate against Terran, assuming they are of both skill level and things such as EMP make the matchup favor the Terran. Toss then LOSE MMR since they have a hard time playing vT, and so as I said their MMR goes down, but then they get matched up with a Zerg. Well this Toss' MMR is artificially low because PvT is in favor of Terran, so he is playing a zerg who is a lesser-skilled player than him, which then will create PvZ to favor the toss because toss are actually playing worse players since their MMR is artificially low. | ||
|
hummingbird23
Norway359 Posts
On September 30 2011 18:57 Drey wrote: Who said they are balacing based on MMR, they said they look at specific match ups percentages. I see a lot of people bashing Blizzard for what system they are using but no one is giving suggestions on whats better. My point is that it's pretty hard to use the numbers Blizzard is giving. Each matchup for each player differs. One way to test this system would be to throw every match of a particular matchup (eg. Terran player always quits whenever he sees Zerg), and see if the overall win/lose still tends towards 50%. If it does, it means that whenever the player plays against Protoss or Terran, he's playing inferior opponents. No amount of TvP balance will thus fix a TvZ imbalance, for example. | ||
|
FeyFey
Germany10114 Posts
On August 03 2011 13:34 KiLL_ORdeR wrote: The third and arguably most important factor that they exclude from that though is map balance. They will never get a perfect rating unless the system takes the maps in account. you already forgot their statement about maps, balance it on small maps and it works everywhere. and it may be 2 unknowns but they relate to each other. So once you know who did win you can calculate them both. Its actually a clever system to sort out the ladder system searching equal opponents for you. It gives you a good overview of how things changed after a patch at the different levels, which is probably the most important thing. Though you might not see which race has the less effort do do to win, but since they address this as well I guess somehow they know. | ||
|
Deleted User 101379
4849 Posts
They have really, really clever guys working on that, people that are about twice as smart as all posters here combined (including me) and they will have considered everything that is posted here and found ways to work around the limitations. Example for how it could be, eventhough i'm not half as clever as the guys who thought about those things 8 hours a day for month: Blizzard has 3 MMRs, 1 for each race Player A (Zerg): vT: 60% Winrate, MMR 1800 vZ: 30% Winrate, MMR 1200 vP: 40% Winrate, MMR 1400 We can assume that his average MMR is somewhere around 1500-1600, he just is very strong vs Terrans and less strong vs Protoss. Against Zerg, he sucks. Player B (Terran): vZ: 40% Winrate, MMR 1400 vP: 60% Winrate, MMR 1800 vT: 50% Winrate, MMR 1600 This player has has an average MMR of 1600, he is slightly stronger vs Protoss, slightly weaker vs Zerg. Player C (Terran): vZ: 60% Winrate, MMR 1800 vP: 50% Winrate, MMR 1600 vT: 50% Winrate, MMR 1600 This player has an MMR of about 1600-1700 and is slightly stronger vs Zerg. If Player A and Player B meet 100 times on the ladder and Player A wins 40% of the matches while Player A and Player C end up in ~55% for Player C, the game should be balanced, eventhough for the point of Player B, it's imbalanced since he loses a lot. Personal skill in different matchups is accounted for and everyone should be happy. Ofc as mentioned, there are more intelligent people whos job it is to calculate that stuff. They had month to figure everything out, i though of this in about a minute so i might be totally wrong. | ||
|
Hider
Denmark9378 Posts
On September 30 2011 21:25 Morfildur wrote: Does almost everyone in this thread really think Blizzard is stupid and ignores such obvious things? They have really, really clever guys working on that, people that are about twice as smart as all posters here combined (including me) and they will have considered everything that is posted here and found ways to work around the limitations. Example for how it could be, eventhough i'm not half as clever as the guys who thought about those things 8 hours a day for month: Blizzard has 3 MMRs, 1 for each race Player A (Zerg): vT: 60% Winrate, MMR 1800 vZ: 30% Winrate, MMR 1200 vP: 40% Winrate, MMR 1400 We can assume that his average MMR is somewhere around 1500-1600, he just is very strong vs Terrans and less strong vs Protoss. Against Zerg, he sucks. Player B (Terran): vZ: 40% Winrate, MMR 1400 vP: 60% Winrate, MMR 1800 vT: 50% Winrate, MMR 1600 This player has has an average MMR of 1600, he is slightly stronger vs Protoss, slightly weaker vs Zerg. Player C (Terran): vZ: 60% Winrate, MMR 1800 vP: 50% Winrate, MMR 1600 vT: 50% Winrate, MMR 1600 This player has an MMR of about 1600-1700 and is slightly stronger vs Zerg. If Player A and Player B meet 100 times on the ladder and Player A wins 40% of the matches while Player A and Player C end up in ~55% for Player C, the game should be balanced, eventhough for the point of Player B, it's imbalanced since he loses a lot. Personal skill in different matchups is accounted for and everyone should be happy. Ofc as mentioned, there are more intelligent people whos job it is to calculate that stuff. They had month to figure everything out, i though of this in about a minute so i might be totally wrong. This isn't the main problem. The main problem is if race X is UP in all both mathcups. Then blizzards statisctics wont be able to realize that all players of race x actually have to low an mmr. If they used stats like average mmr of races they would have to assume that players of different skills are somewhat evenly distributed over the different leagues, however that is most likely not true. | ||
|
Deleted User 101379
4849 Posts
On September 30 2011 21:55 Hider wrote: This isn't the main problem. The main problem is if race X is UP in all both mathcups. Then blizzards statisctics wont be able to realize that all players of race x actually have to low an mmr. If they used stats like average mmr of races they would have to assume that players of different skills are somewhat evenly distributed over the different leagues, however that is most likely not true. They can still see it when the MMR of many players against one race is higher than against the other races. Example: We can assume that over all Protoss players, their relative skill in other matchups is about even or the difference is statistically insignificant. Taking all 500'000 protoss into account, some are stronger in other matchups, some are weaker. Some are better in mirror matches, some are worse. With that much data, it will even out. We can then check the MMR of the mirror match compared to the other players MMR in mirror matches and check the results of the match. For each player we go through each match, show how the person has faired in mirror matches, check the others MMR (again: by mirror) and check who should win. Then we look at who really won and compare the results. If the win was unexpected, we note it and move to the next game. For a single game, the results will be wrong. Everyone can have a bad day, made a mistake, etc., but statistically, over the huge number of games, the results will even out. Not perfectly, for that the number is still too low, but good enough to show big differences (that might be why they say that between 45% and 55% there is no reason for concern. It's just statistics after all). If we then find out that Protoss only wins 80% of the matches they should have won, we can put it down as 40% winrate for Protoss and therefor a big imbalance. Anyways, as i said, there are people who think about it 8 hours a day 5 days a week. They are not stupid, they learned that stuff and know more about it than we do. They have a lot more data than we have access to and more than we might even assume, so we should trust that they don't just pull numbers out of their a**, just because we don't know how they do it. | ||
|
DarQraven
Netherlands553 Posts
On September 30 2011 22:52 Morfildur wrote: They can still see it when the MMR of many players against one race is higher than against the other races. Example: We can assume that over all Protoss players, their relative skill in other matchups is about even or the difference is statistically insignificant. Taking all 500'000 protoss into account, some are stronger in other matchups, some are weaker. Some are better in mirror matches, some are worse. With that much data, it will even out. We can then check the MMR of the mirror match compared to the other players MMR in mirror matches and check the results of the match. For each player we go through each match, show how the person has faired in mirror matches, check the others MMR (again: by mirror) and check who should win. Then we look at who really won and compare the results. If the win was unexpected, we note it and move to the next game. For a single game, the results will be wrong. Everyone can have a bad day, made a mistake, etc., but statistically, over the huge number of games, the results will even out. Not perfectly, for that the number is still too low, but good enough to show big differences (that might be why they say that between 45% and 55% there is no reason for concern. It's just statistics after all). If we then find out that Protoss only wins 80% of the matches they should have won, we can put it down as 40% winrate for Protoss and therefor a big imbalance. Anyways, as i said, there are people who think about it 8 hours a day 5 days a week. They are not stupid, they learned that stuff and know more about it than we do. They have a lot more data than we have access to and more than we might even assume, so we should trust that they don't just pull numbers out of their a**, just because we don't know how they do it. Those very smart people, thinking about this 8 hours per day, 5 days per week, are also governed by deadlines, budgetting and compromising due to demands of managers, programmers and networking technicians. What they can think up as a perfect system isn't necessarily what they end up shipping. I'd like to point you to the state of bnet 2.0 at launch for a pretty clear example of "very smart people" shipping absolute garbage. Blind faith in corporations is a bad idea. | ||
|
MockHamill
Sweden1798 Posts
1. Create separate bell curves for each race. Place every single Zerg player on the the Zerg bell curve based on their MMR. Do the same for Terran and Protoss. 2. Compare the outcome of matches between Protoss and Zerg players at the same place at each race bell cure. 3. If Zerg players that are top 5.5-6.5% in skill on the Zerg bell curve win 60% of their matches against Protoss players that are on top 5.5-6.5% on the Protoss bell curve, there is an imbalance at that specific skill range. This assumes that that for most of the curve the skill of the players of the races are about the same. This is very likely given the huge amount of players. However, very far out on the curve it is possible this does not hold true due to the numbers of players at those positions on the curve are so few. | ||
| ||
