EPTIt seems that P is indeed the easiest race to play.
Inferring race imbalances from PSR distributions - Page 3
| Forum Index > SC2 General |
|
Nightfall.589
Canada766 Posts
It seems that P is indeed the easiest race to play. | ||
|
Mearis
Italy76 Posts
On October 09 2010 04:17 darmousseh wrote: Well, the problem is that i've studied these algorithms that they use. I have a degree in freaking mathematics which i don't even use for work (I am in computer science now) and I know that unless they've come out with a breakthrough in probability, that the information they posted is absolutely meaningless. It is impossible to separate player skill from performance when you play a single race. There simply is not a way of doing it. The algorithm for calculating rating itself is based on on a probabilities, but the probabilities themselves are based on ratings. It is circular. It has no meaning because the conditions are not the same. If players were forced to play random 100% of the time then yes! But they are not. I want to define something. Performance is a measurement of how well you did in a game. Rating is a measurement of how well you did in a set of games under a single condition. Skill is a measurement of how well you did in a large set of games under all conditions equally. Unless blizzard is redefining skill, which they probably are, this simply does not exist. They could be making wild assumptions like "players with similar ratings are similarly skilled" or "players are equally skilled at all of the different matchups ", but these by definition are not acceptable assumptions to make unless you want to "have something to show your boss". There is absolutely no way that they could extract this information. I think i can prove my point simply by stating that they put the percentages in the format 59.3% without including a +/- why? Because there is no such formula to calculate it. That's kinda where I am leaning myself - unless you make some very strong assumptions about the priors in terms of skill distribution, I don't see how you can calculate the imbalance between races. There was a really clever assumption on the first page about examining random players, but that involves some assumptions as well, however, potentially that's one of the more creative solutions that I have read. | ||
|
kevmo
United States38 Posts
Contrived example: For instance, consider 26 games in say, PvZ: three are fully favored towards the P, twenty are even match, and three are fully favored towards the Z. P wins 9 even match while zerg wins 11 even matches P wins all of their favored matches and one where Z is favored (3 + 1) Z wins 2/3 of their favored matches and none where P is favored. Both P and Z have a 50% win ratio, but P is probably slightly ahead because they performed much better than expected in winning a favored match. In other words, scale weighting of matches based on the computed probability of one player winning. I think that would be a reasonable way to calculate "skill independent" win ratios. Again, I am not proposing that this is what Blizzard does, merely an example showing that there are possibilities for taking skill into account when looking at match results in aggregate. Edit: Computing the numbers in this way would essentially be a predictor for how fast a given race is rising or falling through the ladder. It is similar to looking only at the tail ends (top diamond and low bronze), only you can see how the balance is shifting race distribution through the entire range of the ladder. | ||
|
Galleon.frigate
Canada721 Posts
| ||
|
Tef
Sweden443 Posts
A simple example would be guessing a number between 0-100. First you guess 50, then if the balance is pointing towards 100 you will guess 75 the next time. Thus eliminating the max and minimum. The difference is that you have to guess a multiple set of numbers that are dependent on each other. This system usually result with a flavour of the month within the first couple of changes. I am just saying that even with the most advance analytical tools the core issue will be the same. What, how much and in which order should we change things to reach balance in the shortest amount of time? | ||
|
MrCon
France29748 Posts
On October 09 2010 02:52 Nightfall.589 wrote: And I can't see the barracks change being aimed at anything other then fast reaper. 9/8/7rax reaper in particular - which is much more effective when combined with a 10/8/6 pool or 10/10/10 proxy gate in 2v2 than it is in 1v1. I really don't get how that can be aimed to 2v2+ play. Rax before depot is the only way to stop proxy gates and 6 pool from multiple opponents (like 2toss or 2zergs, an it's worse in 33) So this change stop some cheeses but makes other cheeses unstopable, that makes so no sense.. | ||
|
darmousseh
United States3437 Posts
On October 09 2010 04:40 kevmo wrote: I am not saying this is how Blizzard calculates win ratings normalized to skill, but if it were me, I would calculate "performance against expected". Contrived example: For instance, consider 26 games in say, PvZ: three are fully favored towards the P, twenty are even match, and three are fully favored towards the Z. P wins 9 even match while zerg wins 11 even matches P wins all of their favored matches and one where Z is favored (3 + 1) Z wins 2/3 of their favored matches and none where P is favored. Both P and Z have a 50% win ratio, but P is probably slightly ahead because they performed much better than expected in winning a favored match. In other words, scale weighting of matches based on the computed probability of one player winning. I think that would be a reasonable way to calculate "skill independent" win ratios. Again, I am not proposing that this is what Blizzard does, merely an example showing that there are possibilities for taking skill into account when looking at match results in aggregate. Edit: Computing the numbers in this way would essentially be a predictor for how fast a given race is rising or falling through the ladder. It is similar to looking only at the tail ends (top diamond and low bronze), only you can see how the balance is shifting race distribution through the entire range of the ladder. The trueskill algorithm gives you an odd of winning a game, if you win a bunch of even match games in a row it means that you are not rated correctly. The only thing that this can possibly tell us is the average variance in performance between the different races (a low variance means that the race is stable and that different strategies have a small impact on gameplay, a high variance means that the race is unstable and that changing strategies can have a huge impact on gameplay) Like I said in an earlier post, it is impossible to extrapolate skill. | ||
|
NoXious90
United Kingdom160 Posts
| ||
|
darmousseh
United States3437 Posts
(And it makes sense. pylon -> gateway, pool takes forever to build and 200 minerals, supply -> racks) I do respect their effort at balancing. The one complaint i have is how gimmicky protoss seems in air. I've stopped making air units altogether and just use blink stalkers instead. 1. I save money since i never spend resources on that tech. 2. Ground upgrades help everything out of the gateway and robo bay 3. teching to blink -> ht is not nearly as risky and is good against almost all compositions. If i see tanks, i make more stalkers, if i see more bio, i make more ht's. If i see hydras i make collosus or ht, if i see roach i make chargelots. | ||
|
Grond
599 Posts
| ||
|
Mearis
Italy76 Posts
Darnousseh 'gets it' basically - in a trueskill type algorithm, you have very little way of cleaving the hidden variables that obtain match win % without making some very questionable assumptions about your bayesian priors. Match up win % is however completely and utterly worthless as a statistic because your trueskill algorithm will push all win % to 50%, eventually. | ||
|
darmousseh
United States3437 Posts
http://www.teamliquid.net/blogs/viewblog.php?topic_id=159207 | ||
|
ideadude
United States7 Posts
1. I'm pretty sure that the Blizzard guys are the most experience people to be doing this kind of balancing... they are at least on the short list. We should still stay on top of them, but this makes me feel a little better. I doubt they are "clueless". 2. Do you think it's possible they turn off the matchmaking algorithm for a certain number of games to remove the impact of the match-making from their numbers. With thousands of games played, they could easily do this for 1 in 50 games and get some sizable data to come up with general race vs. race win percentages. If #2 were in affect, you could be a certified bronze player and all of a sudden be playing against a pro diamond and get rolled. Or a diamond and all of sudden play against a complete noob. Has anyone who plays a lot noticed something like this? You are randomly paired up against someone much better or worse (suppose it happens in placement matches too). | ||
|
Grond
599 Posts
On October 09 2010 07:29 Mearis wrote: I think that discussing the gameplay mechanics that cause race imbalances is important and interesting, but that's not really what I was trying to get at in my post. Darnousseh 'gets it' basically - in a trueskill type algorithm, you have very little way of cleaving the hidden variables that obtain match win % without making some very questionable assumptions about your bayesian priors. Match up win % is however completely and utterly worthless as a statistic because your trueskill algorithm will push all win % to 50%, eventually. My point is racial imbalance is irrelvent if it is bad gameplay. However fixing bad gameplay is going to effect balance so fix the gameplay first and then the balance. | ||
|
ToxNub
Canada805 Posts
Some things can be balanced (or at least attempted): win rate, for one. That's the matchmaker. Two: appeal, the number of players playing the race. Because those are quantitative. Easy. But you can't simply say that making a drone every 17 seconds, using actions that require 4.2 eye movements, 3 clicks, expending 0.1 joules of energy in one muscle, 0.2 joules in another muscle, with a 45 second delay for spawn larvae that relies on memory, is "equal" to remembering that 3 independent scvs are in the process of building stuff at 30seconds, 10 seconds, and 4 seconds remaining respectively. Then you factor in how difficult a skill is to learn, perform, execute quickly, how easy it is to forget, and then realize that all people might not be able to do each with the same ability, and on and on, until you quickly conclude that this approach is utterly hopeless. Trying to compare these things quantitatively is going to fail. Apples to oranges. You're not going to isolate skill anytime soon if you have no control variables. As kevmo pointed out, if you match based on your points (averaged for all your matchups) you can determine the win rate in various matchups and use this. However, as long as blizzard is taking the matchup into account when actually setting up the match, you can't conclude anything. It's ironic that the thing that keeps the game balanced (in actual play) is the same thing that prevents the game itself from being balanced. But if you remove that one factor, then discovering balance becomes fairly easy. ZvT (expected) = Avg( ZvP,ZvZ) and for your opponent TvZ (expected) = Avg (TvZ, TvT) If you then match 2 players with the same expected values and compare the outcome of these battles, you can infer which matchup is actually "harder". You still can't determine "skill", because for all you know the TvZ matchup is much more demanding on skills a,b,c (than say, ZvP) but player 1 happens to be very good at a,b,c. But you can at least average the difficulty level of playing the race. | ||
|
lu_cid
United States428 Posts
| ||
Excalibur_Z
United States12274 Posts
This will be explained by Dustin in more depth at BlizzCon during the SC2 Multiplayer panel. The system uses the hidden ratings as normal to determine which player is favored to win. The system also looks at the race matchup as sort of a “third player”. It’s tabulating through thousands of games. As an example, if there is a trend of terran players who are the underdogs in their respective 1v1 matchups winning against favored zerg players, the system will recognize that imbalance and predict that the terran race is favored over zerg. These numbers fluctuate (or not) with every single match played based on predicted outcomes versus actual outcomes. So in the example Dustin provided me, let’s say the system thinks Bob (protoss) has a 55% chance of winning against Dave (zerg), who has a 45% chance. Based on other matches, the system also thinks protoss has a 53% chance of winning against the zerg’s 47% chance. Now if Dave loses, the numbers won’t fluctuate much (if at all) since the outcome was expected. If Bob wins though, the system may recalculate that Bob and Dave will each have a 50% chance of winning against one another if they compete again. The system may also now say that protoss are favored by 52% against the zerg’s 48%. What if the Z player had the higher MMR and was favored in skill, but not favored in balance, and he won? That would be an "expected" outcome for MMR, but what effect would that have on balance? The two stats the system is tracking are independent of each other in that regard. So if the player matchup leads to an expected outcome, the numbers for each player won’t change much. In contrast, if the favored player who won was playing the un-favored race, those race numbers would adjust accordingly. How does the system determine who is expected to win in a matchup when looking to update the racial balance figure? The system determines this by literally analyzing millions of matches, from the first match ever played, to whenever the snapshot is taken. It’s constantly fluctuating and we can pull that data to see its progress, as well as how the meta-game might be changing. The numbers – particularly the numbers for players – will fluctuate a lot at first until the system can more accurately lock on a predictable pattern, similar to the way in which players are placed on the ladder. | ||
|
Cloak
United States816 Posts
| ||
|
Annihilisk
United States6 Posts
I made a post saying this on the official forum and I was BANNED for "Harassment"... apparently of the entire company of Blizzard for calling them stupid! But you should read what some of the Blues say over there in answer to questions and how they contradict themselves multiple times in the same thread... it's just sad. I have no confidence in their statistical or logical abilities. Edit: Maybe that was off topic but I had to vent. To be a little more on-topic: I don't think you can balance the races with statistics across large samples. There are too many variables and circular arguments involved with the matchmaking and player ratings. I think the best option is just to look at the gameplay of pro tournaments and make sure the races are winning around the correct % of the time and that there's no strategies that are clearly imbalanced. One place I do think they could use statistics is looking at games that end quickly so they can nerf early harassment/cheese that's too strong... which I'm guessing is what they did to determine the Terran nerfs for the next patch even though they didn't explain it. | ||
|
mati
Argentina114 Posts
On October 09 2010 06:49 darmousseh wrote: The one complaint i have is how gimmicky protoss seems in air. I've stopped making air units altogether and just use blink stalkers instead. 1. I save money since i never spend resources on that tech. 2. Ground upgrades help everything out of the gateway and robo bay 3. teching to blink -> ht is not nearly as risky and is good against almost all compositions. If i see tanks, i make more stalkers, if i see more bio, i make more ht's. If i see hydras i make collosus or ht, if i see roach i make chargelots. Yes it feel kinda weird, leaving a side the ultra big air units (carriers, BC, broodlords). Air units turn up to be more of support units than anything. With the exception of harrasing mutas. Yes you can sneak some void or banshees... but at the long way, air is just support for the ground battles. And it well may be different, but i dont think is bad, or even a problem. My point, isnt just a "protoss thingy" look more like a "SC2 thingy" :p And back to statistics... You can always get "false conclusion" from then... i can give you many of the data of how i did it, i can always hide a single data, to my advantage and probe almost anything :p... statistics are good to study your own work, but it horrible to probe point to a 3rd party, because they are so easy to manipulate. Alice got red hair and have cancer Brad is red hair and have cancer Jhon is blond hair and NOT have cancer Thomas have red hair and have cancer. Conclusion, having red hair give you cancer... Of course, if some extra data is reveled like... Alice brad n thomas where brother carring a genetical disease... you could also jump to another conclusion about cancer. We cant ever know for sure if blizzard is really given us the data or not... so as mentioned many times... ITS USELESS >.< i got no "math degree" or anything... but im a Biochemicalist (or whatever is typed lol, my english is bad). thats why the cancer example :p, and there are millon of examples of manipulated statistics that end up in false conclusions. | ||
| ||
