|
???
This is horrible. He should include the fallacies that he makes too. There might be 3 things in this that make sense?
Why are we balancing a game around X's in a tournament who are better but underpowered, and who start crushing when it becomes balanced? I don't think Blizzard is...
And how do you know this is due to balance and not metagame shift, or players finally "getting" their races and utilizing what the race has to offer? Well, since your post is all anecdotal...you don't.
this means that players who are better stay in the tournament for longer, therefore they contribute more to the amount of games played
That's true.
since these are the players that overcome balance, again, it skews to 50-50 more than it is
That's not necessarily true.
Stephano is fucking good
True
If they were independent, sample size would be enough by a large margin to say something, but they are not independent. Because you're dealing with players, not just games.
That's a good point.
'Clearly since this is taken from the absolute top, TvZ is highly imbalanced as a thousand games is a very large sample size'. No! you cannot
Explanation? No? Just say something as matter of fact cause it's true? Is he saying MKP's skills is much higher than DRG's or something? Also, why is he using extreme examples that have 0 validity to prove his points? Why doesn't he use actual examples or realistic things? I don't think he can cause they wouldn't support his argument...
IF a ZvP game ends in the 0-5 minute range, the chance is 60% it ends in Z's favour.
Now the 'if' is so bloody important here, the game needn't end.
He goes on to describe the process of 7 pooling and the events following.
Isn't the goal for Blizzard to make the game balanced from start to end? Of course, that's what this whole new Call o Action: Antiga Shipyard is about. Minor tweaks to make vZ more balanced in early game and they are also looking at PvT endgame. If zerg has a 60% win rate up to the 5 minute mark, as shown by a good set of data (good sample size, players of relative equal skill), then the match up is imbalanced during that time frame, but perhaps not overall. If you saw that 7 pools, and any strategy were 50% effective across the entire course of the game, then you have a successfully balanced game. Obviously if Zerg wins 75% of games that end before the 5 minute mark, early game balance is an issue. If they win much less than 50%, then it's imbalanced in the opposite direction.
I'm not really sure what he is saying in this last part. He anecdotally describes a 7 pool, a P potentially scouting it and walling, and the result following the Z's drone deficit. I don't think this story gives us any insight on balance, or on anything that he's arguing. What IS he arguing? I think his mistake is thinking that games ending before the 5 minute mark aren't subjected to analysis, because they are early pools so balance isn't an issue, because the Win/Loss is due to player mistakes??? No matter if it's a mistake, an inability to scout, or whatever...if the results heavily favor one side, then some mechanism of compensation is needed. Ideally, you are looking for 50% win ratio across the entire game time and between all combination of races.
I'm not a statistician...but i'm guessing this guy isn't either...?
Also...I don't think Blizzard makes all of it's decisions based on tournaments, where like said here, player skill is a factor.
|
He's right on what he said, not sure if people didn't know this already though...
|
because reddit is so much better. Lol.
i wouldnt be surprised if that guy had a tl account and got banned and got so butthurt he posted that.
|
On May 04 2012 09:56 corose wrote:
He goes on to describe the process of 7 pooling and the events following.
Isn't the goal for Blizzard to make the game balanced from start to end? Of course, that's what this whole new Call o Action: Antiga Shipyard is about. Minor tweaks to make vZ more balanced in early game and they are also looking at PvT endgame. If zerg has a 60% win rate up to the 5 minute mark, as shown by a good set of data (good sample size, players of relative equal skill), then the match up is imbalanced during that time frame, but perhaps not overall. If you saw that 7 pools, and any strategy were 50% effective across the entire course of the game, then you have a successfully balanced game. Obviously if Zerg wins 75% of games that end before the 5 minute mark, early game balance is an issue. If they win much less than 50%, then it's imbalanced in the opposite direction.
I'm not really sure what he is saying in this last part. He anecdotally describes a 7 pool, a P potentially scouting it and walling, and the result following the Z's drone deficit. I don't think this story gives us any insight on balance, or on anything that he's arguing. What IS he arguing? I think his mistake is thinking that games ending before the 5 minute mark aren't subjected to analysis, because they are early pools so balance isn't an issue, because the Win/Loss is due to player mistakes??? No matter if it's a mistake, an inability to scout, or whatever...if the results heavily favor one side, then some mechanism of compensation is needed. Ideally, you are looking for 50% win ratio across the entire game time and between all combination of races.
I'm not a statistician...but i'm guessing this guy isn't either...?
Also...I don't think Blizzard makes all of it's decisions based on tournaments, where like said here, player skill is a factor.
He is saying that seeing high Zerg win rates in the first 5 minutes and assuming early pools are imbalanced is wrong because games won by Zergs with early pools are often decided with in 5 minutes, but games won by Protoss against early pools often last longer than 5 minutes.
It has nothing to do with making mistakes/better player/whatever. It's just winrate graphs showing very incomplete, limited analysis of balance that are often very misleading.
|
While I agree that there are significant issues with the way that many statistics on TL are presented - I've posted on this before and I was the guy who nudged the monthly winrates graphs to add error bars in the first place - you're not doing the discourse any favors by reposting this, I think. There are hardly any statistical arguments actually made in the post - for instance:
On May 04 2012 09:00 Cyberonic wrote:
The TLPD winrate graphs are praetentious and amateuristic, sorry to say it but that's how it is, the error bars there are pure bollocks and are calculated using the rules of independent probability experiments, that is to say, it is assumed that the results of every series has no effect on the others, as if you flip a coin. If they were independent, sample size would be enough by a large margin to say something, but they are not independent. Because you're dealing with players, not just games. Good players simply ruin the idea of independent experiments.
There is something deeply hypocritical about decrying statistics discussions on TL for being superficial and then totally failing to present statistical evidence for your assertion that games outcomes are not independent. One would think that the actual mathematics would be pretty trivial, so simply asserting that "they are not independent because they are players" is committing exactly the sin that you're supposedly railing against.
Please reconsider reposting topics like this in the future, or at the very least, try to be productive and rigorous in your arguments if you truly want TL to be a community that is rigorous in its discussion!
|
On May 04 2012 10:05 Reptilia wrote:
because reddit is so much better. Lol.
i wouldnt be surprised if that guy had a tl account and got banned and got so butthurt he posted that.
Did you read anything past the qualifier that was added on after his original post on Reddit?
Doesn't look like it. Your post has nothing to do with the vast majority of his post. Or where you saying he got butthurt that he got banned from TL so he went to Reddit and wrote an intelligent post about balance statistics and how they can be misleading?
All these douchey little TL > Reddit posts are the type of annoying shit that makes people think TL is pretentious anyways. I know he isn't any better for his equally douchey qualifier, but at least he followed it up with an informative, well-written post. Something I haven't seen much of from the Reddit bashers here. At least on Reddit your pathetic contributions would be downvoted enough so that I wouldn't have to waste time responding to them and could help clarify things for people who give a shit about having a meaningful discussion.
|
All of what OP says about statistics is right -- that his tone is arrogant does not jeopardize the merit of this argument a bit.
Is he tactless in his approach? Probably. Is he right about statistics? Surely.
Don't be put off by the qualifier right away. That only proves TL to be not open-minded to the truth, no matter how harshly stated.
|
On May 04 2012 10:08 shaldengeki wrote:While I agree that there are significant issues with the way that many statistics on TL are presented - I've posted on this before and I was the guy who nudged the monthly winrates graphs to add error bars in the first place - you're not doing the discourse any favors by reposting this, I think. There are hardly any statistical arguments actually made in the post - for instance: Show nested quote +On May 04 2012 09:00 Cyberonic wrote:
The TLPD winrate graphs are praetentious and amateuristic, sorry to say it but that's how it is, the error bars there are pure bollocks and are calculated using the rules of independent probability experiments, that is to say, it is assumed that the results of every series has no effect on the others, as if you flip a coin. If they were independent, sample size would be enough by a large margin to say something, but they are not independent. Because you're dealing with players, not just games. Good players simply ruin the idea of independent experiments.
There is something deeply hypocritical about decrying statistics discussions on TL for being superficial and then totally failing to present statistical evidence for your assertion that games outcomes are not independent. One would think that the actual mathematics would be pretty trivial, so simply asserting that "they are not independent because they are players" is committing exactly the sin that you're supposedly railing against.
I don't agree with you at all.
I think it is pretty clear that game results are not independent outcomes. Consider a 10game match between DRG and Joe "Code B Protoss" Schmoe.
DRG wins the first 9 games. Any rational, logical observer would favor him greatly to win the 10th game, right? But winrate graphs still assume the outcome of Game 10 should have a 50/50 chance of going either way, like it's just a coin flip. Now, in massive sample sizes this would be corrected by enough players from every race being better than their opponent in any series so that it would smooth out any errors, but in month long samples from a tiny group of pros the deviations don't get corrected.
I think the math for showing that is extremely hard, but the logic behind it is very strong. Similar to how I'm sure you won't debate that 2+2 is valid, but you would have a pretty damn hard time mathematically proving addition to me.
|
A college drop out lecturing people on latin, how all social science are inherently pseudoscience, and statistics has no place in anything but the 'real sciences'. This is one moment where I wish reddit has a serious moderation policy.
|
Okay, I wanted to stay away from this site, but I couldn't let some stuff go unanswered, I am drab, anyone can message me on reddit to verify this.
On May 04 2012 09:40 windsupernova wrote:
As much as I agree with him in some points. I don't like how he comes off as someone pretty arrogant and doesn't even present some kind of credentials on why he understand statistics more than 99% of people.I mean for all we know he could be some arrogant College kid who just passed his 1st statistics class. I'm not making an argument from authority, I don't need credentials, even if I was a cow or an anencephalic protozoan, it doesn't matter, there is no need for credentials because I'm making an argument from reason, not from authority, I do not even need to cite any sources because my argument is purely rational, not empirical. If you ask for 'credentials' to verify this post then you lost and don't know how to verify academic literature.
My credentials are irrelevant, I'm not making an argument from authority. If you do not find yourself to have the confidence to check the correctness of my argument then you shouldn't agree or disagree either way. Say to yourself 'I don't understand what he's saying', above all, don't comment on a thread whose opening you don't understand, and move on with life.
Yes, I am very smug, I'm not even smug, I'm condescending, I'm not condescending because I have a higher education, I'm condescending because I'm fed up with stupidity, the arguments I put out are very easy and basic to understand and honestly, anyone reading those graphs should come to those conclusions, yet I've seen countless and countless people misinterpreting all those graphs without coming to the realization of these very basic givens, on both reddit and TL. I've seen 50 pages of TL posts discussing those graphics about probability of races to win at certain time intervals in matchups, and maybe 1-2 people pointed out how misleading it was because of the arguments I put out, and no one listened and other people go discuss trivial and unrelated stuff like 'sample size' while there are much bigger problems. I've seen the TLPD winrates posted on both TL and reddit and people discussed them for days and so few people initially pointed out that the lines between the graphs in the old aesthetic were completely ludicrous and they should be bars, and even fewer people were critical of the fact that the error bars were calculated by a means which assumes independent experiments, which they are not.
It doesn't take a genius to see this, it just takes allowing yourself to be critical. As soon as I ask a lot of those people 'There are some grave fallacies with those stats,c an you point them out?' they will most likely come with at least 80% of the shit I pointed out and probably with some things I overlooked. It doesn't take a brain, it takes not being a mindless drone and being critical of stuff that is being posted. As for credentials, I guarantee you that the people who post those TLPD winrate graphs either have no statistical credentials, or are wilfully lying to people and oversimplifying it, because it's just statistical faux pas.
But then he doesn't say how we should go about interpreting those statistics and providing proof. We should interpret them as what they are. They are the winrates for this month, it says nothing of balance or any other interpretations you can make of them. You see what you get, and the error bars are, simply put, incorrect and a statistical gaffe. I'm not sure what they are supposed to mean, they don't mean anything if the map scores aren't independent probability experiments.
That being said I do think most of the people take a really simplistic approach to statistics, but well statistics are a hard subject to tackle Nope, it's very easy, it's more that people like to see things that you can't conclude from stuff.
On May 04 2012 10:09 LaM wrote:Show nested quote +On May 04 2012 10:05 Reptilia wrote:
because reddit is so much better. Lol.
i wouldnt be surprised if that guy had a tl account and got banned and got so butthurt he posted that. Did you read anything past the qualifier that was added on after his original post on Reddit? Doesn't look like it. Your post has nothing to do with the vast majority of his post. Or where you saying he got butthurt that he got banned from TL so he went to Reddit and wrote an intelligent post about balance statistics and how they can be misleading? All these douchey little TL > Reddit posts are the type of annoying shit that makes people think TL is pretentious anyways. I know he isn't any better for his equally douchey qualifier, but at least he followed it up with an informative, well-written post. Something I haven't seen much of from the Reddit bashers here. At least on Reddit your pathetic contributions would be downvoted enough so that I wouldn't have to waste time responding to them and could help clarify things for people who give a shit about having a meaningful discussion.
As linked in the OP, I did not add that qualifier on top myself, I never added the qualifier formally, someone asked me 'Have you posted it on TL' (the OP here), I said 'Nope', he asked 'why?', I said that which he quoted.
That said, I never mentioned TL in the original post, I was mainly critical of screddit and its continued misuse of statistics and it got upvoted to be the #1 post on the screddit first page. This exemplifies a quality of screddit that I feel TL heavily lacks.
Edit: Also:
Pepper_MD just sent you a month of reddit gold! Wasn't that nice?
Here's a note that was included:
I have degree in Stats. All I have to say is Thank You.
I have no idea what reddit gold is, is it good?
User was banned for this post.
|
On May 04 2012 10:13 Quochobao wrote:
All of what OP says about statistics is right -- that his tone is arrogant does not jeopardize the merit of this argument a bit.
Is he tactless in his approach? Probably. Is he right about statistics? Surely.
Don't be put off by the qualifier right away. That only proves TL to be not open-minded to the truth, no matter how harshly stated.
But the thing is, what he says in his arrogant manner ("99% of the people don't know this") is basically common knowledge.
Is it true? For the most part yes. Is it advanced mathematics or anything ground-breaking, thought-provocing or fallacy-revealing? No. He's just arguing common sense against an imagined 99%.
|
On May 04 2012 10:17 LaM wrote:Show nested quote +On May 04 2012 10:08 shaldengeki wrote:While I agree that there are significant issues with the way that many statistics on TL are presented - I've posted on this before and I was the guy who nudged the monthly winrates graphs to add error bars in the first place - you're not doing the discourse any favors by reposting this, I think. There are hardly any statistical arguments actually made in the post - for instance: On May 04 2012 09:00 Cyberonic wrote:
The TLPD winrate graphs are praetentious and amateuristic, sorry to say it but that's how it is, the error bars there are pure bollocks and are calculated using the rules of independent probability experiments, that is to say, it is assumed that the results of every series has no effect on the others, as if you flip a coin. If they were independent, sample size would be enough by a large margin to say something, but they are not independent. Because you're dealing with players, not just games. Good players simply ruin the idea of independent experiments.
There is something deeply hypocritical about decrying statistics discussions on TL for being superficial and then totally failing to present statistical evidence for your assertion that games outcomes are not independent. One would think that the actual mathematics would be pretty trivial, so simply asserting that "they are not independent because they are players" is committing exactly the sin that you're supposedly railing against. I don't agree with you at all. I think it is pretty clear that game results are not independent outcomes. Consider a 10game match between DRG and Joe "Code B Protoss" Schmoe. DRG wins the first 9 games. Any rational, logical observer would favor him greatly to win the 10th game, right? But winrate graphs still assume the outcome of Game 10 should have a 50/50 chance of going either way, like it's just a coin flip. Now, in massive sample sizes this would be corrected by enough players from every race being better than their opponent in any series so that it would smooth out any errors, but in month long samples from a tiny group of pros the deviations don't get corrected. I think the math for showing that is extremely hard, but the logic behind it is very strong. Similar to how I'm sure you won't debate that 2+2 is valid, but you would have a pretty damn hard time mathematically proving addition to me.
I think you're probably mistaking what the purpose of the winrate charts is, and what "independent outcomes" means in the context of repeated experiments. Of course you wouldn't apply the winrate charts to the situation you're describing - what they do is aggregate results across several skill levels and regions to provide a general indicator of race balance. Nobody is claiming that every single game between a protoss and a zerg has a 50/50 chance of going either way, and if this is how you're interpreting the winrate charts, that's definitely a problem on your end!
The issue you describe with skill impacting win chances is actually not an issue of independent events at all. If the events were dependent, then the results from all prior games between all zergs and all protosses would impact the win probability of the next game between a zerg and a protoss. This is not the issue at hand in your scenario, where you're talking about skill level of each player impacting win probabilities. That's the realm of ELO, and the winrate charts make no attempt at gauging the skill levels of each player.
|
It is annoying when people look at winrates and automatically assume X race is OP. However, the condescending tone of the post will probably a turn off for most people and they will continue to believe in these statistics instead of actually playing the game.
|
On May 04 2012 10:27 shaldengeki wrote:Show nested quote +On May 04 2012 10:17 LaM wrote:On May 04 2012 10:08 shaldengeki wrote:While I agree that there are significant issues with the way that many statistics on TL are presented - I've posted on this before and I was the guy who nudged the monthly winrates graphs to add error bars in the first place - you're not doing the discourse any favors by reposting this, I think. There are hardly any statistical arguments actually made in the post - for instance: On May 04 2012 09:00 Cyberonic wrote:
The TLPD winrate graphs are praetentious and amateuristic, sorry to say it but that's how it is, the error bars there are pure bollocks and are calculated using the rules of independent probability experiments, that is to say, it is assumed that the results of every series has no effect on the others, as if you flip a coin. If they were independent, sample size would be enough by a large margin to say something, but they are not independent. Because you're dealing with players, not just games. Good players simply ruin the idea of independent experiments.
There is something deeply hypocritical about decrying statistics discussions on TL for being superficial and then totally failing to present statistical evidence for your assertion that games outcomes are not independent. One would think that the actual mathematics would be pretty trivial, so simply asserting that "they are not independent because they are players" is committing exactly the sin that you're supposedly railing against. I don't agree with you at all. I think it is pretty clear that game results are not independent outcomes. Consider a 10game match between DRG and Joe "Code B Protoss" Schmoe. DRG wins the first 9 games. Any rational, logical observer would favor him greatly to win the 10th game, right? But winrate graphs still assume the outcome of Game 10 should have a 50/50 chance of going either way, like it's just a coin flip. Now, in massive sample sizes this would be corrected by enough players from every race being better than their opponent in any series so that it would smooth out any errors, but in month long samples from a tiny group of pros the deviations don't get corrected. I think the math for showing that is extremely hard, but the logic behind it is very strong. Similar to how I'm sure you won't debate that 2+2 is valid, but you would have a pretty damn hard time mathematically proving addition to me. I think you're probably mistaking what the purpose of the winrate charts is, and what "independent outcomes" means in the context of repeated experiments. Of course you wouldn't apply the winrate charts to the situation you're describing - what they do is aggregate results across several skill levels and regions to provide a general indicator of race balance. Nobody is claiming that every single game between a protoss and a zerg has a 50/50 chance of going either way, and if this is how you're interpreting the winrate charts, that's definitely a problem on your end! The issue you describe with skill impacting win chances is actually not an issue of independent events at all. If the events were dependent, then the results from all prior games between all zergs and all protosses would impact the win probability of the next game between a zerg and a protoss. This is not the issue at hand in your scenario, where you're talking about skill level of each player impacting win probabilities. That's the realm of ELO, and the winrate charts make no attempt at gauging the skill levels of each player.
And you are again making the mistake that winrate charts indicate win probabilities and balance, which is the whole point, THEY DON'T!
You wanted error bars added to winrate charts? Why? What error? They are cataloging winrates from the past month, where is the error coming from?
My mistake was in even ceding that error bars should be part of the chart and make any sense with them. They don't. I agree my explanation isn't applicable to the charts, but that isn't because my explanation is assuming things incorrectly, it's because the error bars shouldn't be there in the first place...
|
This is a pretty clear example of how you say things being as important as what you say. The content of the post is very good, correctly pointing out the people really like to conclude things from statistics that are totally not what the statistics say. It would make a much more compelling argument if it wasn't written by such an asshole.
|
I prefer IdrA's way of saying this:
if statistics get you hard make one of those ladder analysis pages or something, but stop interfering with balance discussions.
|
On May 04 2012 09:26 dmasterding wrote:
Did any of you guys actually read the thing? He didn't actually give any opinions about the matchups, he was just trying to get rid of some misconceptions people had about interpretation of results. I am pretty sure that if the OP never mentioned this person was from r/SC you guys wouldn't be so biased against the author.
Frankly, he was asking for the bias by being biased against TL.
He didn't say anything new or ground breaking. I thought this was supposed to be some statistical elephant in the room where some expert of statistics would show us something very important. Instead, it basically said, "don't make poor assumptions." Well, good advice, I say.
|
This reminds of an old point that a lot of old brood war veterans like Artosis used to make about how the older maps were actually imbalanced despite the winrates they had because at the time dominant and superior players like boxer, iloveoov, and nada were skewing the results to the point where the map statistic should have been more than 50% win rate for terran because those players were much better than their opponents and had higher than 50% win rates and there weren't enough games from other average or less skilled players to reasonably counteract that fact. This is kind of related to how some maps were balanced at the top korean level for certain matchups, but not balanced for a top foreigner level. Essentially the balance in brood war and in brood war maps was relevant to the skill of the players at the time. Older maps might have sported balanced win rates only because of the strategy and skill of the players at the time. Applying modern strategies and the improved skill of players can suddenly reveal undiscovered imbalance.
I'm not saying whether or not that old point is correct, but it certainly had better concrete concepts than whatever this guy seems to be pointing out.
This extends even further, most tournaments have qualifiers, so say X is underpowered, the players who play X that get into the tournament are simply better because thety got in despite the imbalance, therefore as they are better, they will continue to win even despite the imbalance vested against them, thereby skewing the results to more 50-50 than it actually is.
This is kind of the bonjwa terran balance argument for old broodwar maps. However a lot of people didn't agree with this because they didn't believe the bonjwas were really that statistically significant. This causes issues when you talk about the sample size not being large enough.
If the sample size truly isn't large enough to make star players statistically insignificant then you would need to be able to identify "bonjwas" or something similar in player dominance to even hint that one race is underpowered with a 50% winrate. If you can't do that then you are making a judgement call that a few star players that play a specific race are that much superior to everyone else without winning enough to actually prove it. Without good proof the only other scenario you could argue is that it isn't the game that is balanced, but just the number of star players playing for each race (Which is silly because of the bizarre concept of improvement).
|
On May 04 2012 10:35 LaM wrote:Show nested quote +On May 04 2012 10:27 shaldengeki wrote:On May 04 2012 10:17 LaM wrote:On May 04 2012 10:08 shaldengeki wrote:While I agree that there are significant issues with the way that many statistics on TL are presented - I've posted on this before and I was the guy who nudged the monthly winrates graphs to add error bars in the first place - you're not doing the discourse any favors by reposting this, I think. There are hardly any statistical arguments actually made in the post - for instance: On May 04 2012 09:00 Cyberonic wrote:
The TLPD winrate graphs are praetentious and amateuristic, sorry to say it but that's how it is, the error bars there are pure bollocks and are calculated using the rules of independent probability experiments, that is to say, it is assumed that the results of every series has no effect on the others, as if you flip a coin. If they were independent, sample size would be enough by a large margin to say something, but they are not independent. Because you're dealing with players, not just games. Good players simply ruin the idea of independent experiments.
There is something deeply hypocritical about decrying statistics discussions on TL for being superficial and then totally failing to present statistical evidence for your assertion that games outcomes are not independent. One would think that the actual mathematics would be pretty trivial, so simply asserting that "they are not independent because they are players" is committing exactly the sin that you're supposedly railing against. I don't agree with you at all. I think it is pretty clear that game results are not independent outcomes. Consider a 10game match between DRG and Joe "Code B Protoss" Schmoe. DRG wins the first 9 games. Any rational, logical observer would favor him greatly to win the 10th game, right? But winrate graphs still assume the outcome of Game 10 should have a 50/50 chance of going either way, like it's just a coin flip. Now, in massive sample sizes this would be corrected by enough players from every race being better than their opponent in any series so that it would smooth out any errors, but in month long samples from a tiny group of pros the deviations don't get corrected. I think the math for showing that is extremely hard, but the logic behind it is very strong. Similar to how I'm sure you won't debate that 2+2 is valid, but you would have a pretty damn hard time mathematically proving addition to me. I think you're probably mistaking what the purpose of the winrate charts is, and what "independent outcomes" means in the context of repeated experiments. Of course you wouldn't apply the winrate charts to the situation you're describing - what they do is aggregate results across several skill levels and regions to provide a general indicator of race balance. Nobody is claiming that every single game between a protoss and a zerg has a 50/50 chance of going either way, and if this is how you're interpreting the winrate charts, that's definitely a problem on your end! The issue you describe with skill impacting win chances is actually not an issue of independent events at all. If the events were dependent, then the results from all prior games between all zergs and all protosses would impact the win probability of the next game between a zerg and a protoss. This is not the issue at hand in your scenario, where you're talking about skill level of each player impacting win probabilities. That's the realm of ELO, and the winrate charts make no attempt at gauging the skill levels of each player. And you are again making the mistake that winrate charts indicate win probabilities and balance, which is the whole point, THEY DON'T! You wanted error bars added to winrate charts? Why? What error? They are cataloging winrates from the past month, where is the error coming from? My mistake was in even ceding that error bars should be part of the chart and make any sense with them. They don't. I agree my explanation isn't applicable to the charts, but that isn't because my explanation is assuming things incorrectly, it's because the error bars shouldn't be there in the first place...
While that's exactly the point, they don't indicate balance, everyone seems to completely ignore this fact, and treat them as irrefutable evidence of "P is UP, T is OP, because they have been losing/winning, look at the graph, it's obvious."
When people start looking past the stats and actually thinking, you can use the winrates as just another fun useless fact
|
On May 04 2012 10:35 LaM wrote:Show nested quote +On May 04 2012 10:27 shaldengeki wrote:On May 04 2012 10:17 LaM wrote:On May 04 2012 10:08 shaldengeki wrote:While I agree that there are significant issues with the way that many statistics on TL are presented - I've posted on this before and I was the guy who nudged the monthly winrates graphs to add error bars in the first place - you're not doing the discourse any favors by reposting this, I think. There are hardly any statistical arguments actually made in the post - for instance: On May 04 2012 09:00 Cyberonic wrote:
The TLPD winrate graphs are praetentious and amateuristic, sorry to say it but that's how it is, the error bars there are pure bollocks and are calculated using the rules of independent probability experiments, that is to say, it is assumed that the results of every series has no effect on the others, as if you flip a coin. If they were independent, sample size would be enough by a large margin to say something, but they are not independent. Because you're dealing with players, not just games. Good players simply ruin the idea of independent experiments.
There is something deeply hypocritical about decrying statistics discussions on TL for being superficial and then totally failing to present statistical evidence for your assertion that games outcomes are not independent. One would think that the actual mathematics would be pretty trivial, so simply asserting that "they are not independent because they are players" is committing exactly the sin that you're supposedly railing against. I don't agree with you at all. I think it is pretty clear that game results are not independent outcomes. Consider a 10game match between DRG and Joe "Code B Protoss" Schmoe. DRG wins the first 9 games. Any rational, logical observer would favor him greatly to win the 10th game, right? But winrate graphs still assume the outcome of Game 10 should have a 50/50 chance of going either way, like it's just a coin flip. Now, in massive sample sizes this would be corrected by enough players from every race being better than their opponent in any series so that it would smooth out any errors, but in month long samples from a tiny group of pros the deviations don't get corrected. I think the math for showing that is extremely hard, but the logic behind it is very strong. Similar to how I'm sure you won't debate that 2+2 is valid, but you would have a pretty damn hard time mathematically proving addition to me. I think you're probably mistaking what the purpose of the winrate charts is, and what "independent outcomes" means in the context of repeated experiments. Of course you wouldn't apply the winrate charts to the situation you're describing - what they do is aggregate results across several skill levels and regions to provide a general indicator of race balance. Nobody is claiming that every single game between a protoss and a zerg has a 50/50 chance of going either way, and if this is how you're interpreting the winrate charts, that's definitely a problem on your end! The issue you describe with skill impacting win chances is actually not an issue of independent events at all. If the events were dependent, then the results from all prior games between all zergs and all protosses would impact the win probability of the next game between a zerg and a protoss. This is not the issue at hand in your scenario, where you're talking about skill level of each player impacting win probabilities. That's the realm of ELO, and the winrate charts make no attempt at gauging the skill levels of each player. And you are again making the mistake that winrate charts indicate win probabilities and balance, which is the whole point, THEY DON'T! You wanted error bars added to winrate charts? Why? What error? They are cataloging winrates from the past month, where is the error coming from? My mistake was in even ceding that error bars should be part of the chart and make any sense with them. They don't. I agree my explanation isn't applicable to the charts, but that isn't because my explanation is assuming things incorrectly, it's because the error bars shouldn't be there in the first place...
Please remain calm. I'd love to have a level-headed discussion with you!
The winrate charts indicate win probabilities aggregated across each race. This is indisputable. They provide what I believe is a general indicator of balance - I don't believe that there are statistically-significant differences in skill between races, so it stands to reason that in the aggregate, this provides some information on the balance between races.
The error bars allow you to determine whether one month's average is significantly different from previous months. This is hugely important as before their addition people were making all sorts of wild claims as to how certain patches were throwing race balance off. Now that we can determine whether or not each month was significantly-different from previous months, we can more reasonably talk about whether or not changes to the game are having effects on winrates.
|
|
|
|
|
|
EPT
