|
This post is originally written by Drabzalver on Reddit. Since he does not have a TL account I asked and was allowed to repost if I include this:
Drabzalver on Reddit:TL is the [swear word] of praetentious [swear word] where moderators reward supposed 'high quality posts' which are full of statistical and scientific garbage I just outlined just because they are 'praesented nicely' and the mods basically think that any long post with a lot of images and no swear words is intellectually advanced while often a lot of it is total garbage filled with wrong interpretations and grave statistical errors.
Also, I don't have a TL account, you're welcome to repost, but do include this qualifier, should be fun,
Clarification: This is not my opinion of this community and I only posted the quote because I was asked to and respect his author rights. The content is very interesting.
EDIT: A lot of people claim the OP/author just wants to bash the TL community. That is wrong! The text is directed towrds the reddit Community and was originally called "What reddit doesn't know about statistics" with me liberately altering the title... I see this might have let to some confusion.
-----------------------------------------------------------------------------------------------------------------------------------
What you don't know about statistics
by Drabzalver
Not a day goes by without some stat or graph being posted here or whenever in the community, 99% of the time this deals with winrates and balance, 99% of the people commenting on it have no clue how to interpret these numbers and how little they actually mean, and no, this has nothing to do with sample size. Stats and probability theory is actually something you study on for years and is a specialized field to avoid people from making the errors people make here. So stay a while and listen:
- Fallacy: winrates indicate balance
Yes and no, for the most part, they do not indicate balance, rather, they indicate balance shifts. It's so trivial to see that it boggles me that people don't figure this out on their own. Assume that race X was actually underpowered last month and is balanced this month. This means that the X players who actually qualified last month and stayed in tournaments are actually better than the Y and Z players, therefore, now that it gets balanced, as they are overal better, they start smashing Y and Z because they are better, thereby suddenly making the graph appear as X has been 'overbuffed' or whatever else while simply the few X players that were around in the scene were better. This will continue on until the mediocre Y and Z players who were more so carried by their race than the X players get weeded out.
This extends even further, most tournaments have qualifiers, so say X is underpowered, the players who play X that get into the tournament are simply better because thety got in despite the imbalance, therefore as they are better, they will continue to win even despite the imbalance vested against them, thereby skewing the results to more 50-50 than it actually is.
In fact, there are even more things wrong with this idea, because tournaments generally feature some form of elimination, this means that players who are better stay in the tournament for longer, therefore they contribute more to the amount of games played, since these are the players that overcome balance, again, it skews to 50-50 more than it is.
So yes, no matter how you have it, balance will skew to 50-50 more than it actually is, and monthily spikes and variation indicate balance shift more so than absolute balance. The only way to find out absolute balance is to get a random pool of pros, force them to play random, and have a round robin tournament to ensure that everyone plays the same amount of games. Very unfeasible to get enough games with that for a reasonable sample size.
- Fallacy: sample size is big enough
The TLPD winrate graphs are praetentious and amateuristic, sorry to say it but that's how it is, the error bars there are pure bollocks and are calculated using the rules of independent probability experiments, that is to say, it is assumed that the results of every series has no effect on the others, as if you flip a coin. If they were independent, sample size would be enough by a large margin to say something, but they are not independent. Because you're dealing with players, not just games. Good players simply ruin the idea of independent experiments. The fact that Stephano won 4-0 this time and 4-0 the last time is not independent, they have a common cause, Stephano is fucking good. More so than that, as indicated before, most tournaments feature a form of elimination, which means that because Stephano is fucking good he simply contributes more to the sample size than players who are not as good. One has to realize that in some single elimination tournaments, it's possible for the champion to have played 40% of all the games in the tournament...
As an extreme example to show that the idea is fundamentally flawed. Say you have 2 and only 2 players, the best 2 in the world, let's say you have MKP vs DRG, they play a thousand games, this ends 720-280 in MKP's favour. Can you then say 'Clearly since this is taken from the absolute top, TvZ is highly imbalanced as a thousand games is a very large sample size'. No! you cannot, and while this is an extreme example it shows the general idea and the fallacy thereof, the games are not independent probability experiments.
Especially in Korea, it is very likely that the TLPD graphs we're all so fond of do not indicate balance overall, they just indicate whichever race has a couple of good players this month that dominated everything. The Korean graph just shifts around every month while the amount of games should be large enough to stop that from happening, if they were independent experiments, but they aren't, they are quite dependent and how much the KR graph flips around each months demonstrates how unreliable it is.
Simply put, the amount of games has a large enough sample size to be significant, but the amount of players is way too small.
- Fallacy: advantages at certain times of matchups expressed in graphs
There are also a lot of graphs posted which supposedly indicate that some races may have an advantage at certain matchups. Oh boy do people misread what these graphs mean. Take this bad boy.
A naïve way to read it would simply be 'Hmm, Z has an early game advantage in ZvP, then it becomes about even, then P has a slight advantage up to the end, then Z again.', wrong; look closely, what does the graph actually say, it says this:
IF a ZvP game ends in the 0-5 minute range, the chance is 60% it ends in Z's favour.
Now the 'if' is so bloody important here, the game needn't end. Now, everyone of course realizes that that part is caused by early pools. Does Zerg really have a large advantage at that point? Can Zerg force a win at that point if they want to, are early pools overpowered? No, not at all, so what is going on?
Imagine a ZvP, Z decides to 7pool, P doesn't scout soon enough, lings get in, kill every probe, traalalala, P GG's. Game over in the first 5 minutes in Z's favour.
Okay, imagine a ZvP, Z decides to 7pool, P scouts in time, gets his wall up, damn, Z's like 'fuck man, shouldn't have done that'. But not necesarily GG's unless IdrA, the game goes on, Z however plays at such a disadvantage that in the next 5-10 minutes surely P will claim his victory unless P messes up.
See the fallacy? That Z has that 'early game advantage' doesn't mean that Z is more powerful or that 7pools are too powerful, it just means that IF the game immediately ends due to a 7pool it will most likely be in Z's favour. If the 7pool fails, the game doesn't end at that point, Z will most likely stay in the game and play from a significant disadvantage to lose later.
It is a grave statistical error of the magnitude of interpreting 'If a 8 year old child dies, the chance is the greatest he dies from a car swoop' as 'It is very likely 8 year old children die from car swoops'.
The graph doesn't even say how likely it is that the game ends at certain intervals. For all you it's far more likely for P to win in the late game than in the mid game, even though the graph indicates that if the game ends in the late game, the chance is higher that Z takes it. And even so, that still says nothing about advantages of races at certain times. One would assume that if a race is likely to win at time X, that race enjoys an advantage slightly before that time, no?
What would be far more intersting, though also not conclusive, would be a graph which outlines 'How large was the percentage of Z wins in ZvP at each interval', which is fundamentally different from 'at each interval, if the game ends, how often does it end into Z's favour in ZvP'. My bet is that because 7pools are actually quite rare, it would not at all show the huge spike for Z in the early game.
|
praetentious?
really?
i hope that how he spelt it too, because it would be really funny if he wrote this much about people being pretentious and used big words that he couldn't even spell.
|
None of this is new or even in-depth. He starts of by saying that probability and stats are a filed that people study and continues on to bring arguments anybody could make. Not really proving a point there.
Not that I'm saying he's necessarily wrong, but having such an arrogant tone and then bringing something this trivial to the table is a bit... pretentious.
|
reddit is a cesspool of memes and retardation
that being said, i love the job the mods do on these forums.
as far as stats etc go i really dont pay any mind to them. the only ones i think have any validity would be the GSL player stats because they are tournament level and reliable.
|
On May 04 2012 09:09 halfies wrote:
praetentious?
really?
i hope that how he spelt it too, because it would be really funny if he wrote this much about people being pretentious and used big words that he couldn't even spell.
Well he explains...
it's worse, he spelled it that way intentionally. still, he makes some good points in the substance of what he says.
|
So basically the person who came up with this thinks that he is smarter than 99% of everyone else and then proceeds to list "fallacies" which tend to skew game results. Then he explains that ZvP ends early more often in favor of the zerg because of early pools.
I don't really see the value in this. It only shows what we already knew, that statistics aren't always true due to a couple factors. He also calls us pretentious but spells it oddly :/. The author himself strikes me as very pretentious.
|
Yes, statistics taken out of context and without understanding of what they are suppose to prove makes them misleading. Good for reddit that they finally understand this 
EDIT. @Chocolate: there is no value in the post, the poster just wanted to bash on TL.Net community and used statistics as an excuse.
|
On May 04 2012 09:13 imMUTAble787 wrote:
reddit is a cesspool of memes and retardation
that being said, i love the job the mods do on these forums.
as far as stats etc go i really dont pay any mind to them. the only ones i think have any validity would be the GSL player stats because they are tournament level and reliable.
Please stop bashing the r/sc forums if you don't actually know what goes on over there. That was a lot more prevalent before, but even if you look at the front page right now you can see that there's a lot of starcraft legitimate related content and not just random shitty memes.
|
Sorry he's theory is totally garbage as he thinks the skill gap between the races are massive(it's minimum) and screws with the statistic.
|
Did any of you guys actually read the thing? He didn't actually give any opinions about the matchups, he was just trying to get rid of some misconceptions people had about interpretation of results. I am pretty sure that if the OP never mentioned this person was from r/SC you guys wouldn't be so biased against the author.
|
While tournaments are indeed not reliable to calculate balance statistics because of the small pool and elimination process as Drabzalver said; tournaments and statistics still contain quite useful information. Many people might not interpret or analyze the numbers correctly but that doesn't mean that the numbers are useless.
|
On May 04 2012 09:26 dmasterding wrote:
Did any of you guys actually read the thing? He didn't actually give any opinions about the matchups, he was just trying to get rid of some misconceptions people had about interpretation of results. I am pretty sure that if the OP never mentioned this person was from r/SC you guys wouldn't be so biased against the author.
I think so, too... There's no reason for bashing around. I wrote why I included his text but I'd rather have a discussion on the topic and not its preface.
|
At least a very interesting read.
|
IdrA has been always been referring to point 1 and 2 when asked about winrates.
|
As much as I agree with him in some points. I don't like how he comes off as someone pretty arrogant and doesn't even present some kind of credentials on why he understand statistics more than 99% of people.I mean for all we know he could be some arrogant College kid who just passed his 1st statistics class.
Arguing from authority only works if you prove you are have some kind of authority. But his arguments are nice and he does seems to have some kind of understanding of statistics. But then he doesn't say how we should go about interpreting those statistics and providing proof.
That being said I do think most of the people take a really simplistic approach to statistics, but well statistics are a hard subject to tackle
|
On May 04 2012 09:13 imMUTAble787 wrote:
reddit is a cesspool of memes and retardation
that being said, i love the job the mods do on these forums.
as far as stats etc go i really dont pay any mind to them. the only ones i think have any validity would be the GSL player stats because they are tournament level and reliable.
Maybe you should read the OP instead of just the qualifier.
|
On May 04 2012 09:23 Trumpstyle wrote:
Sorry he's theory is totally garbage as he thinks the skill gap between the races are massive(it's minimum) and screws with the statistic.
He makes literally 0 claims about the skill gap between races. Reread what he said.
|
Russian Federation396 Posts
i dont understand the point of this... most of this is common sense yet it states that 99% of people dont know this?
really?
|
On May 04 2012 09:26 dmasterding wrote:
Did any of you guys actually read the thing? He didn't actually give any opinions about the matchups, he was just trying to get rid of some misconceptions people had about interpretation of results. I am pretty sure that if the OP never mentioned this person was from r/SC you guys wouldn't be so biased against the author.
I read it. He is extremely aggressive so he begets extremely aggressive answers.
Regardless of his tone and of any tl vs reddit thing brewing here, his points should accompany threads (better phrased, of course) like the TLPD win rates thread in my opinion so that users with no background in statistics (for whatever reason) can have a better grasp of what the data presented means and/or doesn't.
|
If that is a top of the line post from reddit then I'm glad I stick to TL lol.
He just said random stuff in an aggressive way (stuff everyone knows anyway) and called TLPD statistics pretentious? hypocrite much?
|
???
This is horrible. He should include the fallacies that he makes too. There might be 3 things in this that make sense?
Why are we balancing a game around X's in a tournament who are better but underpowered, and who start crushing when it becomes balanced? I don't think Blizzard is...
And how do you know this is due to balance and not metagame shift, or players finally "getting" their races and utilizing what the race has to offer? Well, since your post is all anecdotal...you don't.
this means that players who are better stay in the tournament for longer, therefore they contribute more to the amount of games played
That's true.
since these are the players that overcome balance, again, it skews to 50-50 more than it is
That's not necessarily true.
Stephano is fucking good
True
If they were independent, sample size would be enough by a large margin to say something, but they are not independent. Because you're dealing with players, not just games.
That's a good point.
'Clearly since this is taken from the absolute top, TvZ is highly imbalanced as a thousand games is a very large sample size'. No! you cannot
Explanation? No? Just say something as matter of fact cause it's true? Is he saying MKP's skills is much higher than DRG's or something? Also, why is he using extreme examples that have 0 validity to prove his points? Why doesn't he use actual examples or realistic things? I don't think he can cause they wouldn't support his argument...
IF a ZvP game ends in the 0-5 minute range, the chance is 60% it ends in Z's favour.
Now the 'if' is so bloody important here, the game needn't end.
He goes on to describe the process of 7 pooling and the events following.
Isn't the goal for Blizzard to make the game balanced from start to end? Of course, that's what this whole new Call o Action: Antiga Shipyard is about. Minor tweaks to make vZ more balanced in early game and they are also looking at PvT endgame. If zerg has a 60% win rate up to the 5 minute mark, as shown by a good set of data (good sample size, players of relative equal skill), then the match up is imbalanced during that time frame, but perhaps not overall. If you saw that 7 pools, and any strategy were 50% effective across the entire course of the game, then you have a successfully balanced game. Obviously if Zerg wins 75% of games that end before the 5 minute mark, early game balance is an issue. If they win much less than 50%, then it's imbalanced in the opposite direction.
I'm not really sure what he is saying in this last part. He anecdotally describes a 7 pool, a P potentially scouting it and walling, and the result following the Z's drone deficit. I don't think this story gives us any insight on balance, or on anything that he's arguing. What IS he arguing? I think his mistake is thinking that games ending before the 5 minute mark aren't subjected to analysis, because they are early pools so balance isn't an issue, because the Win/Loss is due to player mistakes??? No matter if it's a mistake, an inability to scout, or whatever...if the results heavily favor one side, then some mechanism of compensation is needed. Ideally, you are looking for 50% win ratio across the entire game time and between all combination of races.
I'm not a statistician...but i'm guessing this guy isn't either...?
Also...I don't think Blizzard makes all of it's decisions based on tournaments, where like said here, player skill is a factor.
|
He's right on what he said, not sure if people didn't know this already though...
|
because reddit is so much better. Lol.
i wouldnt be surprised if that guy had a tl account and got banned and got so butthurt he posted that.
|
On May 04 2012 09:56 corose wrote:
He goes on to describe the process of 7 pooling and the events following.
Isn't the goal for Blizzard to make the game balanced from start to end? Of course, that's what this whole new Call o Action: Antiga Shipyard is about. Minor tweaks to make vZ more balanced in early game and they are also looking at PvT endgame. If zerg has a 60% win rate up to the 5 minute mark, as shown by a good set of data (good sample size, players of relative equal skill), then the match up is imbalanced during that time frame, but perhaps not overall. If you saw that 7 pools, and any strategy were 50% effective across the entire course of the game, then you have a successfully balanced game. Obviously if Zerg wins 75% of games that end before the 5 minute mark, early game balance is an issue. If they win much less than 50%, then it's imbalanced in the opposite direction.
I'm not really sure what he is saying in this last part. He anecdotally describes a 7 pool, a P potentially scouting it and walling, and the result following the Z's drone deficit. I don't think this story gives us any insight on balance, or on anything that he's arguing. What IS he arguing? I think his mistake is thinking that games ending before the 5 minute mark aren't subjected to analysis, because they are early pools so balance isn't an issue, because the Win/Loss is due to player mistakes??? No matter if it's a mistake, an inability to scout, or whatever...if the results heavily favor one side, then some mechanism of compensation is needed. Ideally, you are looking for 50% win ratio across the entire game time and between all combination of races.
I'm not a statistician...but i'm guessing this guy isn't either...?
Also...I don't think Blizzard makes all of it's decisions based on tournaments, where like said here, player skill is a factor.
He is saying that seeing high Zerg win rates in the first 5 minutes and assuming early pools are imbalanced is wrong because games won by Zergs with early pools are often decided with in 5 minutes, but games won by Protoss against early pools often last longer than 5 minutes.
It has nothing to do with making mistakes/better player/whatever. It's just winrate graphs showing very incomplete, limited analysis of balance that are often very misleading.
|
While I agree that there are significant issues with the way that many statistics on TL are presented - I've posted on this before and I was the guy who nudged the monthly winrates graphs to add error bars in the first place - you're not doing the discourse any favors by reposting this, I think. There are hardly any statistical arguments actually made in the post - for instance:
On May 04 2012 09:00 Cyberonic wrote:
The TLPD winrate graphs are praetentious and amateuristic, sorry to say it but that's how it is, the error bars there are pure bollocks and are calculated using the rules of independent probability experiments, that is to say, it is assumed that the results of every series has no effect on the others, as if you flip a coin. If they were independent, sample size would be enough by a large margin to say something, but they are not independent. Because you're dealing with players, not just games. Good players simply ruin the idea of independent experiments.
There is something deeply hypocritical about decrying statistics discussions on TL for being superficial and then totally failing to present statistical evidence for your assertion that games outcomes are not independent. One would think that the actual mathematics would be pretty trivial, so simply asserting that "they are not independent because they are players" is committing exactly the sin that you're supposedly railing against.
Please reconsider reposting topics like this in the future, or at the very least, try to be productive and rigorous in your arguments if you truly want TL to be a community that is rigorous in its discussion!
|
On May 04 2012 10:05 Reptilia wrote:
because reddit is so much better. Lol.
i wouldnt be surprised if that guy had a tl account and got banned and got so butthurt he posted that.
Did you read anything past the qualifier that was added on after his original post on Reddit?
Doesn't look like it. Your post has nothing to do with the vast majority of his post. Or where you saying he got butthurt that he got banned from TL so he went to Reddit and wrote an intelligent post about balance statistics and how they can be misleading?
All these douchey little TL > Reddit posts are the type of annoying shit that makes people think TL is pretentious anyways. I know he isn't any better for his equally douchey qualifier, but at least he followed it up with an informative, well-written post. Something I haven't seen much of from the Reddit bashers here. At least on Reddit your pathetic contributions would be downvoted enough so that I wouldn't have to waste time responding to them and could help clarify things for people who give a shit about having a meaningful discussion.
|
All of what OP says about statistics is right -- that his tone is arrogant does not jeopardize the merit of this argument a bit.
Is he tactless in his approach? Probably. Is he right about statistics? Surely.
Don't be put off by the qualifier right away. That only proves TL to be not open-minded to the truth, no matter how harshly stated.
|
On May 04 2012 10:08 shaldengeki wrote:While I agree that there are significant issues with the way that many statistics on TL are presented - I've posted on this before and I was the guy who nudged the monthly winrates graphs to add error bars in the first place - you're not doing the discourse any favors by reposting this, I think. There are hardly any statistical arguments actually made in the post - for instance: Show nested quote +On May 04 2012 09:00 Cyberonic wrote:
The TLPD winrate graphs are praetentious and amateuristic, sorry to say it but that's how it is, the error bars there are pure bollocks and are calculated using the rules of independent probability experiments, that is to say, it is assumed that the results of every series has no effect on the others, as if you flip a coin. If they were independent, sample size would be enough by a large margin to say something, but they are not independent. Because you're dealing with players, not just games. Good players simply ruin the idea of independent experiments.
There is something deeply hypocritical about decrying statistics discussions on TL for being superficial and then totally failing to present statistical evidence for your assertion that games outcomes are not independent. One would think that the actual mathematics would be pretty trivial, so simply asserting that "they are not independent because they are players" is committing exactly the sin that you're supposedly railing against.
I don't agree with you at all.
I think it is pretty clear that game results are not independent outcomes. Consider a 10game match between DRG and Joe "Code B Protoss" Schmoe.
DRG wins the first 9 games. Any rational, logical observer would favor him greatly to win the 10th game, right? But winrate graphs still assume the outcome of Game 10 should have a 50/50 chance of going either way, like it's just a coin flip. Now, in massive sample sizes this would be corrected by enough players from every race being better than their opponent in any series so that it would smooth out any errors, but in month long samples from a tiny group of pros the deviations don't get corrected.
I think the math for showing that is extremely hard, but the logic behind it is very strong. Similar to how I'm sure you won't debate that 2+2 is valid, but you would have a pretty damn hard time mathematically proving addition to me.
|
A college drop out lecturing people on latin, how all social science are inherently pseudoscience, and statistics has no place in anything but the 'real sciences'. This is one moment where I wish reddit has a serious moderation policy.
|
Okay, I wanted to stay away from this site, but I couldn't let some stuff go unanswered, I am drab, anyone can message me on reddit to verify this.
On May 04 2012 09:40 windsupernova wrote:
As much as I agree with him in some points. I don't like how he comes off as someone pretty arrogant and doesn't even present some kind of credentials on why he understand statistics more than 99% of people.I mean for all we know he could be some arrogant College kid who just passed his 1st statistics class. I'm not making an argument from authority, I don't need credentials, even if I was a cow or an anencephalic protozoan, it doesn't matter, there is no need for credentials because I'm making an argument from reason, not from authority, I do not even need to cite any sources because my argument is purely rational, not empirical. If you ask for 'credentials' to verify this post then you lost and don't know how to verify academic literature.
My credentials are irrelevant, I'm not making an argument from authority. If you do not find yourself to have the confidence to check the correctness of my argument then you shouldn't agree or disagree either way. Say to yourself 'I don't understand what he's saying', above all, don't comment on a thread whose opening you don't understand, and move on with life.
Yes, I am very smug, I'm not even smug, I'm condescending, I'm not condescending because I have a higher education, I'm condescending because I'm fed up with stupidity, the arguments I put out are very easy and basic to understand and honestly, anyone reading those graphs should come to those conclusions, yet I've seen countless and countless people misinterpreting all those graphs without coming to the realization of these very basic givens, on both reddit and TL. I've seen 50 pages of TL posts discussing those graphics about probability of races to win at certain time intervals in matchups, and maybe 1-2 people pointed out how misleading it was because of the arguments I put out, and no one listened and other people go discuss trivial and unrelated stuff like 'sample size' while there are much bigger problems. I've seen the TLPD winrates posted on both TL and reddit and people discussed them for days and so few people initially pointed out that the lines between the graphs in the old aesthetic were completely ludicrous and they should be bars, and even fewer people were critical of the fact that the error bars were calculated by a means which assumes independent experiments, which they are not.
It doesn't take a genius to see this, it just takes allowing yourself to be critical. As soon as I ask a lot of those people 'There are some grave fallacies with those stats,c an you point them out?' they will most likely come with at least 80% of the shit I pointed out and probably with some things I overlooked. It doesn't take a brain, it takes not being a mindless drone and being critical of stuff that is being posted. As for credentials, I guarantee you that the people who post those TLPD winrate graphs either have no statistical credentials, or are wilfully lying to people and oversimplifying it, because it's just statistical faux pas.
But then he doesn't say how we should go about interpreting those statistics and providing proof. We should interpret them as what they are. They are the winrates for this month, it says nothing of balance or any other interpretations you can make of them. You see what you get, and the error bars are, simply put, incorrect and a statistical gaffe. I'm not sure what they are supposed to mean, they don't mean anything if the map scores aren't independent probability experiments.
That being said I do think most of the people take a really simplistic approach to statistics, but well statistics are a hard subject to tackle Nope, it's very easy, it's more that people like to see things that you can't conclude from stuff.
On May 04 2012 10:09 LaM wrote:Show nested quote +On May 04 2012 10:05 Reptilia wrote:
because reddit is so much better. Lol.
i wouldnt be surprised if that guy had a tl account and got banned and got so butthurt he posted that. Did you read anything past the qualifier that was added on after his original post on Reddit? Doesn't look like it. Your post has nothing to do with the vast majority of his post. Or where you saying he got butthurt that he got banned from TL so he went to Reddit and wrote an intelligent post about balance statistics and how they can be misleading? All these douchey little TL > Reddit posts are the type of annoying shit that makes people think TL is pretentious anyways. I know he isn't any better for his equally douchey qualifier, but at least he followed it up with an informative, well-written post. Something I haven't seen much of from the Reddit bashers here. At least on Reddit your pathetic contributions would be downvoted enough so that I wouldn't have to waste time responding to them and could help clarify things for people who give a shit about having a meaningful discussion.
As linked in the OP, I did not add that qualifier on top myself, I never added the qualifier formally, someone asked me 'Have you posted it on TL' (the OP here), I said 'Nope', he asked 'why?', I said that which he quoted.
That said, I never mentioned TL in the original post, I was mainly critical of screddit and its continued misuse of statistics and it got upvoted to be the #1 post on the screddit first page. This exemplifies a quality of screddit that I feel TL heavily lacks.
Edit: Also:
Pepper_MD just sent you a month of reddit gold! Wasn't that nice?
Here's a note that was included:
I have degree in Stats. All I have to say is Thank You.
I have no idea what reddit gold is, is it good?
User was banned for this post.
|
On May 04 2012 10:13 Quochobao wrote:
All of what OP says about statistics is right -- that his tone is arrogant does not jeopardize the merit of this argument a bit.
Is he tactless in his approach? Probably. Is he right about statistics? Surely.
Don't be put off by the qualifier right away. That only proves TL to be not open-minded to the truth, no matter how harshly stated.
But the thing is, what he says in his arrogant manner ("99% of the people don't know this") is basically common knowledge.
Is it true? For the most part yes. Is it advanced mathematics or anything ground-breaking, thought-provocing or fallacy-revealing? No. He's just arguing common sense against an imagined 99%.
|
On May 04 2012 10:17 LaM wrote:Show nested quote +On May 04 2012 10:08 shaldengeki wrote:While I agree that there are significant issues with the way that many statistics on TL are presented - I've posted on this before and I was the guy who nudged the monthly winrates graphs to add error bars in the first place - you're not doing the discourse any favors by reposting this, I think. There are hardly any statistical arguments actually made in the post - for instance: On May 04 2012 09:00 Cyberonic wrote:
The TLPD winrate graphs are praetentious and amateuristic, sorry to say it but that's how it is, the error bars there are pure bollocks and are calculated using the rules of independent probability experiments, that is to say, it is assumed that the results of every series has no effect on the others, as if you flip a coin. If they were independent, sample size would be enough by a large margin to say something, but they are not independent. Because you're dealing with players, not just games. Good players simply ruin the idea of independent experiments.
There is something deeply hypocritical about decrying statistics discussions on TL for being superficial and then totally failing to present statistical evidence for your assertion that games outcomes are not independent. One would think that the actual mathematics would be pretty trivial, so simply asserting that "they are not independent because they are players" is committing exactly the sin that you're supposedly railing against. I don't agree with you at all. I think it is pretty clear that game results are not independent outcomes. Consider a 10game match between DRG and Joe "Code B Protoss" Schmoe. DRG wins the first 9 games. Any rational, logical observer would favor him greatly to win the 10th game, right? But winrate graphs still assume the outcome of Game 10 should have a 50/50 chance of going either way, like it's just a coin flip. Now, in massive sample sizes this would be corrected by enough players from every race being better than their opponent in any series so that it would smooth out any errors, but in month long samples from a tiny group of pros the deviations don't get corrected. I think the math for showing that is extremely hard, but the logic behind it is very strong. Similar to how I'm sure you won't debate that 2+2 is valid, but you would have a pretty damn hard time mathematically proving addition to me.
I think you're probably mistaking what the purpose of the winrate charts is, and what "independent outcomes" means in the context of repeated experiments. Of course you wouldn't apply the winrate charts to the situation you're describing - what they do is aggregate results across several skill levels and regions to provide a general indicator of race balance. Nobody is claiming that every single game between a protoss and a zerg has a 50/50 chance of going either way, and if this is how you're interpreting the winrate charts, that's definitely a problem on your end!
The issue you describe with skill impacting win chances is actually not an issue of independent events at all. If the events were dependent, then the results from all prior games between all zergs and all protosses would impact the win probability of the next game between a zerg and a protoss. This is not the issue at hand in your scenario, where you're talking about skill level of each player impacting win probabilities. That's the realm of ELO, and the winrate charts make no attempt at gauging the skill levels of each player.
|
It is annoying when people look at winrates and automatically assume X race is OP. However, the condescending tone of the post will probably a turn off for most people and they will continue to believe in these statistics instead of actually playing the game.
|
On May 04 2012 10:27 shaldengeki wrote:Show nested quote +On May 04 2012 10:17 LaM wrote:On May 04 2012 10:08 shaldengeki wrote:While I agree that there are significant issues with the way that many statistics on TL are presented - I've posted on this before and I was the guy who nudged the monthly winrates graphs to add error bars in the first place - you're not doing the discourse any favors by reposting this, I think. There are hardly any statistical arguments actually made in the post - for instance: On May 04 2012 09:00 Cyberonic wrote:
The TLPD winrate graphs are praetentious and amateuristic, sorry to say it but that's how it is, the error bars there are pure bollocks and are calculated using the rules of independent probability experiments, that is to say, it is assumed that the results of every series has no effect on the others, as if you flip a coin. If they were independent, sample size would be enough by a large margin to say something, but they are not independent. Because you're dealing with players, not just games. Good players simply ruin the idea of independent experiments.
There is something deeply hypocritical about decrying statistics discussions on TL for being superficial and then totally failing to present statistical evidence for your assertion that games outcomes are not independent. One would think that the actual mathematics would be pretty trivial, so simply asserting that "they are not independent because they are players" is committing exactly the sin that you're supposedly railing against. I don't agree with you at all. I think it is pretty clear that game results are not independent outcomes. Consider a 10game match between DRG and Joe "Code B Protoss" Schmoe. DRG wins the first 9 games. Any rational, logical observer would favor him greatly to win the 10th game, right? But winrate graphs still assume the outcome of Game 10 should have a 50/50 chance of going either way, like it's just a coin flip. Now, in massive sample sizes this would be corrected by enough players from every race being better than their opponent in any series so that it would smooth out any errors, but in month long samples from a tiny group of pros the deviations don't get corrected. I think the math for showing that is extremely hard, but the logic behind it is very strong. Similar to how I'm sure you won't debate that 2+2 is valid, but you would have a pretty damn hard time mathematically proving addition to me. I think you're probably mistaking what the purpose of the winrate charts is, and what "independent outcomes" means in the context of repeated experiments. Of course you wouldn't apply the winrate charts to the situation you're describing - what they do is aggregate results across several skill levels and regions to provide a general indicator of race balance. Nobody is claiming that every single game between a protoss and a zerg has a 50/50 chance of going either way, and if this is how you're interpreting the winrate charts, that's definitely a problem on your end! The issue you describe with skill impacting win chances is actually not an issue of independent events at all. If the events were dependent, then the results from all prior games between all zergs and all protosses would impact the win probability of the next game between a zerg and a protoss. This is not the issue at hand in your scenario, where you're talking about skill level of each player impacting win probabilities. That's the realm of ELO, and the winrate charts make no attempt at gauging the skill levels of each player.
And you are again making the mistake that winrate charts indicate win probabilities and balance, which is the whole point, THEY DON'T!
You wanted error bars added to winrate charts? Why? What error? They are cataloging winrates from the past month, where is the error coming from?
My mistake was in even ceding that error bars should be part of the chart and make any sense with them. They don't. I agree my explanation isn't applicable to the charts, but that isn't because my explanation is assuming things incorrectly, it's because the error bars shouldn't be there in the first place...
|
This is a pretty clear example of how you say things being as important as what you say. The content of the post is very good, correctly pointing out the people really like to conclude things from statistics that are totally not what the statistics say. It would make a much more compelling argument if it wasn't written by such an asshole.
|
I prefer IdrA's way of saying this:
if statistics get you hard make one of those ladder analysis pages or something, but stop interfering with balance discussions.
|
On May 04 2012 09:26 dmasterding wrote:
Did any of you guys actually read the thing? He didn't actually give any opinions about the matchups, he was just trying to get rid of some misconceptions people had about interpretation of results. I am pretty sure that if the OP never mentioned this person was from r/SC you guys wouldn't be so biased against the author.
Frankly, he was asking for the bias by being biased against TL.
He didn't say anything new or ground breaking. I thought this was supposed to be some statistical elephant in the room where some expert of statistics would show us something very important. Instead, it basically said, "don't make poor assumptions." Well, good advice, I say.
|
This reminds of an old point that a lot of old brood war veterans like Artosis used to make about how the older maps were actually imbalanced despite the winrates they had because at the time dominant and superior players like boxer, iloveoov, and nada were skewing the results to the point where the map statistic should have been more than 50% win rate for terran because those players were much better than their opponents and had higher than 50% win rates and there weren't enough games from other average or less skilled players to reasonably counteract that fact. This is kind of related to how some maps were balanced at the top korean level for certain matchups, but not balanced for a top foreigner level. Essentially the balance in brood war and in brood war maps was relevant to the skill of the players at the time. Older maps might have sported balanced win rates only because of the strategy and skill of the players at the time. Applying modern strategies and the improved skill of players can suddenly reveal undiscovered imbalance.
I'm not saying whether or not that old point is correct, but it certainly had better concrete concepts than whatever this guy seems to be pointing out.
This extends even further, most tournaments have qualifiers, so say X is underpowered, the players who play X that get into the tournament are simply better because thety got in despite the imbalance, therefore as they are better, they will continue to win even despite the imbalance vested against them, thereby skewing the results to more 50-50 than it actually is.
This is kind of the bonjwa terran balance argument for old broodwar maps. However a lot of people didn't agree with this because they didn't believe the bonjwas were really that statistically significant. This causes issues when you talk about the sample size not being large enough.
If the sample size truly isn't large enough to make star players statistically insignificant then you would need to be able to identify "bonjwas" or something similar in player dominance to even hint that one race is underpowered with a 50% winrate. If you can't do that then you are making a judgement call that a few star players that play a specific race are that much superior to everyone else without winning enough to actually prove it. Without good proof the only other scenario you could argue is that it isn't the game that is balanced, but just the number of star players playing for each race (Which is silly because of the bizarre concept of improvement).
|
On May 04 2012 10:35 LaM wrote:Show nested quote +On May 04 2012 10:27 shaldengeki wrote:On May 04 2012 10:17 LaM wrote:On May 04 2012 10:08 shaldengeki wrote:While I agree that there are significant issues with the way that many statistics on TL are presented - I've posted on this before and I was the guy who nudged the monthly winrates graphs to add error bars in the first place - you're not doing the discourse any favors by reposting this, I think. There are hardly any statistical arguments actually made in the post - for instance: On May 04 2012 09:00 Cyberonic wrote:
The TLPD winrate graphs are praetentious and amateuristic, sorry to say it but that's how it is, the error bars there are pure bollocks and are calculated using the rules of independent probability experiments, that is to say, it is assumed that the results of every series has no effect on the others, as if you flip a coin. If they were independent, sample size would be enough by a large margin to say something, but they are not independent. Because you're dealing with players, not just games. Good players simply ruin the idea of independent experiments.
There is something deeply hypocritical about decrying statistics discussions on TL for being superficial and then totally failing to present statistical evidence for your assertion that games outcomes are not independent. One would think that the actual mathematics would be pretty trivial, so simply asserting that "they are not independent because they are players" is committing exactly the sin that you're supposedly railing against. I don't agree with you at all. I think it is pretty clear that game results are not independent outcomes. Consider a 10game match between DRG and Joe "Code B Protoss" Schmoe. DRG wins the first 9 games. Any rational, logical observer would favor him greatly to win the 10th game, right? But winrate graphs still assume the outcome of Game 10 should have a 50/50 chance of going either way, like it's just a coin flip. Now, in massive sample sizes this would be corrected by enough players from every race being better than their opponent in any series so that it would smooth out any errors, but in month long samples from a tiny group of pros the deviations don't get corrected. I think the math for showing that is extremely hard, but the logic behind it is very strong. Similar to how I'm sure you won't debate that 2+2 is valid, but you would have a pretty damn hard time mathematically proving addition to me. I think you're probably mistaking what the purpose of the winrate charts is, and what "independent outcomes" means in the context of repeated experiments. Of course you wouldn't apply the winrate charts to the situation you're describing - what they do is aggregate results across several skill levels and regions to provide a general indicator of race balance. Nobody is claiming that every single game between a protoss and a zerg has a 50/50 chance of going either way, and if this is how you're interpreting the winrate charts, that's definitely a problem on your end! The issue you describe with skill impacting win chances is actually not an issue of independent events at all. If the events were dependent, then the results from all prior games between all zergs and all protosses would impact the win probability of the next game between a zerg and a protoss. This is not the issue at hand in your scenario, where you're talking about skill level of each player impacting win probabilities. That's the realm of ELO, and the winrate charts make no attempt at gauging the skill levels of each player. And you are again making the mistake that winrate charts indicate win probabilities and balance, which is the whole point, THEY DON'T! You wanted error bars added to winrate charts? Why? What error? They are cataloging winrates from the past month, where is the error coming from? My mistake was in even ceding that error bars should be part of the chart and make any sense with them. They don't. I agree my explanation isn't applicable to the charts, but that isn't because my explanation is assuming things incorrectly, it's because the error bars shouldn't be there in the first place...
While that's exactly the point, they don't indicate balance, everyone seems to completely ignore this fact, and treat them as irrefutable evidence of "P is UP, T is OP, because they have been losing/winning, look at the graph, it's obvious."
When people start looking past the stats and actually thinking, you can use the winrates as just another fun useless fact
|
On May 04 2012 10:35 LaM wrote:Show nested quote +On May 04 2012 10:27 shaldengeki wrote:On May 04 2012 10:17 LaM wrote:On May 04 2012 10:08 shaldengeki wrote:While I agree that there are significant issues with the way that many statistics on TL are presented - I've posted on this before and I was the guy who nudged the monthly winrates graphs to add error bars in the first place - you're not doing the discourse any favors by reposting this, I think. There are hardly any statistical arguments actually made in the post - for instance: On May 04 2012 09:00 Cyberonic wrote:
The TLPD winrate graphs are praetentious and amateuristic, sorry to say it but that's how it is, the error bars there are pure bollocks and are calculated using the rules of independent probability experiments, that is to say, it is assumed that the results of every series has no effect on the others, as if you flip a coin. If they were independent, sample size would be enough by a large margin to say something, but they are not independent. Because you're dealing with players, not just games. Good players simply ruin the idea of independent experiments.
There is something deeply hypocritical about decrying statistics discussions on TL for being superficial and then totally failing to present statistical evidence for your assertion that games outcomes are not independent. One would think that the actual mathematics would be pretty trivial, so simply asserting that "they are not independent because they are players" is committing exactly the sin that you're supposedly railing against. I don't agree with you at all. I think it is pretty clear that game results are not independent outcomes. Consider a 10game match between DRG and Joe "Code B Protoss" Schmoe. DRG wins the first 9 games. Any rational, logical observer would favor him greatly to win the 10th game, right? But winrate graphs still assume the outcome of Game 10 should have a 50/50 chance of going either way, like it's just a coin flip. Now, in massive sample sizes this would be corrected by enough players from every race being better than their opponent in any series so that it would smooth out any errors, but in month long samples from a tiny group of pros the deviations don't get corrected. I think the math for showing that is extremely hard, but the logic behind it is very strong. Similar to how I'm sure you won't debate that 2+2 is valid, but you would have a pretty damn hard time mathematically proving addition to me. I think you're probably mistaking what the purpose of the winrate charts is, and what "independent outcomes" means in the context of repeated experiments. Of course you wouldn't apply the winrate charts to the situation you're describing - what they do is aggregate results across several skill levels and regions to provide a general indicator of race balance. Nobody is claiming that every single game between a protoss and a zerg has a 50/50 chance of going either way, and if this is how you're interpreting the winrate charts, that's definitely a problem on your end! The issue you describe with skill impacting win chances is actually not an issue of independent events at all. If the events were dependent, then the results from all prior games between all zergs and all protosses would impact the win probability of the next game between a zerg and a protoss. This is not the issue at hand in your scenario, where you're talking about skill level of each player impacting win probabilities. That's the realm of ELO, and the winrate charts make no attempt at gauging the skill levels of each player. And you are again making the mistake that winrate charts indicate win probabilities and balance, which is the whole point, THEY DON'T! You wanted error bars added to winrate charts? Why? What error? They are cataloging winrates from the past month, where is the error coming from? My mistake was in even ceding that error bars should be part of the chart and make any sense with them. They don't. I agree my explanation isn't applicable to the charts, but that isn't because my explanation is assuming things incorrectly, it's because the error bars shouldn't be there in the first place...
Please remain calm. I'd love to have a level-headed discussion with you!
The winrate charts indicate win probabilities aggregated across each race. This is indisputable. They provide what I believe is a general indicator of balance - I don't believe that there are statistically-significant differences in skill between races, so it stands to reason that in the aggregate, this provides some information on the balance between races.
The error bars allow you to determine whether one month's average is significantly different from previous months. This is hugely important as before their addition people were making all sorts of wild claims as to how certain patches were throwing race balance off. Now that we can determine whether or not each month was significantly-different from previous months, we can more reasonably talk about whether or not changes to the game are having effects on winrates.
|
On May 04 2012 10:40 shaldengeki wrote:Show nested quote +On May 04 2012 10:35 LaM wrote:On May 04 2012 10:27 shaldengeki wrote:On May 04 2012 10:17 LaM wrote:On May 04 2012 10:08 shaldengeki wrote:While I agree that there are significant issues with the way that many statistics on TL are presented - I've posted on this before and I was the guy who nudged the monthly winrates graphs to add error bars in the first place - you're not doing the discourse any favors by reposting this, I think. There are hardly any statistical arguments actually made in the post - for instance: On May 04 2012 09:00 Cyberonic wrote:
The TLPD winrate graphs are praetentious and amateuristic, sorry to say it but that's how it is, the error bars there are pure bollocks and are calculated using the rules of independent probability experiments, that is to say, it is assumed that the results of every series has no effect on the others, as if you flip a coin. If they were independent, sample size would be enough by a large margin to say something, but they are not independent. Because you're dealing with players, not just games. Good players simply ruin the idea of independent experiments.
There is something deeply hypocritical about decrying statistics discussions on TL for being superficial and then totally failing to present statistical evidence for your assertion that games outcomes are not independent. One would think that the actual mathematics would be pretty trivial, so simply asserting that "they are not independent because they are players" is committing exactly the sin that you're supposedly railing against. I don't agree with you at all. I think it is pretty clear that game results are not independent outcomes. Consider a 10game match between DRG and Joe "Code B Protoss" Schmoe. DRG wins the first 9 games. Any rational, logical observer would favor him greatly to win the 10th game, right? But winrate graphs still assume the outcome of Game 10 should have a 50/50 chance of going either way, like it's just a coin flip. Now, in massive sample sizes this would be corrected by enough players from every race being better than their opponent in any series so that it would smooth out any errors, but in month long samples from a tiny group of pros the deviations don't get corrected. I think the math for showing that is extremely hard, but the logic behind it is very strong. Similar to how I'm sure you won't debate that 2+2 is valid, but you would have a pretty damn hard time mathematically proving addition to me. I think you're probably mistaking what the purpose of the winrate charts is, and what "independent outcomes" means in the context of repeated experiments. Of course you wouldn't apply the winrate charts to the situation you're describing - what they do is aggregate results across several skill levels and regions to provide a general indicator of race balance. Nobody is claiming that every single game between a protoss and a zerg has a 50/50 chance of going either way, and if this is how you're interpreting the winrate charts, that's definitely a problem on your end! The issue you describe with skill impacting win chances is actually not an issue of independent events at all. If the events were dependent, then the results from all prior games between all zergs and all protosses would impact the win probability of the next game between a zerg and a protoss. This is not the issue at hand in your scenario, where you're talking about skill level of each player impacting win probabilities. That's the realm of ELO, and the winrate charts make no attempt at gauging the skill levels of each player. And you are again making the mistake that winrate charts indicate win probabilities and balance, which is the whole point, THEY DON'T! You wanted error bars added to winrate charts? Why? What error? They are cataloging winrates from the past month, where is the error coming from? My mistake was in even ceding that error bars should be part of the chart and make any sense with them. They don't. I agree my explanation isn't applicable to the charts, but that isn't because my explanation is assuming things incorrectly, it's because the error bars shouldn't be there in the first place... Please remain calm. I'd love to have a level-headed discussion with you! The winrate charts indicate win probabilities aggregated across each race. This is indisputable. They provide what I believe is a general indicator of balance - I don't believe that there are statistically-significant differences in skill between races, so it stands to reason that in the aggregate, this provides some information on the balance between races. The error bars allow you to determine whether one month's average is significantly different from previous months. This is hugely important as before their addition people were making all sorts of wild claims as to how certain patches were throwing race balance off. Now that we can determine whether or not each month was significantly-different from previous months, we can more reasonably talk about whether or not changes to the game are having effects on winrates.
I'm very calm thanks 
It's just you seem to be very oddly ignoring the relevant points of discussion in a biased attempt to defend your error bars.
You keep making the claim, with no backing, that the win rates provide some reflection of balance. People, including you in this discussion, always like to attack the straw man that people saying winrates don't indicate balance are making that assumption because of differences in the skill level of the races. If you read the OP, it is addressing exactly why they don't show any useful information about balance, for a set of reasons completely different than the one straw man reason you do address.
I'm still confused as to what you think the error bars add to the discussion. The winrates are results, not predictions. That means the only source of error would be missing results or incorrectly entering results into the graph. But that certainly isn't how you are presenting them. In this case you seem to be advocating them so that people can see a win rate of X% with an imagined deviation indicating it falls within a range from Y% to Z% and therefore not freak out since last month the win rate was A% which also falls between Y% and Z%. That's not really a useful error bar, its just tacking some extra shit on so that people can continue to try and draw erroneous balance conclusions from something that indicates nothing about balance.
Again, your viewing winrate graphs as balance graphs. I think that's the root of why we disagree about the error bars.
|
I guess I was hoping for a little bit more of a comprehensive approach to this. Rather than a set of three, relatively specific, "fallacies."
I think there are a lot of people who don't think critically on statistics shown to them, but this doesn't do much to help them. You're just pointing out three possible ways mistakes could be made and then explaining them away with common sense. It's not really an approach from statistics and its not really about statistics, it's logic. All of what you explained was logic.
Now while people certainly could use a reminder to use logic more often, this is not related to their misunderstanding of the fundamentals of statistics. From you introduction, I got the impression that you were going to explain some fundamentals of statistics and explain how you should apply them to SC2 data.
While this post does indeed mention statistics, it is just used as an arbitrary example for illustrating logical fallacies which are not fundamentally related to statistics in any way. A more apt title would be "Mistakes in reasoning."
|
Reddit is the new 4chan. That said, he has some points in the methodology and idea behind some of the things people commonly mistake; can't say I'm a fan of the way he went about it, but he uses Reddit, so that explains that.
|
All trivially obvious points. I suppose some people have these misconceptions, but this is hardly news. Based on the title, I was hoping for some deeper insights, or at least something more thoughtful, like this.
|
Are win rates an indicator of true balance? I guess not.
But can we even know what the true balance is? I guess not either.
What we use win rates for is to see if the game remains competitive, which will reflect a perceived balance. That is what almost everybody cares about more than a remote fantasy of true balance.
|
Makes solid points. If there was one thing I learned in my stats class is how broken statistics can get. It is a very tricky subject to understand.
|
Why do people have to be aggressive and insulting when they're trying to have an intellectual argument?
|
The working assumption of balancing is that each race has equal skill representation among the pool of players. It's a flawed assumption, of course, but there isn't any better way. The samples of professional games aren't big enough, but that's all we got to work with; the ladder is less reliable in terms of quality. Should we be cautious in our claims, sure. Finally, the time intervals winrates just need added weight in number of games. Otherwise Tasteless is like omg 100% winrate 0-5 minutes; while it most likely means only one game ended so soon and it happened to be a win.
|
Even though not every one can agree on this subject, we're likely to agree that the blizzard dev team is trying TOO HARD to tweak the game. They screw it up the majority of the time instead of waiting and doing minor tweaks every 6 months and such. The game never gets to develop before someone gets the nerf bat just because another race is winning in the metagame. Terran continually get screwed over in late game because the early game is relatively powerful to try and make a ~50% ratio. They've shown countless times that they want a pretty statistic more than any other aspect... and they do it based off their crappy map pool of the past which is down right idiotic. Bigger maps would've been better than the barracks +5 second build time, for example, instead of having to deal with bigger maps and the change now.
|
I was going to read the OP until I read the first post. He talks about statistics but starts off with saying 99% people don't know things twice. I don't think thats a proper way to start an argument about statistics. Especially if you're going to ask the "re-poster" if you will to post that first paragraph about TL calling us out on how we aren't the best at writing threads.
|
His points are all sensible, and to me they all seemed like common sense.
|
On May 04 2012 09:00 Cyberonic wrote:This post is originally written by Drabzalver on Reddit. Since he does not have a TL account I asked and was allowed to repost if I include this:Drabzalver on Reddit:Show nested quote +TL is the [swear word] of praetentious [swear word] where moderators reward supposed 'high quality posts' which are full of statistical and scientific garbage I just outlined just because they are 'praesented nicely' and the mods basically think that any long post with a lot of images and no swear words is intellectually advanced while often a lot of it is total garbage filled with wrong interpretations and grave statistical errors.
Also, I don't have a TL account, you're welcome to repost, but do include this qualifier, should be fun,
I somewhat agree about the above statement and I think the fallacies stuff just posted were targeted to those balance whiners in TL
|
With a bunch of you lot replying because of his statement and not because of the article proves his point, I'm just saying.
|
On May 04 2012 10:20 SappigeKutVolKots wrote:Okay, I wanted to stay away from this site, but I couldn't let some stuff go unanswered, I am drab, anyone can message me on reddit to verify this.Show nested quote +On May 04 2012 09:40 windsupernova wrote:
As much as I agree with him in some points. I don't like how he comes off as someone pretty arrogant and doesn't even present some kind of credentials on why he understand statistics more than 99% of people.I mean for all we know he could be some arrogant College kid who just passed his 1st statistics class. I'm not making an argument from authority, I don't need credentials, even if I was a cow or an anencephalic protozoan, it doesn't matter, there is no need for credentials because I'm making an argument from reason, not from authority, I do not even need to cite any sources because my argument is purely rational, not empirical. If you ask for 'credentials' to verify this post then you lost and don't know how to verify academic literature. My credentials are irrelevant, I'm not making an argument from authority. If you do not find yourself to have the confidence to check the correctness of my argument then you shouldn't agree or disagree either way. Say to yourself 'I don't understand what he's saying', above all, don't comment on a thread whose opening you don't understand, and move on with life. Yes, I am very smug, I'm not even smug, I'm condescending, I'm not condescending because I have a higher education, I'm condescending because I'm fed up with stupidity, the arguments I put out are very easy and basic to understand and honestly, anyone reading those graphs should come to those conclusions, yet I've seen countless and countless people misinterpreting all those graphs without coming to the realization of these very basic givens, on both reddit and TL. I've seen 50 pages of TL posts discussing those graphics about probability of races to win at certain time intervals in matchups, and maybe 1-2 people pointed out how misleading it was because of the arguments I put out, and no one listened and other people go discuss trivial and unrelated stuff like 'sample size' while there are much bigger problems. I've seen the TLPD winrates posted on both TL and reddit and people discussed them for days and so few people initially pointed out that the lines between the graphs in the old aesthetic were completely ludicrous and they should be bars, and even fewer people were critical of the fact that the error bars were calculated by a means which assumes independent experiments, which they are not. It doesn't take a genius to see this, it just takes allowing yourself to be critical. As soon as I ask a lot of those people 'There are some grave fallacies with those stats,c an you point them out?' they will most likely come with at least 80% of the shit I pointed out and probably with some things I overlooked. It doesn't take a brain, it takes not being a mindless drone and being critical of stuff that is being posted. As for credentials, I guarantee you that the people who post those TLPD winrate graphs either have no statistical credentials, or are wilfully lying to people and oversimplifying it, because it's just statistical faux pas. Show nested quote +But then he doesn't say how we should go about interpreting those statistics and providing proof. We should interpret them as what they are. They are the winrates for this month, it says nothing of balance or any other interpretations you can make of them. You see what you get, and the error bars are, simply put, incorrect and a statistical gaffe. I'm not sure what they are supposed to mean, they don't mean anything if the map scores aren't independent probability experiments. Show nested quote +That being said I do think most of the people take a really simplistic approach to statistics, but well statistics are a hard subject to tackle Nope, it's very easy, it's more that people like to see things that you can't conclude from stuff. Show nested quote +On May 04 2012 10:09 LaM wrote:On May 04 2012 10:05 Reptilia wrote:
because reddit is so much better. Lol.
i wouldnt be surprised if that guy had a tl account and got banned and got so butthurt he posted that. Did you read anything past the qualifier that was added on after his original post on Reddit? Doesn't look like it. Your post has nothing to do with the vast majority of his post. Or where you saying he got butthurt that he got banned from TL so he went to Reddit and wrote an intelligent post about balance statistics and how they can be misleading? All these douchey little TL > Reddit posts are the type of annoying shit that makes people think TL is pretentious anyways. I know he isn't any better for his equally douchey qualifier, but at least he followed it up with an informative, well-written post. Something I haven't seen much of from the Reddit bashers here. At least on Reddit your pathetic contributions would be downvoted enough so that I wouldn't have to waste time responding to them and could help clarify things for people who give a shit about having a meaningful discussion. As linked in the OP, I did not add that qualifier on top myself, I never added the qualifier formally, someone asked me 'Have you posted it on TL' (the OP here), I said 'Nope', he asked 'why?', I said that which he quoted. That said, I never mentioned TL in the original post, I was mainly critical of screddit and its continued misuse of statistics and it got upvoted to be the #1 post on the screddit first page. This exemplifies a quality of screddit that I feel TL heavily lacks.Edit: Also: Pepper_MD just sent you a month of reddit gold! Wasn't that nice?
Here's a note that was included:
I have degree in Stats. All I have to say is Thank You.I have no idea what reddit gold is, is it good?
Even if your argument is true, it was poorly written and therefore deserves the terrible response that you're getting.
You need to realize that most people don't really care about the content of the OP, and it's not because they're stupid. What you're writing about is a niche interest. Stupidity isn't rampant because more people (and by people I mean forum posters, which I imagine is primarily below college age, by the way) don't spend their time critically evaluating video game statistics.
It's amazing to me that you're so indignant about people reacting negatively towards your condescension, simply because your argument is true or whatever it is you think justifies your tone. You're writing as if all that matters is the accuracy of what you're saying and not how you present it. Statistics can't teach you common sense I guess
|
Lol at how pretentiously he spells pretentious. (Also, wrongly, this is English not Latin)
|
On May 04 2012 10:08 shaldengeki wrote:While I agree that there are significant issues with the way that many statistics on TL are presented - I've posted on this before and I was the guy who nudged the monthly winrates graphs to add error bars in the first place - you're not doing the discourse any favors by reposting this, I think. There are hardly any statistical arguments actually made in the post - for instance: Show nested quote +On May 04 2012 09:00 Cyberonic wrote:
The TLPD winrate graphs are praetentious and amateuristic, sorry to say it but that's how it is, the error bars there are pure bollocks and are calculated using the rules of independent probability experiments, that is to say, it is assumed that the results of every series has no effect on the others, as if you flip a coin. If they were independent, sample size would be enough by a large margin to say something, but they are not independent. Because you're dealing with players, not just games. Good players simply ruin the idea of independent experiments.
There is something deeply hypocritical about decrying statistics discussions on TL for being superficial and then totally failing to present statistical evidence for your assertion that games outcomes are not independent. One would think that the actual mathematics would be pretty trivial, so simply asserting that "they are not independent because they are players" is committing exactly the sin that you're supposedly railing against. Please reconsider reposting topics like this in the future, or at the very least, try to be productive and rigorous in your arguments if you truly want TL to be a community that is rigorous in its discussion!
I think this is really more helpful than the OP. For something that starts with a complaint about TL being pretentious, I found the post really condesending and a gross over simplification. Yes, it is true that people on TL do awful things with statistics, but as a statistician at a University, the people that I provide numbers to at work often do just as awful things with much more serious numbers. As well, for a post decrying people's statistical knowledge, it doesn't help to begin your post by making up some stats.
Also, if you want to prove something about math, you have to use math. Don't just tell people that they are wrong, help people understand how to use stats properly.
|
On May 04 2012 10:59 LaM wrote:Show nested quote +On May 04 2012 10:40 shaldengeki wrote:On May 04 2012 10:35 LaM wrote:On May 04 2012 10:27 shaldengeki wrote:On May 04 2012 10:17 LaM wrote:On May 04 2012 10:08 shaldengeki wrote:While I agree that there are significant issues with the way that many statistics on TL are presented - I've posted on this before and I was the guy who nudged the monthly winrates graphs to add error bars in the first place - you're not doing the discourse any favors by reposting this, I think. There are hardly any statistical arguments actually made in the post - for instance: On May 04 2012 09:00 Cyberonic wrote:
The TLPD winrate graphs are praetentious and amateuristic, sorry to say it but that's how it is, the error bars there are pure bollocks and are calculated using the rules of independent probability experiments, that is to say, it is assumed that the results of every series has no effect on the others, as if you flip a coin. If they were independent, sample size would be enough by a large margin to say something, but they are not independent. Because you're dealing with players, not just games. Good players simply ruin the idea of independent experiments.
There is something deeply hypocritical about decrying statistics discussions on TL for being superficial and then totally failing to present statistical evidence for your assertion that games outcomes are not independent. One would think that the actual mathematics would be pretty trivial, so simply asserting that "they are not independent because they are players" is committing exactly the sin that you're supposedly railing against. I don't agree with you at all. I think it is pretty clear that game results are not independent outcomes. Consider a 10game match between DRG and Joe "Code B Protoss" Schmoe. DRG wins the first 9 games. Any rational, logical observer would favor him greatly to win the 10th game, right? But winrate graphs still assume the outcome of Game 10 should have a 50/50 chance of going either way, like it's just a coin flip. Now, in massive sample sizes this would be corrected by enough players from every race being better than their opponent in any series so that it would smooth out any errors, but in month long samples from a tiny group of pros the deviations don't get corrected. I think the math for showing that is extremely hard, but the logic behind it is very strong. Similar to how I'm sure you won't debate that 2+2 is valid, but you would have a pretty damn hard time mathematically proving addition to me. I think you're probably mistaking what the purpose of the winrate charts is, and what "independent outcomes" means in the context of repeated experiments. Of course you wouldn't apply the winrate charts to the situation you're describing - what they do is aggregate results across several skill levels and regions to provide a general indicator of race balance. Nobody is claiming that every single game between a protoss and a zerg has a 50/50 chance of going either way, and if this is how you're interpreting the winrate charts, that's definitely a problem on your end! The issue you describe with skill impacting win chances is actually not an issue of independent events at all. If the events were dependent, then the results from all prior games between all zergs and all protosses would impact the win probability of the next game between a zerg and a protoss. This is not the issue at hand in your scenario, where you're talking about skill level of each player impacting win probabilities. That's the realm of ELO, and the winrate charts make no attempt at gauging the skill levels of each player. And you are again making the mistake that winrate charts indicate win probabilities and balance, which is the whole point, THEY DON'T! You wanted error bars added to winrate charts? Why? What error? They are cataloging winrates from the past month, where is the error coming from? My mistake was in even ceding that error bars should be part of the chart and make any sense with them. They don't. I agree my explanation isn't applicable to the charts, but that isn't because my explanation is assuming things incorrectly, it's because the error bars shouldn't be there in the first place... Please remain calm. I'd love to have a level-headed discussion with you! The winrate charts indicate win probabilities aggregated across each race. This is indisputable. They provide what I believe is a general indicator of balance - I don't believe that there are statistically-significant differences in skill between races, so it stands to reason that in the aggregate, this provides some information on the balance between races. The error bars allow you to determine whether one month's average is significantly different from previous months. This is hugely important as before their addition people were making all sorts of wild claims as to how certain patches were throwing race balance off. Now that we can determine whether or not each month was significantly-different from previous months, we can more reasonably talk about whether or not changes to the game are having effects on winrates. I'm very calm thanks It's just you seem to be very oddly ignoring the relevant points of discussion in a biased attempt to defend your error bars. You keep making the claim, with no backing, that the win rates provide some reflection of balance. People, including you in this discussion, always like to attack the straw man that people saying winrates don't indicate balance are making that assumption because of differences in the skill level of the races. If you read the OP, it is addressing exactly why they don't show any useful information about balance, for a set of reasons completely different than the one straw man reason you do address. I'm still confused as to what you think the error bars add to the discussion. The winrates are results, not predictions. That means the only source of error would be missing results or incorrectly entering results into the graph. But that certainly isn't how you are presenting them. In this case you seem to be advocating them so that people can see a win rate of X% with an imagined deviation indicating it falls within a range from Y% to Z% and therefore not freak out since last month the win rate was A% which also falls between Y% and Z%. That's not really a useful error bar, its just tacking some extra shit on so that people can continue to try and draw erroneous balance conclusions from something that indicates nothing about balance. Again, your viewing winrate graphs as balance graphs. I think that's the root of why we disagree about the error bars.
I want to stick up for the error bars. Just because the graph displays results and not predictions, doesn't mean that the error bars are meaningless. If I measure something in the lab, I still place an uncertainty on that value even though it is a direct measurement. Likewise, even if I directly measure winrates, you still have an error rate on it. Why? Because not all games are included. Because these games aren't scientific outcomes and are not reproducable. Because the factors from one month to the next change and error rates are a simple way to show that. Sure, they aren't perfect, but don't think that just because they are a direct measurement that they don't have error and uncertainty attached.
|
You guys don't get it, his post has got nothing to do with statistics, it's about the TL community. He wants to prove his point that you create a thread in TL with a bunch of fancy writing, but the bottom line is, his true intention is to bash the people in TL. Again, I somewhat agree.
|
On May 04 2012 12:34 pOnarreT wrote:
You guys don't get it, his post has got nothing to do with statistics, it's about the TL community. He wants to prove his point that you create a thread in TL with a bunch of fancy writing, but the bottom line is, his true intention is to bash the people in TL. Again, I somewhat agree. I'm sure that's why he originally named his thread "What reddit does not understand about statistics" and posted it on reddit. Clearly all to do with TL.
|
Exactly true. What the OP doesn't mention is that Blizz "balances" based on statistics gathered this way. I can't say they are foolish in interpreting the data, but the skewing of data towards balanced by the matchmaker and the statistical points in the OP make balancing really hard.
|
He's quite the controversial character on r/sc as well, I've seen quite a few posts by him that buried by downvotes. He brings up a few good points, though it is interesting to see how he bears significant ill-will towards TL.
|
On May 04 2012 12:41 figq wrote:Show nested quote +On May 04 2012 12:34 pOnarreT wrote:
You guys don't get it, his post has got nothing to do with statistics, it's about the TL community. He wants to prove his point that you create a thread in TL with a bunch of fancy writing, but the bottom line is, his true intention is to bash the people in TL. Again, I somewhat agree. I'm sure that's why he originally named his thread "What reddit does not understand about statistics" and posted it on reddit. Clearly all to do with TL.
He got you
|
He makes good points, it's foolish to ignore a rude person's good ideas (though fine to abuse him for it).
|
On May 04 2012 12:55 Bagration wrote:
He's quite the controversial character on r/sc as well, I've seen quite a few posts by him that buried by downvotes. He brings up a few good points, though it is interesting to see how he bears significant ill-will towards TL.
All the "this isn't reddit/4chan" stuff I've seen here at times is a bit sanctimonious. When you see how many people have to get warned/banned all the time you realize TL isn't some shining center for civility.
|
I love the "99%" statistic provided, then the post goes on to talk about fallacies. It's humorously ironic, especially when the post is about statistics and the first one provided is a blatant lie.
|
Most of his objections are just basic statistical limitations that are well-known on TL, and are usually brought up in any related thread. Inevitably, there will always be some ignorant posters who don't know about them and draw false conclusions, but they usually get deservedly panned.
His claim that people are more likely to get away with a rubbish post if it's nicely formatted does have some credence, but it's hardly a damning indictment.
|
The only thing I find stupid is that he has no conclusion to how statistics can be used or what should be used. It just seems to me like he refutes any kind of information just to refute it. Maybe he is just tired of people basing balance off of win rates which is flimsy by itself to say the least. I don't see anything positive or informative about this.
|
OP is pretty freaking right. TL is swamped by ridiculous statistics that mean next to nothing. Every week there is some new "mathematical study" that claims that race X is op/up because of a number that is nearly irrelevant to the game (like the analysis of mining per race which found T has an advantage of .5 minerals over a minute or something).
There is too much overanalysis done with very little understanding of the way the game operates and the evidence generated from these experiments just fuels more race wars than solving anything. It isn't exclusive to TL however, and to be honest, if you're going to put some statistics you found anywhere, it'd be here, if for no other reason than the quality of the posters and admins relative to other sites.
The TLPD stats too are just a summary of a wide range of talent across many games and many people treat them like they are the EXACT evaluation of ability and "race imbalance" when it is much more subtle than just simply who wins more. Its surprising how often people focus on the mathematical parts of Starcraft as rationalization for wins/losses when a good part of the game is based on things you can't measure like positioning, star sense, psychological advantages, etc. Cost effectiveness is a part of the game, obviously, but its blown so horribly out of proportion.
Actually, a lot of this post is really reasonable and makes quite a bit of sense to me. Even his attitude is understandable when you read the comments on posts in the strategy forums and LR posts. Why are people so pissed? He isn't even really attacking TL.
|
Why do people even bother to respond if they don't even read the entire OP? This is is a problem rampant in forums anywhere on the internet, not just TL or Reddit.
On topic, the only real logical conclusion you can come to from the win/rate charts is this. If I select a random PvZ game from last month, the probability of selecting a game where P wins is x%, based on the chart (i don't know what it actually is, havent looked at the latest charts).
Arguments can be made that certain win/loss rates might be some indication of balance but with so many factors involved (and no raw data), any kind of statistical analysis is pointless.
Edit: win/loss rate can be used to identify a POSSIBILITY of an imbalance in a certain matchup but the actual win/loss rate stat is not evidence of any imbalance.
|
The fallacy of Drabzalvers intelligence; he stumbles through his logic like a dog. Moving sloppily from one point to the next, he is so stuck in his narrow minded thinking that he can only focus on concepts as situations. While I understand relativism is a wonderful argument Drabzalver , it also holds many fallacies of its own. Your arguments are coincidental; they are so dependent on the situation at hand. While you are attempting to thwart a scientific study, you don’t realize that you’re only exemplifying stupidity by having an online epiphany involving the fundamental logic that a chimp involves stacking blocks. Drabzalver, where do you get off? The fact that your post has even made it into the TL community speaks loudly that others feel the same as you. Why I must not know; is it not obvious that these studies must take place? Even if you want to bring all of your player and race balances into the numbers, are statistics not necessary in performing formal calculations of balance?
MustNotSleep
|
he's right. however same as in politics, the public opinion enjoys simplified thinking and therefore bashes him :D
|
relevent ideas and well formed badly presented with a lack of emotional detachment that this sort of matter requires (imo ofc). TBH havent actually looked graphs on win rates at all but to get an idea what he is talking about. The data itself imo is incomplete and is not modified to take into account different variables that would have an impact on the result. For example we could say that korean players at MLG performed less optimal than foriegnors as they had a significantlly lower win rate than occurs when they are at a GSL event however this over view of the win rate does not take into the many variables that would impact on said winrate such as jet-lag, 1st set win rate (btw interesting bit of triva is that the player who gets to 2 wins in the OSL will win overall), degree that they have had to practise due to travel, fact that they are not sleeping their regular beds even. Though hese examples are somewat strange it highlights the fact that the winrate of a player and or a race (which is a sample of players) is not that usefull considering it dosent take into account the effects of lots of other impacts that could effect the outcome. That is why for any sort of deep analyisis they are sub-optimal as they do not take into account maps, players, 1st set wins, recent patch changes, etc. For a general sort of overview they are interesting but they are not representitive of true balance.
For example medusa (this taken from liquidpedia) The original map was very difficult for Terran against Protoss; though the statistics do not appear terribly imbalanced at first glance (39-18 in favor of Protoss), they become much more damning once you realize that Flash went 7-1 against Protoss on the map, without which the statistics reduce to a dismal 38-11.
if one were to take the overall winrate and use to form a complex argument about the balance on this map it would not take into account flash's skill (which btw is ridiculous) thus one can be given a false impression on the balance on the map as significant portion of the T wins are from a single player
|
This is basic statistics 101 in university. He's not saying anything new, just assuming we're unqualified to understand the underlining meanings of stats.
|
There are no facts, only interpretations.
|
unfortunately, i aggree with the dude from reddit.
|
LMAO.
A criticism of the use of statistics with PURE WORDS. If this was an assignment topic in a statistics subject at university you would get 0%.
|
What I don't know about statistics? Who's being pretentious here lmao...
I'm pretty sure when people see TLPD winrates they don't instantly assume balances/imbalances and take it as gospel, and I'm pretty sure that blizzard doesn't balance the game around statistics gathered from the pro-level.
|
So despite all the hate going on in this thread, I see people making points every single month in the TLPD Winrates thread over and over and over that use some of the fallacies he listed as arguments. And he's right - everything he warned against is something that you shouldn't do if you're trying to interpret stats in a meaningful way. If everyone read this before they commented on winrates we'd be a better place. Fuck everyone hating on Reddit.
|
|
|
On May 04 2012 09:09 halfies wrote:
praetentious?
really?
i hope that how he spelt it too, because it would be really funny if he wrote this much about people being pretentious and used big words that he couldn't even spell.
You don't even begin a sentence with a capitalized letter, made even more fun by the fact that it was "I". Thereby forfeiting all rights to complain about spelling. Also if you have a problem with his argument, attack his argument.
In American English, spelt means exclusively a hardy wheat grown mostly in Europe, and the verb spell makes spelled in its past-tense and past-participial forms.
Ps. I put my money on you being the creator of the bad boy ^^
|
On May 04 2012 15:34 arbitrageur wrote:
LMAO.
A criticism of the use of statistics with PURE WORDS. If this was an assignment topic in a statistics subject at university you would get 0%.
So what you say is you need statistics to prove statistics are used wrongly?
|
Haven't learned anything of note. Thought this guy had something to add.
|
99% of the time this deals with winrates and balance, 99% of the people commenting on it have no clue how to interpret these numbers
Scumbag Drabzalver - rages at false statistics - makes up statistics in the first paragraph...
|
I was expecting some hard-core math in this thread when I saw the title. Or at least some math. I was disappointed to find that it was just a regurgitation of what should be blatantly obvious mixed with bad assumptions and lots of words linked together with shaky logic.
|
Firstly i would like to say that being polite, disliking swearing and poorly writting language is a GOOD thing. It keeps everyone civil, and makes text easily legible, thereby reducing misinformation.
On May 04 2012 09:00 Cyberonic wrote:- Fallacy: advantages at certain times of matchups expressed in graphs
There are also a lot of graphs posted which supposedly indicate that some races may have an advantage at certain matchups. Oh boy do people misread what these graphs mean. Take this bad boy. A naïve way to read it would simply be 'Hmm, Z has an early game advantage in ZvP, then it becomes about even, then P has a slight advantage up to the end, then Z again.', wrong; look closely, what does the graph actually say, it says this: IF a ZvP game ends in the 0-5 minute range, the chance is 60% it ends in Z's favour. Now the 'if' is so bloody important here, the game needn't end. Now, everyone of course realizes that that part is caused by early pools. Does Zerg really have a large advantage at that point? Can Zerg force a win at that point if they want to, are early pools overpowered? No, not at all, so what is going on? Imagine a ZvP, Z decides to 7pool, P doesn't scout soon enough, lings get in, kill every probe, traalalala, P GG's. Game over in the first 5 minutes in Z's favour. Okay, imagine a ZvP, Z decides to 7pool, P scouts in time, gets his wall up, damn, Z's like 'fuck man, shouldn't have done that'. But not necesarily GG's unless IdrA, the game goes on, Z however plays at such a disadvantage that in the next 5-10 minutes surely P will claim his victory unless P messes up. See the fallacy? That Z has that 'early game advantage' doesn't mean that Z is more powerful or that 7pools are too powerful, it just means that IF the game immediately ends due to a 7pool it will most likely be in Z's favour. If the 7pool fails, the game doesn't end at that point, Z will most likely stay in the game and play from a significant disadvantage to lose later. It is a grave statistical error of the magnitude of interpreting 'If a 8 year old child dies, the chance is the greatest he dies from a car swoop' as 'It is very likely 8 year old children die from car swoops'. The graph doesn't even say how likely it is that the game ends at certain intervals. For all you it's far more likely for P to win in the late game than in the mid game, even though the graph indicates that if the game ends in the late game, the chance is higher that Z takes it. And even so, that still says nothing about advantages of races at certain times. One would assume that if a race is likely to win at time X, that race enjoys an advantage slightly before that time, no? What would be far more intersting, though also not conclusive, would be a graph which outlines 'How large was the percentage of Z wins in ZvP at each interval', which is fundamentally different from 'at each interval, if the game ends, how often does it end into Z's favour in ZvP'. My bet is that because 7pools are actually quite rare, it would not at all show the huge spike for Z in the early game.
But I do have to say that THIS section, was a very good point, that people completely misunderstand.
Frankly i like any post that rejects that we have any real understanding of imbalance.
|
good job mods. i'm glad you keep people like this out of our community. this guy calling TL pretentious.........
|
What the guy says is nothing new, but a lot of posters on this forum take the tlpd statistics for more than it is.It's true what he says, you can't judge balance by those statistics, especially not over the time period of 2 or even 3 months.
|
The tone is annoying but all the points he makes are valid. And not all of them are obvious either.
|
So basically that guy on reddit says something that we already know and he thinks that we don't know in a very arrogant and condescending tone. I don't see a reason for this thread to exists since these points have already been expresses multiple times in for say the win rates thread every month. The only reason I can think of is for him to feel smarter then everyone else which is a dick move.
|
I'm pretty sure anyone who actually cares about the stats understands the basics he is attempting to explain. Anyone who has taken a statistics course will learn about logical fallacies before almost anything else. It's wonderful to see that he thinks people who frequent TL are incapable of pointing out obvious errors. I'm pretty sure almost any thread with stats in it gets analyze critically by multiple people...in fact it's probably the first thing that happens when a long post with a lot of graphs and charts pops up. The only thing about it is that the people who really don't care post in the thread first saying how great the OP is without having read it all. Jump to page 3 or 4 in those threads and someone will be correcting all of the mistakes or critiquing the post. TL isn't full of idiots and I think the winrate charts are widely understood to simply be something interesting to see...not to balance a game off of. Of course there are people who open up TL and find threads to whine about balance in. These people have major confirmation bias and it should be expected to see it pop up in a thread where balance is potential up for discussion.
Keep your ego in check...your high horse is a lot shorter than you think. If you'd stop to look up you'd be able to see that there are plenty of people on your level and above you and your pretentious comments.
|
On May 04 2012 09:16 Ripper41 wrote:Show nested quote +On May 04 2012 09:09 halfies wrote:
praetentious?
really?
i hope that how he spelt it too, because it would be really funny if he wrote this much about people being pretentious and used big words that he couldn't even spell. Well he explains...it's worse, he spelled it that way intentionally. still, he makes some good points in the substance of what he says.
so hes spelling pretentious wrong because hes pretentious? oh the irony.
he certainly doesnt seem to say anything wrong, but theres not much thats really enlightening either
|
Please, could you all stop being so insulted (wrongly insulted at that, since it was aimed at redditors in the first place) and actually try to read it? Contrary to what many have posted in their attempts to bring some of the hurt from their bleeding egos back to the proxied OP, it was an interesting read with interesting perspectives.
|
On May 04 2012 18:29 straycat wrote:
Please, could you all stop being so insulted (wrongly insulted at that, since it was aimed at redditors in the first place) and actually try to read it? Contrary to what many have posted in their attempts to bring some of the hurt from their bleeding egos back to the proxied OP, it was an interesting read with interesting perspectives.
That most people already knew.
|
On May 04 2012 18:45 Chunhyang wrote:Show nested quote +On May 04 2012 18:29 straycat wrote:
Please, could you all stop being so insulted (wrongly insulted at that, since it was aimed at redditors in the first place) and actually try to read it? Contrary to what many have posted in their attempts to bring some of the hurt from their bleeding egos back to the proxied OP, it was an interesting read with interesting perspectives.
That most people already knew.
If that was true, why would there be so many posts containing balance whine with "supporting" their viewpoints by exactly the type of graphs criticized?
|
On May 04 2012 18:55 Cyberonic wrote:Show nested quote +On May 04 2012 18:45 Chunhyang wrote:On May 04 2012 18:29 straycat wrote:
Please, could you all stop being so insulted (wrongly insulted at that, since it was aimed at redditors in the first place) and actually try to read it? Contrary to what many have posted in their attempts to bring some of the hurt from their bleeding egos back to the proxied OP, it was an interesting read with interesting perspectives.
That most people already knew. If that was true, why would there be so many posts containing balance whine with "supporting" their viewpoints by exactly the type of graphs criticized?
Hey, Mr. Statistician, your degree in statistics should give you the answer.
|
On May 04 2012 18:55 Cyberonic wrote:Show nested quote +On May 04 2012 18:45 Chunhyang wrote:On May 04 2012 18:29 straycat wrote:
Please, could you all stop being so insulted (wrongly insulted at that, since it was aimed at redditors in the first place) and actually try to read it? Contrary to what many have posted in their attempts to bring some of the hurt from their bleeding egos back to the proxied OP, it was an interesting read with interesting perspectives.
That most people already knew. If that was true, why would there be so many posts containing balance whine with "supporting" their viewpoints by exactly the type of graphs criticized?
This is the biggest issue I have with people saying this is 'common knowledge', if people were/are already well aware of the specific issues with these graphs (i.e. not just for example the general idea of what a confounding variable is) then many of the stat-pointing balance whine posts would either A) not exist or B) not be taken as seriously by the majority. Yet people freely point to these graphs as a beacon of balance truth whenever they get posted and nobody says a word.
Also, if these issues were common knowledge, then why haven't statistic-based threads such as the 'win rate over game length' one addressed them specifically? This kind of scrutiny should be common practice in statistics threads, and if it isn't mentioned by the OP then posters should be critical of this kind of omission because of how integral it is to the meaningfulness of the statistics.
The winrates post is an even worse example of a statistics thread, created completely devoid of any statistical analysis whatsoever and with very few people being critical of the graph and none of them being critical of the thread. Why do simple undeveloped threads always get removed yet simple undeveloped graphs persist? Regardless of whether people were aware of what OP said or not, it really needs to start being put into practice if we want to have proper discussions about trends in SC2.
|
On May 04 2012 19:06 KuKri wrote:Show nested quote +On May 04 2012 18:55 Cyberonic wrote:On May 04 2012 18:45 Chunhyang wrote:On May 04 2012 18:29 straycat wrote:
Please, could you all stop being so insulted (wrongly insulted at that, since it was aimed at redditors in the first place) and actually try to read it? Contrary to what many have posted in their attempts to bring some of the hurt from their bleeding egos back to the proxied OP, it was an interesting read with interesting perspectives.
That most people already knew. If that was true, why would there be so many posts containing balance whine with "supporting" their viewpoints by exactly the type of graphs criticized? Hey, Mr. Statistician, your degree in statistics should give you the answer.
haha, good point. However note that OP/author are not the same!
|
Why are people in this thread badmouthing this? It's completely correct and it's something a lot of people get wrong. If "praetentious" and "reddit" weren't said no one would mind.
|
On May 04 2012 18:45 Chunhyang wrote:Show nested quote +On May 04 2012 18:29 straycat wrote:
it was an interesting read with interesting perspectives.
That most people already knew.
1. Sigh.
2. QED.
|
Oh well, that guy is writing a romane on how winrate statistics are biased because ability is an unobservable variable. That is hardly new, but we take a look on winratios nevertheless, because an important function of statistics is to facilitate discussion and communication.
|
On May 04 2012 16:31 oZe wrote:Show nested quote +On May 04 2012 09:09 halfies wrote:
praetentious?
really?
i hope that how he spelt it too, because it would be really funny if he wrote this much about people being pretentious and used big words that he couldn't even spell. You don't even begin a sentence with a capitalized letter, made even more fun by the fact that it was "I". Thereby forfeiting all rights to complain about spelling. Also if you have a problem with his argument, attack his argument. In American English, spelt means exclusively a hardy wheat grown mostly in Europe, and the verb spell makes spelled in its past-tense and past-participial forms. Ps. I put my money on you being the creator of the bad boy ^^
i wasn't complaining about anything except how pretentious it is to spell pretentious like that for the reasons he does it. i don't care about the spelling aspect as much as the reasons for it, which also explains why i don't bother to capitalize everything properly on a forum, since it doesn't matter and is easier.
|
His overly agressive tone notwithstanding, he actually made some excellent points, the most important of all being that what matters is the number of players, not the number of games. Also good points about all-in defense making the defending race "favored" in the next few minutes.
|
I remember seeing Artosis post something similar before.. Yes people (and hence mods because mods are people right?) like seeing nice and simple graphs/figures/conclusions but I don't think TL has more of that than anywhere else.
On May 04 2012 21:23 halfies wrote:Show nested quote +On May 04 2012 16:31 oZe wrote:On May 04 2012 09:09 halfies wrote:
praetentious?
really?
i hope that how he spelt it too, because it would be really funny if he wrote this much about people being pretentious and used big words that he couldn't even spell. You don't even begin a sentence with a capitalized letter, made even more fun by the fact that it was "I". Thereby forfeiting all rights to complain about spelling. Also if you have a problem with his argument, attack his argument. In American English, spelt means exclusively a hardy wheat grown mostly in Europe, and the verb spell makes spelled in its past-tense and past-participial forms. Ps. I put my money on you being the creator of the bad boy ^^ i wasn't complaining about anything except how pretentious it is to spell pretentious like that for the reasons he does it. i don't care about the spelling aspect as much as the reasons for it, which also explains why i don't bother to capitalize everything properly on a forum, since it doesn't matter and is easier.
The writer's pov was that people don't bother looking into details and try to talk about things they don't know much about. How is your post not proof of that.
|
I agree to the point he is trying to make, that statistic is not something you can trow out without context. And it's true that's a science but what else do we have to try to check if the game is balance? I doubt that even blizzard can check every player if they are pushing it in their favor just by being good player.
Still think it's the best way to check the game.
PS: sorry for the mistake, english is not my primary language.
|
Do I or most people probably like the way he went about that post, probably not at all.
Does he make some points that some people realized exist, but just largely ignore because of the 'not worth it' factor. By all means yes, that's the case.
Let me expound upon what I've said above so it's not misconstrued as to what I was getting at. The VAST majority of people who either A) Care about the game or B) Care about the competitive arena of the game, are on the overall pretty satisfied with the product. Though I'm sure someone can find a statistic that states otherwise
I could make a long winded example, but I'll post the TLDR version of why most people fall under the 'not worth it' factor in regards to statistical posts.
1. The majority of people making a complaint based on a bias and statistics will not be swayed in their opinion. If they took the time to find a stat to try to prove a point, they are probably going to hold fast to that point and will die before the yield. That however, won't stop equally biased people for or against from arguing in a thread.
2. Most people realize someone posting statistics to make a point doesn't care if they are reading the stats wrong or right. They are trying to vent and make a point, so arguing with them will just make you go around in circles.
Overall I find his bashing of a particular community pointless, just as pointless as arguing with someone who has a clearly biased opinion on a matter. I wouldn't condone someone bashing on any community, by making a blanket judgment based on a very small statistical sample size to lump the whole community into it (see what I did there).
|
I thought it was a very bad and mostly uninformative post. the OP admitted to not even knowing that much about statistics, and you can tell by what he writes.
|
On May 05 2012 03:27 latan wrote:
I thought it was a very bad and mostly uninformative post. the OP admitted to not even knowing that much about statistics, and you can tell by what he writes.
Post isn't from the OP, post is quoted from a man on reddit.
|
how about saying what we CAN get from the statistics and how things SHOULD be done instead of saying"this and this and this is bad".
|
Drabzalver just created arbitrary fallacies and then disproved them. They don't even necessarily correlate to what we argue over.
Basically he's just smugly (or, one might say, "praetentiously") knocking down his own strawman arguments and then talking about how shitty TL is.
But now I'm glad to know that 1000 games of MKP vs. DRG doesn't define TvZ balance. Of course, he doesn't talk about how that particular example is nonsensical because it also necessitates how they'll learn each other's playstyles and the like (e.g. MKP's strategies don't represent all high-level Terrans' mindsets), and so Drabzalver's anecdote is pretty much garbage.
But his focus on how the common cause of a player winning two matches in a row 4-0 is that he's "fucking good" was so polished and well-spoken... I don't know how I could have missed such a revelation without it being pointed out to me by this guy.
I'm also glad I learned from him that 7 pools are quite rare though. And that Reddit really does live up to its reputation.
|
Fallacy 1: Mostly true, but when you examine 3 months of data and average it, assuming there haven't been any major changes in metagame, maps, or patching, you can very easily draw conclusions about balance based on winrates alone. April/March/Februrary works pretty well.
Fallacy 2: Not true and plainly really dumb. Internationally, there are enough players in each race to balance out differences in player skill between races. In Korea, this may not be as true, but to claim that MKP alone will shift the balance of a matchup internationally is stupid. Nobody is that dominant in sc2.
Fallacy 3: Agree wholeheartedly, can't really control for build order/bo losses.
Its kind of academic now, as balance is very close to 50/50/50 internationally and nothing is really broken. Teamliquid just has some insane bias against protoss in pvt despite strong evidence contrary to popular beliefs.
|
Hah, he calls out all the statistical bs perfectly. Maybe not in the kindest way, but he nailed it mostly. The TLPD non-korean chart is actually useful, and as SC2 progresses it will become much more accurate as the skill gap between players narrows.
This is also exactly why Blizzard is loathe to release a lot of numbers, because people have no idea what the hell they're looking at, and will make crap up perfectly to fit their answers.
|
Seems like just another guy who is just full of him self and wrote a self masturbation post.. I like how he doesn't have a TL account and seemingly thinks he knows he smarter than 99% of everyone here bwahahaha.
|
everyone that studied at least half year statistics knows that statistics that are used on this forums by players don't mean shit on balance why do you think there has do be done solid math/statistical hypotesis testing to do reliable deduction about balance.
|
People need to stop complaining about the author's writing style and aggression and focus on the content. He is 100% about the use of statistics in the SC2 scene. They are worthless. They are actually worse than worthless because they give people the illusion of knowledge where none exists.
|
The general points are good, the specifics are not and his conclusions are not. The statistics that people generally use to judge balance have lots of fallacies and are used in misleading ways to confirm prejudicial beliefs. They have lots of problems and aren't particularly informative. But the OP needs to provide a better alternative or even some other direction that we should go. Should we abandon trying to statistically analyze StarCraft altogether? Should everyone just STFU about balance?
|
On May 05 2012 06:15 coverpunch wrote:
The general points are good, the specifics are not and his conclusions are not. The statistics that people generally use to judge balance have lots of fallacies and are used in misleading ways to confirm prejudicial beliefs. They have lots of problems and aren't particularly informative. But the OP needs to provide a better alternative or even some other direction that we should go. Should we abandon trying to statistically analyze StarCraft altogether? Should everyone just STFU about balance?
you can statistically analyze your own games, to draw conclusions (to understand your weaknesses to fix them)
Still has nothing to do with balance but just your results.
|
Like the guy or not he's completely right, and this should be included as a disclaimer above any post talking about winrates.
|
On May 05 2012 06:15 coverpunch wrote:
The general points are good, the specifics are not and his conclusions are not. The statistics that people generally use to judge balance have lots of fallacies and are used in misleading ways to confirm prejudicial beliefs. They have lots of problems and aren't particularly informative. But the OP needs to provide a better alternative or even some other direction that we should go. Should we abandon trying to statistically analyze StarCraft altogether? Should everyone just STFU about balance?
Sometimes admitting that there is no strong answer currently is the best option. If you haven't read Taleb's The Black Swan or Fooled By Randomness I really recommend it. The issue isn't just attempting to do statistical analysis with insufficient data and improper methods, but an overconfidence in the results and the blind acceptance by a mathematically inept community.
|
Some pretty obvious shit everyone should know simply by employing some plain old common sense, and probably gets said in every single statistics thread. And then coupled with random bullshit. It should be titled what most people already know. Keep crappy reddit rants on reddit please.
What's even the point of arguing overall balance anymore? There isn't one. Whatever issues remain are marginal at best in the grand scheme of things.
|
people study statistics for years, so that most people don't notice how far they stretched the stats in their favor. Otherwise statistics are fairly simple to read. But if people make graphs out of them and show you, they are fishy either way.
|
Sure is pretentious reddit circlejerk in here. I didn't get to finish that long pretentious post because I don't think you know how statistics work.
Why not make more dramas and upboat more just to get le karma XDXD and let TL do their thing.
|
TL is very praetentious. Lol
|
|
|
I completely agree with the author.
To the people who think it's common sense what he wrote I have to say that there are plenty of posts on team liquid which make exactly the mistakes he mentioned.
|
On May 04 2012 09:00 Cyberonic wrote:
Drabzalver
- Fallacy: winrates indicate balance
Yes and no, for the most part, they do not indicate balance, rather, they indicate balance shifts. It's so trivial to see that it boggles me that people don't figure this out on their own. Assume that race X was actually underpowered last month and is balanced this month. This means that the X players who actually qualified last month and stayed in tournaments are actually better than the Y and Z players, therefore, now that it gets balanced, as they are overal better, they start smashing Y and Z because they are better, thereby suddenly making the graph appear as X has been 'overbuffed' or whatever else while simply the few X players that were around in the scene were better. This will continue on until the mediocre Y and Z players who were more so carried by their race than the X players get weeded out.
...
The only way to find out absolute balance is to get a random pool of pros, force them to play random, and have a round robin tournament to ensure that everyone plays the same amount of games. Very unfeasible to get enough games with that for a reasonable sample size.
It's worse than this. Your definition of balance shifts is that it varies with the distribution of better players in a month's tournaments. But it's not just players that are shifting, it's their meta-game. Say Naniwa figures out a new mothership rush style tomorrow that Zerg has no answer for. The win rate for P vs. Z during that month is going to be lopsided in favor of Protoss because all the Protoss pros are going to switch to it and crush their Zerg opponents. But then a few weeks later, Stephano figures out a three hatch mass ultra style that beats the crap out of the mothership rush. Then the pendulum swings right back and those Protoss who practiced their arses off on how to rush motherships is going to get creamed until they readjust.
Because of this, getting a random pool of pros, forcing them to play random, and having a round robin tournament to ensure that everyone plays the same amount of games is still isn't going to give you absolute balance - because the pros are going to play each race the way the meta-game tells them to play, which gives you the balance of the current meta-game but not the balance of the game.
A lot of obvious imbalances are detectable through observing lopsided match-ups and thinking through the game logic to see that the lopsidedness is due to game design and not player skill, but there is no way to figure out the absolute balance of the game without solving the game, and thereby knowing the optimal strategies in each match-up.
|
Anyone who says "pray-tentious" is probably the most pretentious bastard alive. I don't understand why this shit has to be shared on Team Liquid. What the fuck is reddit even good for if we just discuss it all here?
|
It's too bad the OP of the reddit thread is such a douche, because there are probably feasible ways to account for some of the statistical inaccuracy to produce "truer" results. However, the TLPD graphs are by no means useless as approximations for the true statistics and attempts to characterize them as such are probably equally misguided as carrying them as gospel.
|
Kinda sad that in typical TL fashion, somebody makes an intelligent correct post, and instead of discussing it, they spend 7 pages complaining about his spelling/declaring they already knew everything he said.
Good post, well written, statistics very rarely have useful connotations in the world of Starcraft, and are reserved only for people with bogus arguments they're trying to defend.
|
|
|
|
|
|
EPT
