|
I apologize to everyone in this thread for taking the OP seriously. My mod senses are definitely off today.
-- Chill |
On February 16 2011 05:28 Draconicfire wrote:Show nested quote +On February 16 2011 05:26 Centorian wrote:
A poll could work to get a lot of information very quickly.
Play metalopolis against the computer for one game. Answer poll according to your result.
However, it would be easy enough for people to lie, or use the poll based on their feelings and not actually doe the test. What if we started a research thread where people can post their replays on Metal and then tally it from there? The only issue I see is if the same two people post the same replay, so it would count twice.
Shouldn't happen if they play vs a computer, sounds like a good idea. Another problem is people could selectively give replays of them spawning in a certain position but I see no reason why someone would do that..
|
[QUOTE]On February 16 2011 05:28 Draconicfire wrote:
[QUOTE]On February 16 2011 05:26 Centorian wrote:
The only issue I see is if the same two people post the same replay, so it would count twice.[/QUOTE]
This isn't an overwhelmingly large problem if we have a large sample size. It not a confounding third variable because it doesn't affect close/air/far ratio. It decreases accuracy, but it won't disfigure the results.
|
On February 16 2011 05:26 DueSs wrote:
Ah, I enjoy the banter--but really this is just measuring statistics-dick sizes. The OP just tried something as a hunch and posted his results. It's interesting and perhaps more valuable as something that might be a real thing. It instigates discussion and intrigue. It's a good thing.
ahh but would he have posed anything if his results were within the norm??
|
On February 16 2011 05:21 MonsieurGrimm wrote:
We could make a new thread about it, and have people put their names in and have everyone do a sample of 10-20 then report back to the thread so nobody has to waste an hour on it...
This is a good idea, but we need to be a bit careful about it. For example only count the first 50 people or 100. And this must be decided before counting starts.
Won't make a huge difference, but since this is almost a survey, it is important to be strict about what you are measuring. Otherwise you can keep measuring until you get the answer you want.
I just like rigour. 
|
That's not the whole story. It's possible that thousands of people are bothered enough by the observed frequencies to run tests. Of these only one leads to exceptional result and it's the only one that gets reported. OPs testing method is correct from his point of view, but underlying "drawer effect" should change how we interpret his result.
Yup good point. People have to remember this when they look at polls and surveys too. Etc.
OK guys i have an idea. Everyone Start up ONE game against the computer! then vote in this pole with which spawn you got. This could increase the sample size by alot.
Haha good idea, fast and convenient!
Edit: Air :D
|
On February 16 2011 05:29 italiangymnast wrote:OK guys i have an idea. Everyone Start up ONE game against the computer! then vote in this pole with which spawn you got. This could increase the sample size by alot. Poll: What spawn position did you get?Close Ground (53) 54% Close Air (23) 23% Far (23) 23% 99 total votes Your vote: What spawn position did you get? (Vote): Close Ground
(Vote): Close Air
(Vote): Far
I'm already going to call bullshit on the first couple of votes. I refreshed the page and the poll just came up with no votes. I refreshed 2 seconds later and there were 2. Clearly they did not start a game against the computer like was instructed.
|
On February 16 2011 05:12 Liquid`Tyler wrote:Show nested quote +On February 16 2011 05:09 aristarchus wrote:
TBO wins the thread. This is definitely a statistically significant result, but I still don't believe it. Lots of people probably try this stuff, and a small number get really weird results and then post here. (This is a problem with academic research too - the way you settle it is to do more tests independently, and also to have some healthy skepticism about the likelihood that blizzard screwed something this simple up in that weird a way in the first place.) How does he win the thread? While he makes a relevant point, he hasn't provided the data to prove that the phenomenon he claims is happening is actually happening. At the moment, he's relying on faith that nothing has caused a problem in SC2's ability to give random positions on Metalopolis.
I'm not sure what data he could possibly provide. He's saying that probably a lot more people than this have runs of bad luck and then say "Oh, I'll test this out." Of those people, some number will get statistically significant results even if there's nothing to find. Given the frequency of people on team liquid (the website, not the team) doing this sort of stuff, I think it's a pretty reasonable bet that lots of people try stuff like this and then find nothing and don't post about it. I'm not trying to say this shouldn't be investigated further, just that I think it's reasonable to see the false positive scenario as the more likely scenario. Remember that the other scenario involves blizzard failing to implement a simple rng correctly, and then thousands of other players never noticing that metalopolis spawns close positions with double the frequency it should. (If it was only off by one or two percentage points, it wouldn't explain these numbers at all.)
If you want to solve the problem TBO is talking about, you need to make sure that whoever is doing the confirming experiments for this result will report their results either way. Clearly people should test it themselves, but they should commit to doing so and posting the results regardless of what they find, so that this bias of only seeing significant results can't explain the findings.
|
i started a game and checked. then made the poll. so the first vote is mine, and legit.
|
On February 16 2011 05:31 gaheris wrote:Show nested quote +On February 16 2011 05:26 DueSs wrote:
Ah, I enjoy the banter--but really this is just measuring statistics-dick sizes. The OP just tried something as a hunch and posted his results. It's interesting and perhaps more valuable as something that might be a real thing. It instigates discussion and intrigue. It's a good thing. ahh but would he have posed anything if his results were within the norm??
This is a real problem in science and it is called publication bias. Look it up on wikipedia for example.
Say you have an idea for cold fusion. You then discover it is not possible for reason X. This is actually relevant information for the community, but it won't get published because the research "didn't go anywhere".
|
On February 16 2011 05:33 JoeSchmoe wrote:Show nested quote +On February 16 2011 05:29 italiangymnast wrote:OK guys i have an idea. Everyone Start up ONE game against the computer! then vote in this pole with which spawn you got. This could increase the sample size by alot. Poll: What spawn position did you get?Close Ground (53) 54% Close Air (23) 23% Far (23) 23% 99 total votes Your vote: What spawn position did you get? (Vote): Close Ground
(Vote): Close Air
(Vote): Far
I'm already going to call bullshit on the first couple of votes. I refreshed the page and the poll just came up with no votes. I refreshed 2 seconds later and there were 2. Clearly they did not start a game against the computer like was instructed.
Yeah the issue is people lying. Need to do something that involves posting replays. Which means it works easier if people do more than one.
|
I still say this is anecdotal bad luck. I can flip a coin 10 times and end up with 7-3 ratio, when everyone will agree it should be close to 5-5 assuming a "fair" coin. 3-7 is just as likely when really random is just random.
Unless you want to explore the possibility that results are being skewed based on races, start times, mmrs, random number defect, etc. I don't think you can make any conclusions on 1 result. Who is to say the next 100 games you play aren't skewed in some other fashion and the OP didn't just get a bad string of close positions?
|
On February 16 2011 05:36 Centorian wrote:Show nested quote +On February 16 2011 05:33 JoeSchmoe wrote:On February 16 2011 05:29 italiangymnast wrote:OK guys i have an idea. Everyone Start up ONE game against the computer! then vote in this pole with which spawn you got. This could increase the sample size by alot. Poll: What spawn position did you get?Close Ground (53) 54% Close Air (23) 23% Far (23) 23% 99 total votes Your vote: What spawn position did you get? (Vote): Close Ground
(Vote): Close Air
(Vote): Far
I'm already going to call bullshit on the first couple of votes. I refreshed the page and the poll just came up with no votes. I refreshed 2 seconds later and there were 2. Clearly they did not start a game against the computer like was instructed. Yeah the issue is people lying. Need to do something that involves posting replays. Which means it works easier if people do more than one.
You cant stop people from lying if they really want to. They could just post a few extra close position replays and trick you that way.
|
RNG is RNG. Just as the comic shows, getting the same results might not be the map's fault, but could be the equation they use for picking spawn positions for maps with 4 spawning positions.
I'm gonna assume this, it is probably not how spawning positions are decided. I just assumed it for the sake of explaining.
Let's assume this, the game at first decides where Player A spawns, then where Player B.
1. At first, a random number between 1-120 is generated (RNG-A).
* If [1 = RNG-A > 31] Then player A spawns at 1st position.
* If [31 = RNG-A > 61] Then player A spawns at 2nd position.
* If [61 = RNG-A > 91] Then player A spawns at 3rd position.
* If [90 > RNG-A = 120] Then player A spawns at 4th position.
Now how the 2nd spawning position is decided could probably be done by 2 methods that I can think of on the top of my head, one is redoing the same as 1st step and in case the RNG-B range is the same as RNG-A, then RNG-B is regenerated again until a number in different region is given then the 2nd spawning point is decided, or it could be decided by what I would use if I was to code it, which is:
2. Reassign the remaining spawning points to 1st-B, 2nd-B and 3rd-B.
* If [1 = RNG-B > 41] Then player B spawns at 1st-B position.
* If [41 = RNG-B > 81] Then player B spawns at 2nd-B position.
* If [80 > RNG-B = 120] Then player B spawns at 3rd-B position.
With this method of deciding spawning positions, the game doesn't take into consideration where Player-A spawns to determine where Player-B spawns other than the fact that both players can't have the same spawning position.
RNG is just random, sometimes you get a streak of repeated values due to the equation the coder used to generate the RNG. I remember my days when I played WoW and we'd get the same loot from bosses for weeks eventhough according to Blizzard, all the loot had the same drop chance.
|
On February 16 2011 05:33 aristarchus wrote:
I'm not sure what data he could possibly provide. He's saying that probably a lot more people than this have runs of bad luck and then say "Oh, I'll test this out." Of those people, some number will get statistically significant results even if there's nothing to find. Given the frequency of people on team liquid (the website, not the team) doing this sort of stuff, I think it's a pretty reasonable bet that lots of people try stuff like this and then find nothing and don't post about it. I'm not trying to say this shouldn't be investigated further, just that I think it's reasonable to see the false positive scenario as the more likely scenario. Remember that the other scenario involves blizzard failing to implement a simple rng correctly, and then thousands of other players never noticing that metalopolis spawns close positions with double the frequency it should. (If it was only off by one or two percentage points, it wouldn't explain these numbers at all.)
If you want to solve the problem TBO is talking about, you need to make sure that whoever is doing the confirming experiments for this result will report their results either way. Clearly people should test it themselves, but they should commit to doing so and posting the results regardless of what they find, so that this bias of only seeing significant results can't explain the findings.
It's always possible that the reported results will be so extreme that it's significant even if all negative results remain unreported.
Sure, good protocol is essential if you want to get a very accurate result or the effect is smallish. But if it's blatantly obvious (as it is, if OPs numbers are actually accurate) then you can be more casual about it and still be confident in your conclusion.
|
On February 16 2011 05:09 Liquid`Tyler wrote:Show nested quote +On February 16 2011 05:02 TBO wrote:On February 16 2011 04:58 Frozenserpent wrote:
People are fucking retarded and need to learn some basic statistics before they say something like "not big enough sample size".
Even 50 can be sufficient to obtain a p-value < 5%.
In this case, p-value is definitely low enough to reject the assumption that it's 1/3rd.
Obviously if you want the exact ratios you'd want to expand on the larger sample size, but this is conclusive to determine that it's not 1/3rds. The problem here is that you have millions of players who play 100 games on metalopolis and a few of them will get extreme results (and those will post in the forums), even if it is 1/3 chance. If you have 100.000 people throwing a coin 15 times, you will get quite a few (6 in average) who will get a 15-0 or 0-15 result. Only if lots of people get the same results as the topic creator, one could assume it is conclusive. Show nested quote +On February 16 2011 05:05 Soma.bokforlag wrote:On February 16 2011 04:58 Frozenserpent wrote:
People are fucking retarded and need to learn some basic statistics before they say something like "not big enough sample size".
Even 50 can be sufficient to obtain a p-value < 5%.
In this case, p-value is definitely low enough to reject the assumption that it's 1/3rd.
Obviously if you want the exact ratios you'd want to expand on the larger sample size, but this is conclusive to determine that it's not 1/3rds. you shouldnt call people retarded when you obviously doesnt understand statistics. if you make enough studdies some of them will turn out faulty results even if the method is correct 99.9% of the time. in this example, it is possible that the author of the thread felt "damn, i get alot of close positions" and therefore examined his stats.. the rest 99.99% of players which have a more even distribution never gave it a thought and probably are closer to correct ratios edit. TBO was quicker than me.. Ah, you guys aren't likely to get your PHD's when you can't even understand the OP's testing methods. He didn't examine the games that gave him the feeling that he got close positions. He got that feeling and ignored those games and loaded up 100 more games against the computer and recorded those results.
My bad should read have read the thread more careful. However my statement is still partly true, at least if he has not been the only person doing the test. But just the only person posting it because he got this extreme results. And the results isn't too extreme btw, cumulated probability for close spots being 72 when probability for close spot being 1/3 in a sample size of 100 is 0,1066.
Only way to really find a conclusive result is to just have a few more runs with a sample size of 100 (actually smaller sample size like 50 should be fine as well if we just run enough of them).
Just wanted to start my sample, and then realized that to find out which spots you are you need to scout, which takes a lot of time... I think it would be useful to create a custom map which has fog of war disabled. Or is there a singleplayer cheat which removes FoW? (That assumes that singleplayer positioning is based on the same algorithm, which it should be)
|
If anyone wants to increase the same size, we can all log on and start 10 or more games on metal, and then add those results to the ones from the op. When I get back from
Class I'll do this and edit the post with results. Anyone else down?
|
I usually spawn cross position on metal. I'd be really surprised if it was a ratio as extreme as 7:1:2.
|
On February 16 2011 04:52 magha wrote:Taking sample sizes of 100 would prove that nobody in history has ever won a lottery.
This is just wrong.
If the OP has is not screwing with us.
and if the different trys are independent, getting 72 out of a hundred given that the probability is 1/3 i far more unlikely than winning the lottery.
A standart p-test with x=72 and n=100 and p0= 1/3, will give you a p value of 1.19*10^-16 (thats very low for people who dont know math) The p-value is the likelyhood that the real probability is p0 (or less)
http://en.wikipedia.org/wiki/P-value
So this means that either the OP made a mistake in the test or blizzard has made a mistake in the algorithm.
|
:o, that surprises me, because I get crosspo all the time, as Terran, it doesn't make me too happy
|
We just need to talk to the SC2GEARS dude (Dakotafanning?) and ask him if its possible to whip up a position analyzer bit in the next release. Compared to what he's already done it should be simple and once run on even one regular players replay library you should have solved the issue.
|
|
|
|
|
|
EPT
