|
My guesses:
1. 38 years
2. 4000 miles
3. 13 countries (100% certain)
4. 39 books (46 in the Catholic canon)
5. 2000 miles
6. 250 000 pounds
7. 1756 (100% certain)
8. 350 days
9. 6000 miles
10. 50 000 feet
For #6 I extrapolated approximate weight from the upper estimates on the mass of the largest dinosaurs, which were of similar length to the Boeing 
For #8 I knew that elephants were one of the few animals which had a longer period than humans, but thought it was about a year, or 16 months at most. My memory on this one was obviously faulty.
For #10 my memory was also faulty, as I recalled the deepest point of the Mariana trench being about double the height of Everest.
|
|
|
South Africa4316 Posts
On July 31 2009 01:09 searcher wrote:Show nested quote +On July 31 2009 00:46 Daigomi wrote:
It's funny, I've come across this exact test three times in the last two weeks (all in psychology text books).
I'm busy writing a research proposal for my postgrad studies, and my topic is "the use and prevalence of heuristics in day traders," so I've been reading a ton of stuff on stock market psychology, and this specific overconfidence test is included in almost all of the books. I got 1/10 for it, and missed about 4 of them with about 5%, so clearly if I was just a bit less confident, I would have done much better :p That's a potentially interesting paper that I'd be interested in reading. I think the experiment is designed fine, just that the conclusion to be drawn is "people have no idea what a 90% confidence interval means" rather than "people are overconfident".
Strangely enough, the topic I gave earlier was my preliminary topic before I did my literature review. I've since then decided to focus specifically on the anchoring and adjustment heuristic, and the 90% confidence level you mention is explained quite nicely by anchoring and adjustment.
Basically, when you provide people with a figure, they take that figure and adjust it in the direction they believe the answer lies, but adjust it insufficiently. So if you say "Was Mozart born before or after 1850?", people will look at that 1850, and think "it was definitely before 1850, so I'll say about 1785". In contrast, if you ask people "Was Mozart born before or after 1650?" they will look at the question and think "it was definitely after 1650, so I'll guess about 1715." So basically, you anchor on a starting value, and then adjust insufficiently in the direction the answer lies.
The same happens if you define an anchor for yourself. For example, with the question "how old was MLK when he died?" I did the follwoing. I know that average life expectancy was 65-70, and the fact that I'd never seen a photo of MLK as an old man made me think that he died young, so I took the 65-70, aeddjust it downwards to account for his young age, and I ended up guessing he was about 45-50 when he died (thus, my adjustment wasn't sufficient). This guess should be a reasonable odds guess, lets say 65:35. Then you adjust the value even more to increase the probability of you being right, but once again this adjustment is not significant enough. I adjusted mine to 40-65, missing his birth with 1 year. However, if I really wanted to be 90% sure, I should have adjusted it to 30-75 probably.
|
Can't imagine this is a good judge of over confidence, if you can't admit you don't know something then you're ignorant and conceded...
|
It's more a test of memory. Most people should have come across most of these "facts" sometime in their lifetime, and some people are better with recalling numbers than others.
|
Some questions are related to U.S. users only. Some facts I came across are only in the metric system.
|
South Africa4316 Posts
On July 31 2009 02:11 MoltkeWarding wrote:
It's more a test of memory. Most people should have come across most of these "facts" sometime in their lifetime, and some people are better with recalling numbers than others.
It's not a test of memory. If it was a test of memory, people would simply be asked the answers, not asked to give a range.
The questions were chosen specifically to be arbitrary, with most people not knowing the answer (at least not exactly). If people don't know the answer exactly, they can define a range which they will have confidence in. The point of the exercise is to show that people who are too confident define ranges that are too small.
Also, the fact that these results are unexpected is important. If they just asked you the age of someone who died at what is an expected age (lets say 65), then most people would get the right answer right "by accident", and it would have nothing to do with their confidence. These questions are made to be unexpected, but that's because, to be 90% sure of a question you don't know the answer to, your range should include unexpected results. For example, Mozart's birth. If you know that he was born in the 18th century somewhere, and he died late in the 18th century, then you might be 50% confident that he was born somewhere between 1740-1760, since people didn't get too old in those days. However, to be 90% confident, you should increase that range to 1710 (in case he live until he was 70 or 80 years old) to 1770 in case he died in his twenties still. Obviously the more knowledge you have, the more confident you can be and the smaller your range can be.
The fact is, for you to be 90% confident, you should be willing to lose 90$ for the chance to gain 11$ if you're right. So, if you don't know the answer, you should choose a range that would include unexpected answers, otherwise you'd just lose money on all the questions that you didn't know the answer to.
And to those saying it's not a good judge of confidence, it is only a quick test, so it's not perfect. There are way more detailed psychological tests on the topic. However, both the validity and reliability of the test are fairly high, so it's not a bad test either.
|
On July 31 2009 00:58 Chill wrote:
This is stupid because the answers in themselves are shocking. Sure you should account for that in your interval, but the test is pitted against you. If they were questions with reasonable answers in everyday things this would hold a lot more weight for me.
If someone asked me what was the greatest length from one point to another you could draw across the known universe, in feet, I really don't know the scope of the answer and would have to give an interval that goes across many orders of magnitude- same if I was asked for the weight of an atom, in pounds.
The only reasonable thing to do is to give yourself humongous error margins for questions about which you know very little about. I had a 14-4000 day range for the gestation period of an elephant, and I got 10/10.
|
17068 Posts
Have you been reading The Black Swarn by Taleb?
|
I don't really know the answers to any of these question. The closest I got was at the deepest known point in the ocen, but having to transform meters in feet is pretty hard.
Perhaps you should get an easier set of question so you could actually get more feedback from more people. The less the cultural enviroment counts the better.
|
South Africa4316 Posts
Also, it It's important to point out that this is not a pub quiz or an IQ test, you don't beat the test by getting 10/10, or lose by getting 1/10. It's a measure of overconfidence. Getting 1/10 meant you were too confident in your ranges. Getting 10/10 meant you were not confident enough with your ranges. That's all.
|
On July 31 2009 02:34 Empyrean wrote:Have you been reading The Black Swarn by Taleb?
I've heard a lot about that book, though I haven't read it personally. I'm not interested enough in finance to read it for now, though its conclusions are pretty enlightening.
|
On July 31 2009 02:37 FirstBorn wrote:
I don't really know the answers to any of these question. The closest I got was at the deepest known point in the ocen, but having to transform meters in feet is pretty hard.
Uh... wat? You multiply by 3 and get a pretty close conversion :p
|
Katowice25012 Posts
On July 31 2009 00:46 Daigomi wrote:
It's funny, I've come across this exact test three times in the last two weeks (all in psychology text books).
I'm busy writing a research proposal for my postgrad studies, and my topic is "the use and prevalence of heuristics in day traders," so I've been reading a ton of stuff on stock market psychology, and this specific overconfidence test is included in almost all of the books. I got 1/10 for it, and missed about 4 of them with about 5%, so clearly if I was just a bit less confident, I would have done much better :p
This is an awesome idea for study. I've been doing a lot of casual reading on traders lately and it seems like you could do a lot here that would be interesting.
Taleb's writing is particularly entertaining for how much he hates everyone in his field.
|
South Africa4316 Posts
I can't edit my posts again, damn annoying.
On July 31 2009 02:37 FirstBorn wrote:
I don't really know the answers to any of these question. The closest I got was at the deepest known point in the ocen, but having to transform meters in feet is pretty hard.
Perhaps you should get an easier set of question so you could actually get more feedback from more people. The less the cultural enviroment counts the better.
Culture shouldn't matter. The less you know, the less confident you should be, the larger your ranges should be. Where culture does matter, perhaps, is if people know the exact answer to some of the questions. People who know the exact answer to five of the questions, but then choose way too small ranges for the questions they have to estimate are just as overconfident as a person that chooses way too small ranges for everything because he has to estimate everything, but the test will give them different results.
Like I said though, the questions were chosen so that most people wouldn't know the answers perfectly. It's also meant to show that people tend to be overconfident, rather than measure how overconfident they are. If they wanted to measure actualy confidence levels, they would have to measure the size of your ranges, the size of the ranges you got right, etc.
On July 31 2009 02:46 heyoka wrote:Show nested quote +On July 31 2009 00:46 Daigomi wrote:
It's funny, I've come across this exact test three times in the last two weeks (all in psychology text books).
I'm busy writing a research proposal for my postgrad studies, and my topic is "the use and prevalence of heuristics in day traders," so I've been reading a ton of stuff on stock market psychology, and this specific overconfidence test is included in almost all of the books. I got 1/10 for it, and missed about 4 of them with about 5%, so clearly if I was just a bit less confident, I would have done much better :p This is an awesome idea for study. I've been doing a lot of casual reading on traders lately and it seems like you could do a lot here that would be interesting.
It is quite an interesting field, and there's a fair amount of research on it already. My problem with most of the research though, is that it's done by economists with a basic understanding of psychology, rather than by psychologists with a basic understanding of economy. There's a place for both these groups, but it shouldn't be completely dominated by one group. For example, measuring how people react and interpret specific market situations might be best studied by a mostly-economist, as it would require an in-depth economic understanding of what those specific market situations are, and how they should be reacted to etc. However, studying confidence levels in traders should be studied by a mostly-psychologist, as the confidence test for traders and school-teachers and teenagers are basically the same. So you don't need detailed a detailed economic understanding to do the study, but you might need a detailed psychological understading to interpret the results. Which is why I think there's a gap for psychologists in the field still, and why I'm focusing on it 
|
On July 31 2009 02:11 MoltkeWarding wrote:
It's more a test of memory. Most people should have come across most of these "facts" sometime in their lifetime, and some people are better with recalling numbers than others.
I think you misunderstood the test
Your not supposed to guess the exact numbers but give an upper and lower bound so that you are 90% of covering the exact value between them. I for example know very little about mozart, but I felt 90% sure he was born between 1550 and 1800 which turned out right.
8/10 btw
damn elephants
|
17068 Posts
On July 31 2009 02:41 Zato-1 wrote:Show nested quote +On July 31 2009 02:34 Empyrean wrote:Have you been reading The Black Swarn by Taleb? I've heard a lot about that book, though I haven't read it personally. I'm not interested enough in finance to read it for now, though its conclusions are pretty enlightening.
This exact test is covered in lots of detail in the book. I don't want to ruin it, so I'll spoiler it:
+ Show Spoiler +Basically, Taleb is talking about how in many cases, so called "experts" in a field don't do any better than, say, a cabdriver, but their confidence in their answers and thinking is much higher. If you're interested, Taleb tackles this question in the first chapter of the second part of his book, The Black Swan.
Now, to the OP, if you truly wanted to do something like this, I think you should've asked people about things that pretty much no one knows anything about. I'm sure people remember a ball-park range for things such as the length of the Nile, but if you had, say, a question like "combined ages of ministers of finance in Africa whose name begins with the letter O", you'd be more likely to get better results. People can form educated guesses (ok, number of countries in Africa is ___, so average number of ministers of finance whose name begins with the letter O is ___, and the average age of a minister of finance is ___...), but for some of the things you listed, I actually remembered the exact number from reading about it somewhere random.
|
How can this test confidence when there is no incentive to limit the range of one's guesses? Say I did not know the weight of a Boeing 747, I would be almost certain that one was between 5 000 and 5 000 000 pounds.
|
i really doubt anyone got 10 on that, unless they knew the test questions beforehand.
|
South Africa4316 Posts
On July 31 2009 03:05 MoltkeWarding wrote:
How can this test confidence when there is no incentive to limit the range of one's guesses? Say I did not know the weight of a Boeing 747, I would be almost certain that one was between 5 000 and 5 000 000 pounds.
The point of the exercise is to be 90% certain, not 100% certain. Also, read the other posts on the topic.
|
|
|
|
|
|
EPT
