Friday, June 3, 2011

Flawed Statistics: Who Wants To Be A Millionaire? Edition

So apparently I learned in economics class today that in the game "Who Wants to be a Millionaire?" the "Phone-a-Friend" option is successful 66% of the time while the "Ask the Audience" option is successful 95% of the time. My economics teacher, whom I presume never watched "Who Wants to be a Millionaire?", wrote this off as an anomaly in the logic of people. Why are these supposed experts so much worse at answering questions compared to a supposedly lower IQ audience? I will cover why this statistic exists and what was flawed in that reasoning.

The anomaly that arises seems to be that the audience is smarter than the people that the contestant chooses. This could be true. There could be a super-genius in the audience that you don't know and your friends may not be as smart as that person. But you are working against large numbers. You should expect an average IQ from the audience, and if you played your cards right, an above average IQ from the phone-a-friend. This still does not explain why the Ask the Audience option has such a high success rate. Well, this anomaly can be easily answered by anyone who watches "Who Wants to be a Millionaire?" It is all about the difficulty of the question.

Any real Millionaire contestant knows that the audience is never as smart as your friend. They also know that the audience may not give one absolute answer. Even on easy questions, there are people in the audience who answer blatantly incorrect answers. Could this be a flaw in the system or a ploy by producers? Maybe, but that is a digression for another day. The audience is stupider, and any contestant would know that. So if you are stuck on the $2000 question, do you go to the audience, which has no set opinion and a lower IQ, or your friend, who will give you one answer and has a higher IQ? Any smart person would use the audience for an easy question. If the question costs such a low amount, you could expect that the question has a more obvious answer. Let the audience support your belief or give you the answer. But when it comes to the $64,000 question, who do you want? Most people would choose their friend. The fact that they are using a lifeline means that they are unsure about the answer. The audience would simply muddy the waters more. A friend, who is smarter and more decisive, would at least give you a hunch. If they are wrong, they are wrong. But they are not wrong because they are stupider. They are wrong because they had a more difficult question.

This leads to another problem. There are different sample sizes for the "Ask the Audience" and "Phone a Friend". If the audience is used more on easier questions, then they will be asked more often than a friend. Just because you use the audience does not mean that you will use the friend. Though it is true vice versa, that is typically not the case. So there are not only easier questions for the audience, but also more questions. This in turn means that there are harder questions and less of them for the friend. The friend could be right 95% of the time given the same sample size and question difficulty, but that is not the case on Millionaire.

Are there cases where the friend is stupider than the audience? Most definitely. Is that always the case according to a quick glance at the statistics? Also yes. But if you look at what is behind the statistics, it is very obvious that the analysis was short-changed. In this case, the observation beats the statistic. People observe that the audience is stupider than the friend, but the statistics disagree. But the story behind the statistics supports the observation. So if you are on Millionaire, don't ask the audience for help on the $1,000,000 question. Ask your friend. Or 50-50. Just never ask the audience, no matter how smart the statistics say they are.

No comments:

Post a Comment