How do you check your answers?

Suppose I want to do a world online reality check quiz in Quora. How many minimum answers do I need for good statistics relevance?

• Do you ever have an idea but no clue if that's something that could be real or not? Well, I do that often. Sometimes I'll ask my friends. Other times, I'll ask the internet. More often than not, I get bashed, but that's beyond the point. When doing this "reality check" I never really considered statistics relevance, if that's the proper name. I guess when doing statistics a serious scientific company would try to get input on a number of anything from 50 to 1500 people, depending on the case. Well then, I guess my question is: what would be a good and/or a theoretical minimum number of answers to get a representative world reality view here in Quora?

• Answer:

  first of all, we need an independent and random samples that covers the targeted population. But 'independent', I mean we need avoid correlated samples. Quora may be able to cluster user profiles and you may consider randomly select your participants within each cluster. But "random", I mean we need decide who are the participants before hand; we cannot allow people in just because they are willing to take your quiz. We need figure out a way to ensure most people selected will actually take the quiz. A high missing rate could sometime make the results not interpretable. On this aspect, I think angies' list is doing a good job: they kept calling me about a roofer until I started to talk and asked the questions nicely and firmly and without any monetary reward. After establishing these basics, we can talk about sample size calculation. Here we still need subject matter information: what's the question you try to answer and what's the association you try to establish.