4
$\begingroup$

I have a specific question about random selection, representativeness and inference. It is well known that it's necessary to use random selection to get representative samples from the population of interest. But what happens with non-random samples?

I am working with an intentional sample. I compared the means of some of the main variables of my database with official data. And I can conclude that there aren't important differences between them. So I have a quite representative sample. However, I am confused about how to deal with this sample. When I estimate the means of some variables or the correlations between them, do I need to compute CI or p-values for this estimations?

I guess that it's not necessary, since I got the sample without random selection. Therefore, I have no sampling error, and I can't know how my estimations differ from the population. However, I have read some papers where the authors work with non-random samples and they make estimations (they use CI's and p-values). Moreover, it's difficult to use multivariate techniques (like ANOVA or regression) without the help of statistical significance.

Can anybody help me? I am very confused with this matter.

$\endgroup$
  • $\begingroup$ It isn't quite true that samples have to be random. Rather, they have to be representative of the population you want to draw an inference about. Having a random sample of the population in question implies representativeness, but it is possible for a sample to be representative even if it is not random (of course you have no guarantee & it would be hard to substantiate, but it is possible). For more on this, it may help to read my answer here: identifying-the-population-and-samples-in-a-study. $\endgroup$ – gung - Reinstate Monica Dec 7 '13 at 18:11
4
$\begingroup$

You certainly do have sampling error, even if you don't know what population your sample came from.

As with random samples, if all you want to do is make statements about the sample itself, then you do not need p values or any form of inferential statistics. Indeed, you don't need any specific sample size either. I can measure myself and my wife and say "I am taller than she is" (N of 2). I can just measure myself and say "I am 5 foot 8" (N = 1).

However, even with non-random samples you are usually interested in inference. You therefore have to assume either a) that the non-randomness in your sample isn't affecting things (a dangerous assumption!) or b) That there is some population from which your sample is random, and that that population is interesting.

In real life, this often gets blurred. In the many cases where there is no way to take a random sample (too expensive, too impractical, unethical, illegal, impossible) people often write as if they are inferring to something sort of in between a and b.

$\endgroup$
  • $\begingroup$ Ok, I have sample error. But, as I got a non-random sample, I supose that I can't control this error, since no all individuals of the population have had the same probability to be included in the sample. Therefore, I can't compute the margin of error... Then, I can't understand how to make inference form non-random samples. Could you explain your options in more details? I see the same problem if I choose either a) or b) option. Thank you for the help. $\endgroup$ – Jos Oct 6 '13 at 14:49
  • $\begingroup$ With b) you are being more explicit about what is wrong with your sample $\endgroup$ – Peter Flom - Reinstate Monica Oct 6 '13 at 22:17
0
$\begingroup$

You cannot make statistical inferences (p values, CIs) from non-random samples. Oh, you can compute them, but they will be essentially uninterpretable as they are based on the assumption that your data are from a random sample (or as with a trial, there was random assignment--though this inference is derived, at root, from randomization tests). You can attempt to make a logical case that the results in your sample are similar in all relevant ways to a unmeasured population of interest. The use of p values with non-random samples in research drives me crazy, because it conveys a meaning that is not valid--it is a kind of fraud.

$\endgroup$
  • $\begingroup$ This is too strong a claim. In addition, note that essentially no scientific studies actually use random samples. The closest is polling studies (who are you going to vote for, etc.), & even those aren't really random. $\endgroup$ – gung - Reinstate Monica Dec 7 '13 at 18:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.