2
$\begingroup$

It is understood in mathematical statistics that a sample (as in sampling distribution) may very well contain repeatedly the same item/subject.

In practice though, it would never occur to someone conducting an experiment (e.g. in human sciences studies) to take into account the same subject twice. Samples are made

  • by taking the first people who agree to take the tests and look random enough (e.g. using quota sampling).
  • by taking the only $12$ people living in your country who fit the requirements of the study (e.g. when you're conducting research on a rare disease) (and yes, $12$ is ridiculously small but that's still what is being done when no more subjects are available).

In what situations can this habit turn out to be really bad - induce a huge bias on the results?

What seems clear to me is that if the total population is large enough, then it shouldn't matter too much, since samples with repeats are quite unlikely to occur and don't weigh much in the sampling distribution. When the total population is small though, it feels like we're more likely to miss something.

Improve this question
$\endgroup$
2
  • $\begingroup$ This is called "sampling without replacement", and is commonly discussed in survey sampling and other contexts. The samples are not strictly independent, but they are exchangeable, which implies many of the desirable properties of iid samples (depending on what you're doing with them). $\endgroup$
    – Danica
    Commented Jan 8, 2020 at 17:59
  • $\begingroup$ @Dougal The reason I'm asking is because all hypothesis tests I know (that are used in human sciences) assume replacement. The very first time you see this at work is in the proof that the sample variance is an unbiased estimator: it works with replacement only. $\endgroup$
    – Arnaud Mortier
    Commented Jan 8, 2020 at 22:27

1 Answer 1

Reset to default
0
$\begingroup$

It depends on the subject of the research. For example, treating the same person twice in a medical experiment would be wholly inappropriate if you wanted to know the initial effect. On the other hand if you were running a marketing survey to see what happens during a specific period and quite a few customers were repeat customers, then it would be inappropriate to exclude multiple visits from the same customer

Improve this answer
$\endgroup$
1
  • $\begingroup$ I'm not sure we understand the same thing here: from the mathematical point of view, with replacement here means that if your random generator asks you to pick the same person twice, you're not going to have them actually take the test twice, you're simply going to count the exact same results that they got the first time as if it were a different person with the exact same profile. A ball can't be both black and white, if you select the same ball twice and if it was black, it's still black. What you're talking about would rather fall in the realm of repeated measures. $\endgroup$
    – Arnaud Mortier
    Commented Jan 8, 2020 at 22:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.