7
$\begingroup$

In our project we have a population of 1000+ individuals. We picked a random sample of 107 individuals, but then we realized we needed more precision, so now we want to have a larger sample.

The problem is that sampling is expensive in our case, so we were thinking about picking another random sample of about 50 individuals from the remaining 900+ individuals, and adding it to the first sample in order to get a bigger sample without starting all over.

Would this new sample still be considered random?

Edit: We are taking this sample in order to estimate the distribution of the entire population. With the current sample, however, we are getting too much "holes" in our distribution, which leads to seemingly imprecise results.

$\endgroup$
3
  • 3
    $\begingroup$ you should be more specific about what made you realize that you needed more precision. Did you look at your data or compute any statistics to realize this ? You should also mention if you want to test an hypothesis or just estimate a statistic. $\endgroup$
    – brumar
    Jun 10, 2015 at 14:38
  • 2
    $\begingroup$ I think brumar's comment is critical; if the considerations to sample again were not based on anything in the sample itself, then sampling without replacement from the remaining population should reproduce sampling without replacement from the original (imagine dealing cards off the top of a shuffled deck with 1000 cards in it; you dealt out 107 and pause and then you deal another 50); that's the same as dealing 157 in one go. But if you look at some of the cards you dealt before you decide whether to deal another 50, then it's not random sampling. $\endgroup$
    – Glen_b
    Jun 25, 2015 at 3:53
  • $\begingroup$ You are right @Glen_b. I'm embarassed to say that that I came to the same conclusion 10 years ago when a survey organization enlarged a sample, then transmitted the first and second samples with different weights. $\endgroup$ Jun 26, 2015 at 1:22

3 Answers 3

4
$\begingroup$

Correction

I withdraw the previous answer (below) as wrong. @Glen_b is correct, if the second sample is not based on any results in the first, then the overall probability of selection is $\dfrac{n}{N}$, the same as for a SRS (proof below). Further, you can treat the result as a SRS although the size was not fixed in advance.

However, you stated that you took the second sample because you required more precision. If you made the decision solely because standard errors for estimates were larger than expected, then bias in the final estimated standard errors will be small (Hansen, Hurwitz, and Madow (1953, pp.77-80) (provided that the initial SE themselves were precisely estimated). In that case, I would go ahead and treat the observations as having come from a SRS. If, however, you took the second sample to reduce a borderline p-value to a smaller value, then serious bias is possible.

Proof that probability of selection is n/N:

Let $p_1 =\dfrac{n_1}{N}$ be the probability of selection in the first sample, and let $p_2$ be the conditional probability of selection in the second sample: $p_2 =\dfrac{n_2}{N-n_1}$. Let $n = n_1 + n_2$ be the final sample size. Then the probability of selection of an observation to the final sample is:

\begin{align} p & = p_1 + (1-p_1)\thinspace p_2 \\ & = \frac{n_1}{N} + (1-\frac{n_1}{N}) \frac{n_2}{N-n_1} \\ & = \frac{n_1}{N} + \frac{N-n_1}{N}\frac{n_2}{N-n_1} \\ & = \frac{n_1 + n_2}{N} \\ & = \frac{n}{N} \end{align}

Reference: Hansen, MH, WN Hurwitz, and W Madow. 1953. Sample Survey Methods and Theory. Volume I Methods and Applications. New York: Wiley.

Original Answer:

This kind of problem frequently arises in practice and the solution is similar.

As @David Z implies, you must investigate the possibility of systematic differences between the first and second surveys and samples.

The resulting combined sample can indeed be considered random, but not simple random. The probabilities of selection differ between the two samples, so the analysis will have to be weighted. You compute weights as follows:

Let the number in the population be $N$. Then the probability of selection for the original sample is:

$$ f_1 = \frac{107}{N} $$

To be selected in the second sample, one must first not be selected in the first sample; then be selected in the second:

$$ f_2 = (1 - f_1) \frac{50}{N - 107} $$

The sample weights will be $W = \dfrac{1}{f_1}$ in sample 1 and $W = \dfrac{1}{f_2}$ in sample 2.

To illustrate, suppose $N = 1020$, then the weights in the first and second sample will be $9.5327$ and $20.4$, respectively

These calculations will require modification if there was non-response in either sample.

$\endgroup$
0
$\begingroup$

It depends on whether an assumption about same distribution between the remaining 900+ after the first sampling process and original 1000+ can be held. If possible you can perform an test about mean or heteroscedasticity of the parameter(s) of interested.

$\endgroup$
0
$\begingroup$

I am not quite sure how random this sample could be considered. I don't have a great experience of statistics but the method you suggest doesn't sound very good to me. What I would do is put back the sample I took from the entire population, reorder them and take again a random sample from it. Or I would just take a new larger sample from the population and compare my results with the sample from the first try.

It might be a good idea to describe the limitations you have, as in why sampling is expensive for you, so other people can give a more elaborate answer.

$\endgroup$
1
  • $\begingroup$ The problem with forgetting about the first sample, and then resampling a larger sample is that people might get annoyed when you try to interview them a second time ... $\endgroup$ Jun 25, 2015 at 11:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.