4
$\begingroup$

Let: population size $=N$; sample size $=n$; sampling interval $=\frac{N}{n} = k$, which can be non-integer; and $r=$ random starting point, which can be non-integer, $0 < r < k$.

http://en.wikipedia.org/wiki/Systematic_sampling says we round up $r + mk$ (where $m$ is an integer between $0$ and $n-1$, both inclusive) although the values given (11, 26, 41, 56, 71, 86, 101, and 116) show some rounded-down values.

ocw.jhsph.edu/courses/statmethodsforsamplesurveys/PDFs/Lecture2.pdf says:

1) "if $k=5$ is considered, stop the selection of samples when $n=175$ achieved."

But this means the last few members would not have any chance of being picked.

2) "if $k=6$ is considered, treat the sampling frame as a circular list and continue the selection of samples from the beginning of the list after exhausting the list during the first cycle."

This doesn't give equal chance to each member of being picked, does it?

3) "An alternative procedure is to keep $k$ non-integer and continue the sample selection as follows: Let us consider, $k=5.71$, and $r=4$. So, the first sample is 4th in the list. The second $=(4+5.71) =9.71$ ~ 9th in the list, the third $=(4+2\times5.71) =15.42$ ~ 15th in the list, and so on. (The last sample is: $4+5.71\times(175-1) = 997.54$ ~ 997th in the list)."

This uses rounding down of $r + mk$ (different from the rounding up suggested by the Wikipedia page mentioned above).

Shouldn't we be rounding off instead to give equal chance to each member of being picked?

An even better way is to let random starting point be $R$, randomly selected from the integers 1 to $N$, both inclusive, and use $r + mk$, rounded off and modulo $N$?

$\endgroup$
9
  • 1
    $\begingroup$ Please tell us about your population: what kind of thing are you sampling? This is crucial because the entire point of systematic sampling is to cope with expected associations among nearby subjects in the sampling frame. $\endgroup$
    – whuber
    Commented Oct 25, 2013 at 16:09
  • $\begingroup$ Generally, systematic sampling is a somewhat lousy method as it does not lend itself naturally to unbiased variance estimation. These days, you can devise high-entropy sampling schemes using cube sampling and other modern methods. Systematic sampling is only reasonable when the frame that you have is a sheet of paper with the units on it, and you need to draw your sample before this sheet turns into a pumpkin at midnight; any other uses are deprecated by now. Also, @Neo, please provide LaTeX formatting to make your text better readable. $\endgroup$
    – StasK
    Commented Oct 25, 2013 at 16:38
  • $\begingroup$ Hi, @Whuber, I am thinking about it in the general sense -- no specific population in mind. $\endgroup$
    – Neo
    Commented Oct 27, 2013 at 5:48
  • $\begingroup$ Hi, @StasK, Okay. Would learn LaTeX formatting as soon as I can. $\endgroup$
    – Neo
    Commented Oct 27, 2013 at 5:53
  • $\begingroup$ I cannot interpret this issue in a "general sense" because the concept of systematic sampling implies a particular structure to the population. Without information about that structure, the questions asked here cannot be answered. @StasK I do not understand what you mean: for many purposes of sampling the environment, systematic sampling definitely has not been "deprecated." Could you please qualify your comment by stating for which applications systematic sampling might not be good and who has determined that? $\endgroup$
    – whuber
    Commented Oct 27, 2013 at 14:56

1 Answer 1

6
$\begingroup$

You are correct: there are serious problems with the sampling procedures described in that reference. Its description of how systematic sampling should be carried out is plainly wrong, but the problems can be fixed.

We are talking about systematic sampling of a population of $N$ sampling units. They have been indexed with $1, 2, \ldots, N$ beforehand. Systematic sampling selects a desired number $n$ of individuals in the population by taking them at relatively regular intervals within this set of indexes.

The three methods in the reference begin by choosing the nominal sampling interval $k$ either to be $N/n$ (potentially non-integral), $\lceil N/n \rceil$ (rounding up), or $\lfloor N/n \rfloor$ (rounding down). Starting at a random integer $r$ chosen uniformly from the set $\{1, 2, \ldots, \lfloor k \rfloor\}$ they select the units at indexes $r, [r + k], [r + 2k], \ldots, [r + (n-1)k],$ where "$[x]$" means to round $x$ to an integer according to some definite rule if $x$ is not integral. These indexes are taken modulo $N$ if necessary, circling back to the beginning if the sequence goes beyond $N$.

To see more clearly what can happen, let's consider a small population of, say, $N=7$ from which we wish to obtain a sample of size $n=3.$

  1. Using $k = N/n = 7/3$, the possible starting indexes are $r=1$ and $r=2$. In the first case the selected indexes would be $1, [1+7/3], [1+2*7/3]$ = $1, 3, 6$ and in the second case the indexes would be $2, [2+7/3], [2+2*7/3]$ = $2, 4, 7$. We can picture these two samples--both equally likely to be chosen--by drawing marks below the selected indexes thus:

    Sample   1 2 3 4 5 6 7
       r=1   *   *     *
       r=2     *   *     *
    
  2. Using $k = \lceil 7/3 \rceil = 3$ gives three possible samples, each with a chance of $1/3$ of being chosen:

    Sample   1 2 3 4 5 6 7 
       r=1   *     *     *
       r=2   * *     *   
       r=3     * *     *
    

    Notice how in the second and third samples indexes of $8$ and $9$ were calculated and wrapped around modulo $7$ to the beginning, turning into $1$ and $2$, respectively.

  3. Using $k = \lfloor 7/3 \rfloor = 2$ gives two possible samples, each with a chance of $1/2$ of being chosen:

    Sample   1 2 3 4 5 6 7 
       r=1   *   *   *
       r=2     *   *   *
    

The problems pointed out in the question are apparent:

  • In case 1, index $5$ is in neither possible sample and therefore is never selected.

  • In case 2, indexes $1$ and $2$ each have a $2/3$ chance of being in the sample (because they each appear in two out of the three equally likely samples) while the other indexes have only a $1/3$ chance of being in the sample. Thus this sampling method is biased towards the first two subjects in the list.

  • In case 3, index $7$ is never selected.

In general, for larger $N$ a certain fraction of the population will either never be selected (as in cases 1 and 3) or the initial subjects in the list will have relatively high chances of being selected (case 2). In case 3, the final subjects in the list will never be selected, while in case 1 the non-selected subjects are situated fairly evenly throughout the list.

Normally, systematic sampling is used because there is a natural ordering of the subjects that is related to how they can be sampled: their indexes correspond to their time of enrollment in a human trial or are positions of possible samples in a physical medium or process (like soils in a streambed or products coming off an assembly line). Therefore the problems in cases 1 and 2 are serious ones, because they systematically omit or over-weight one end of the population, whose properties easily could differ from those of the rest of the population. Any study using these methods would thereby be seriously flawed.

There are two solutions. The simplest one is straightforward and is the correct way to conduct such sampling: choose $r$ uniformly at random from the entire list of indexes, not just the first $k$ of them. For instance, here are all the equally-likely samples that can be obtained using the first (round-as-you-go) method. The first two already appeared in (1) above:

Sample   1 2 3 4 5 6 7
   r=1   *   *     *
   r=2     *   *     *
   r=3   *   *   *  
   r=4     *   *   *
   r=5       *   *   *
   r=6   *     *   *
   r=7     *     *   *

By looking down the columns it is easy to see that each index has $3/7$ chance of appearing in the sample: there is no longer any bias.

It is worth reflecting on how such a subtle change--choosing the initial index randomly from $1, \ldots, N$ rather than $1, \ldots, k$--can have a profound effect on the quality of the study.

The other solution is to compensate for the sampling bias using the Horvitz-Thompson estimator. This adjusts for varying probabilities of inclusion in the sample. In method (1) it would effectively recognize that the samples are only from the subpopulation indexed by $1,2,3,4,6,7$ (skipping $5$, which is never included). This wouldn't really fix much, but it would highlight the shortcomings of the results and perhaps prevent invalid inferences.

In method (2) the H-T estimator would give any observations from indexes $1$ and $2$ a weight of just one-half the weights of the other indexes. For example, if the sample happened to be $r=3$ then subjects $2, 3,$ and $6$ would be drawn. Let the values of some quantity observed for these subjects be written $x_2, x_3,$ and $x_6,$ respectively. Because the chance of including $2$ in the sample a priori was $2/3$ and the chances of including $3$ or $6$ were only $1/3,$ the Horvitz-Thompson estimator of the population mean would be

$$\frac{\frac{1}{2/3} x_2 + \frac{1}{1/3} x_3 + \frac{1}{1/3} x_6}{\frac{1}{2/3} + \frac{1}{1/3} + \frac{1}{1/3}} =\frac{\frac{1}{2} x_2 + x_3 + x_6}{\frac{1}{2} + 1 + 1} = \frac{1}{5}x_2 + \frac{2}{5} x_3 + \frac{2}{5} x_6.$$

There are comparable formulas for the sampling variance of this estimate (used to construct confidence intervals and test hypotheses, for instance).

The Horvitz-Thompson estimator comes to the fore when it is impossible to avoid over- and under-weighting some subjects in a sample. It can also be applied to rescue a study that was otherwise ruined using the procedures in the reference: after the fact, if we can compute the actual probability of each subject's inclusion in the sample, then we can apply these probabilities to obtain valid estimates and valid hypothesis tests.

$\endgroup$
2
  • $\begingroup$ the other methods which round the interval before sampling also end up giving each element 3/7 chances of being picked up as long as we pick the initial index from 1,.., N. are they just as correct as the round-as-you-go method? $\endgroup$ Commented Apr 10, 2019 at 21:00
  • $\begingroup$ @Capi Provided the initial index is uniformly sampled, as illustrated in the middle of my answer, this will guarantee every element of the population has an equal chance of being selected. Indeed, you could fix, once and for all, an $n$-element subset $X=\{j_1,j_2,\ldots,j_n\}$ of the population indexes $\{1,2,\ldots, N\}.$ Pick a starting index $i$ in the population uniformly at random, as you state. Let the sample be $X+i-1=\{i+j_1-1,i+j_2-1,\ldots,i+j_n-1\}$ (computed modulo $N,$ so that $N+1=1,N+2=2,$ etc.). All elements still have equal chances of being in the sample. $\endgroup$
    – whuber
    Commented Apr 11, 2019 at 13:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.