2
$\begingroup$

I recently studied sampling from one of online courses and got to know about various sampling methods but i'm confused regarding the probability of an element being selected in a sample. In that course the lecturer just wrote down that the probability of an element being selected in a sample of size $n$ from the population of size $N$ for both with replacement(order matter) and without replacement(order does not matter) is $n/N$. I'm not able to derive it and also not able to get the results for the remaining cases. Assume that the set A = {$a_1,a_2,a_3,....,a_N$} be the population set and you sample $n$ elements from this set using one of the below:

  1. Simple Random Sampling with Replacement(order matters)
  2. Simple Random Sampling with Replacement(order does not matter)
  3. Simple Random Sampling without Replacement(order matters)
  4. Simple Random Sampling without Replacement(order does not matter)

So what is the probability that an element, say $a_i$ is one of the $n$ elements being sampled using above methods.

Edit: (what i mean by order matters)

take population set to be A = {1,2,3} now select a sample of size 2 these can be {1,1}{1,2}{1,3}{2,1}{2,2}{2,3}{3,1}{3,2}{3,3}(depending on the method used) now in the case of order matters {1,2} and {2,1} are counted twice but in case where order does not matter these are counted only once so the samples become {1,1}{1,2}{1,3}{2,2}{2,3}{3,3}

Edit 2: Further clarification

Suppose set A={1,2,3} and then simple random sampling is performed selecting 2 elements using one of the above methods, i want to know the probability of selection of each element. In short what is probability of selection of "1" when sampling is performed. For case (4) it is easy to derive:

number of samples = 3 ({1,2},{1,3},{2,3})
number of samples in which "1" is present = 2 ({1,2},{1,3})
so the probability of selection = 2/3 = n/N (n = 2, N = 3)

but how to derive the probability of including a specified unit in a sample(in general) for this case and also for other cases.

$\endgroup$
5
  • 1
    $\begingroup$ What do you mean by "order matters"? $\endgroup$ Sep 17, 2020 at 8:34
  • $\begingroup$ see edited question. $\endgroup$ Sep 17, 2020 at 8:41
  • $\begingroup$ FWIW, there are far more sampling methods than these, contra the suggestion conveyed by calling them "the" four types. $\endgroup$
    – whuber
    Sep 17, 2020 at 14:38
  • $\begingroup$ Re the edit: to say "order matters" means you want to know more than the mere frequencies (or probabilities) of inclusion in the sample. Otherwise, (2) and (1) are identical and (3) and (4) are identical, reducing this from four to two questions. The answer to (1)/(2) is immediate because it is given by the definition of SRSWR. Thus, you ought to edit your question down to (4) alone. $\endgroup$
    – whuber
    Sep 18, 2020 at 12:06
  • $\begingroup$ I didn't get what you are saying I just wanted to know the probability that an element get selected (not the whole sample get selected, which is easy to calculate) when you perform sampling (and also this is not the same as to calculate the probability of the element being selected "at any stage" which is mostly given in the definition of SRSWR.) So in short i want to know the probability that an element get selected ("no matter when it get selected"). $\endgroup$ Sep 19, 2020 at 0:24

1 Answer 1

3
$\begingroup$

Suppose the population is $a_{1}, a_{2}, \ldots, a_{N}$. We choose a sample of size $n$ from the population using simple random sampling and want to know the probability that a specific element, say $a_{1}$, is in the sample.

First, note that the probability of an element being included in the sample is the same whether the sample is ordered or unordered. (One way to see that is to note that we can change ordered samples into unordered samples by ignoring the ordering. If we know the probability $a_{1}$ is in any of the ordered samples, that must also be the probability that $a_{1}$ is in any of the unordered samples.) So we can find just one probability and get the result for both ordered and unordered cases.

Sampling without replacement (unordered)

There are ${N}\choose{n}$ possible ways to choose an unordered sample of size $n$ from a population of size $N$ without replacement. Crucially, we also know that each one of those options are equally likely. How many ways are there to choose a sample containing $a_{1}$? If we must include $a_{1}$ then we have to choose the remaining $n-1$ elements from the remaining $N-1$ possibilities, which gives ${N-1}\choose{n-1}$ possible samples. Because we know that each possibility is equally likely, we can find the probability of choosing $a_{1}$ by taking the ratio

$$\frac{{N-1}\choose{n-1}}{{N}\choose{n}} =\frac{n}{N}$$

Sampling with replacement (ordered)

There are $N^n$ possible samples and all of them are equally likely. Finding the number of possible samples that contain $a_{1}$ is a bit more tricky, but can be done by finding the complement instead. If we find the number of samples that don't contain $a_{1}$ and subtract that from $N^n$, we will have the number of samples that do contain $a_{1}$. That can be done since the number of samples that don't contain $a_{1}$ is $(N-1)^n$, giving the number of samples that do contain $a_{1}$ as $N^{n} - (N-1)^n$. This gives the probability of $a_{1}$ being in the sample as

$$\frac{N^{n} - (N-1)^n}{N^{n}} = 1-\left(\frac{N-1}{N}\right)^n$$

By symmetry these probabilities are the same for every $a_{i}$.

$\endgroup$
3
  • $\begingroup$ could you please elaborate on note that the probability of an element being included in the sample is the same whether the sample is ordered or unordered. $\endgroup$ Sep 20, 2020 at 3:16
  • 1
    $\begingroup$ @dark_prince I added a bit of explanation, but I'm not sure it's enough. It feels obvious to me the re-arranging the sample after you've taken it won't change the probability of those elements appearing in the sample. I can't think of a simpler way to explain that other than to completely write out a simple case with N=3 to show it explicitly. $\endgroup$ Sep 20, 2020 at 14:33
  • 1
    $\begingroup$ The obvious can be made rigorous by computing the probabilities in both cases. The result comes down to the simple algebraic identity $$\begin{aligned}&\frac{\binom{n-1}{k-1}\binom{1}{1}}{\binom{n}{k}}=\frac{k}{n}\\&=\frac{1}{n}+\left(1-\frac{1}{n}\right)\frac{1}{n-1}+\cdots+\left(1-\frac{1}{n}\right)\left(1-\frac{1}{n-1}\right)\cdots\left(1-\frac{1}{n-k+2}\right)\frac{1}{n-k+1}.\end{aligned}$$ The left hand side counts unordered samples that include a specified value while the right hand side counts ordered samples that include that value. $\endgroup$
    – whuber
    Mar 3, 2022 at 17:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.