A tutorial says
when the sample size is much smaller than the population, like 10%, we can assume that the element in sample approximately independent.
I can imagine 2 possibilities about the figure 10%, a widely recognized rule or an intuition, which one is true? Could someone please give a hint? Thanks in advance.
The rule you outlined is the rule used for something being approximately independent. This is for when the sample is taken without replacement. Consider that if you sample a population randomly with replacement (so that it's possible you can pick the same person twice) this sample is taken completely randomly and fairly, and each measurement is not dependent on the previous (they are independent). But if you take samples without replacement, each sample effects the next, thus to call a sample 'approximately independent' the change to the probability of selecting for a trait in a population needs to be as minimal as possible. This is done by making sure the percentage of the population represented in your sample is as small as possible. If you pick a red card from a deck of playing cards and don't put it back the probability of picking another red is now 25 in 51. But if you combine three decks of cards and take a red one now, the new probability is 77 out of 155, closer to 50%.
It's a widely recognized rule for sampling without replacement. The intuition behind this rule is that in the 10% case even after drawing an element of the population the probability for the next element does not change so much that we simply neglect it. E.g.:
Let's say we have 100 Persons, then the first we ask has a probability to be drawn of $\frac{1}{100} = 0.01$. Person 5 has a probability of $\frac{1}{96} = 0.01041667$ and Person 10 has a probability of $\frac{1}{91} = 0.01098901$. So, our probabilites are approximately identical.
Like many other rules, the value of 10% is not derived from theory but rather from experience. If you conduct experiments you could also say it has to be 5% of the population size or 7.66% or 11.01%.
$\newcommand{\Z}{\mathbf Z}$$\newcommand{\y}{\mathbf y}$This sounds like this is a question about finite populations. Suppose we have a population $\mathscr U = \{1, \dots, N\}$ and each unit has a value $y_i$ associated with it. If we are taking a sample and using design-based inference we will need to consider $$ \pi_{ij} := P(i\in S, j \in S). $$
If sampling was done independently, which here is with replacement, we'll have $\pi_{ij} = \pi_i \pi_j$. But when it isn't then the discrepancy between $\pi_{ij}$ and $\pi_i\pi_j$ can affect estimators of interest. I'll go through an example with estimating the mean of this finite population that shows where terms like that 10% would appear and what their effect is.
For instance, suppose $S\subset\mathscr U$ is our sample of size $n$ and we are estimating the population mean $\mu$ via $$ \bar y = \frac 1n \sum_{i \in S}y_i = \frac 1n \sum_{i=1}^N Z_iy_i $$ for $Z_i = \mathbf 1_{i \in S}$. To get the expected value of this estimator we don't have to worry about independence of sampling due to the linearity of expectation: $$ \text{E}(\hat y) = \frac 1n \sum_{i=1}^N y_i \pi_i. $$ If this is a simple random sample we'll have $\pi_i = \frac nN$ so $\hat y$ is unbiased, but even if $\hat y$ is biased it isn't due to the lack of independence between the $Z_i$ but rather due to some units having too much or too little probability associated with them (this could be fixed with weighting, like with Horvitz-Thompson estimation).
To get the variance of this estimator I'll use $\y = (y_1,\dots,y_N)^T$ and $\Z = (Z_1,\dots,Z_N)^T$ so $\hat y = \frac 1n \y^T\Z$ and $$ \text{Var}(\hat y) = \frac 1{n^2}\y^T\text{Var}(Z)\y. $$ $$ \text{Var}(Z_i) = \text{E}(Z_i^2)- \text{E}(Z_i)^2 = \pi_i(1-\pi_i) $$ and $$ \text{Cov}(Z_i,Z_j) = \pi_{ij} - \pi_i\pi_j $$ so now the dependence matters.
If the samples are taken independently then $\pi_{ij} - \pi_i\pi_j = 0$ so the covariance matrix of $\Z$ is diagonal and $$ \text{Var}(\hat y) = \frac 1{n^2}\sum_i y_i^2 \pi_i(1-\pi_i). $$ If this was a simple random sample (SRS) with replacement then we'd be in this situation with $\pi_i = \frac nN$ so $$ \text{Var}(\hat y) = \frac 1n\left(1-\frac nN\right)\y^T\y. $$
Suppose now that this is a simple random sample without replacement. That means we'll still have $\pi_i = \frac nN$ but now $$ \pi_{ij} = \frac{n-1}{N-1}\frac nN. $$
Note that the larger $n$ and $N$ are the closer this is to $0$, and as $N\to\infty$ the samples are closer and closer to independence too.
Plugging these values in and rearranging a little, $$ \text{Var}(\Z)_{ij} = \begin{cases} \frac nN\left(1 - \frac nN\right) & i = j \\ -\frac{1}{N-1}\frac nN\left(1 - \frac nN\right) & i\neq j\end{cases}. $$
This means $$ \text{Var}(\hat y) = \frac 1{nN}\left(1 - \frac nN\right) \y^T\left(\frac{N}{N-1}I - \frac 1{N-1}\mathbf 1 \mathbf 1^T\right)\y \\ = \frac 1n\left(1 - \frac nN\right) \cdot \frac 1{N-1}\y^T\left(I - \frac 1{N}\mathbf 1 \mathbf 1^T\right)\y \\ = \frac 1n\left(1 - \frac nN\right) S^2 $$
since $\frac 1{N-1}\y^T\left(I - \frac 1N\mathbf 1\mathbf 1^T\right)\y$ can be recognized as the quadratic form of the sample variance.
One key feature of this is that the smaller $1 - \frac nN$ is, the closer this will be to what we'd have if this was an infinite population. This reflects the independence decreasing as there are more and more units. $1 - \frac nN$ shrinks the variance due to exhausting the population, but the dependence incorporated in the $\pi_{ij}$ further shrinks the variance, which makes sense since the more coupled the observations are the less variability there will be. The term $1 - \frac nN$ is often called the finite population correction (FPC) and this also shows why.
This equips you to answer your question: the 10% is about the FPC so it's getting at this idea, that if we aren't exhausting too much of the population then we'll be closer to independent.
