Here is an alternative setup within the framework of the superpopulation model of sampling theory. It differs in notation and conception to classical sampling theory, but I think it is quite simple and intuitive.

Let $X_1,X_2,X_3,...$ be an exchangeable "superpopulation" of values. Take the first $N$ values to be the finite population of interest and the first $n \leqslant N$ values as a sample from this population. (The exchangeability of the superpopulation means that the sample is a simple-random-sample from the population.) Now, consider the mean-difference $\bar{X}_n - \bar{X}_N$ measuring the difference between the sample mean and population mean. This quantity can be written in the form:

$$\begin{align} \bar{X}_n - \bar{X}_N &= \frac{1}{n} \sum_{i=1}^n X_i - \frac{1}{N} \sum_{i=1}^N X_i \\[6pt] &= \Big( \frac{1}{n} - \frac{1}{N} \Big) \sum_{i=1}^n X_i - \frac{1}{N} \sum_{i=n+1}^N X_i \\[6pt] &= \frac{N-n}{nN} \sum_{i=1}^n X_i - \frac{1}{N} \sum_{i=n+1}^N X_i \\[6pt] &= \frac{1}{n} \Bigg[ \frac{N-n}{N} \sum_{i=1}^n X_i - \frac{n}{N} \sum_{i=n+1}^N X_i \Bigg]. \\[6pt] \end{align}$$

We clearly have $\mathbb{E}(\bar{X}_n - \bar{X}_N) = 0$, so we can use the sample mean as an unbiased estimator for the population mean. If we denote the variance of the superpopulation by $\sigma^2$ then our quantity has variance:

$$\begin{align} \mathbb{V}(\bar{X}_n - \bar{X}_N) &= \frac{1}{n^2} \Bigg[ \Big( \frac{N-n}{N} \Big)^2 \sum_{i=1}^n \mathbb{V}(X_i) + \Big( \frac{n}{N} \Big)^2 \sum_{i=n+1}^N \mathbb{V}(X_i) \Bigg] \\[6pt] &= \frac{1}{n^2} \Bigg[ \Big( \frac{N-n}{N} \Big)^2 n \sigma^2 + \Big( \frac{n}{N} \Big)^2 (N-n) \sigma^2 \Bigg] \\[6pt] &= \frac{1}{n^2 N^2} \Bigg[ (N-n)^2 n \sigma^2 + n^2 (N-n) \sigma^2 \Bigg] \\[6pt] &= \frac{1}{n^2 N^2} \cdot (N-n) N n \sigma^2 \\[6pt] &= \frac{N-n}{N} \cdot \frac{\sigma^2}{n}. \\[6pt] \end{align}$$

Suppose we let $S_N^2$ and $S_{N*}^2$ denote the variance values for the population, where the first uses Bessel's correction and the second does not (so we have $S_N^2 = \frac{N}{N-1} S_{N*}^2$). In classical sampling theory the latter quantity is considered to be "the variance" of the population. (Formally it is the variance of the empirical distribution of the population.) However, the first of these quantities is an unbiased estimator for the superpopulation variance, so we can estimate the variance of our mean-difference quantity by:

$$\begin{align} \hat{\mathbb{V}}(\bar{X}_n - \bar{X}_N) &= \frac{N-n}{N} \cdot \frac{S_N^2}{n}. \\[6pt] \end{align}$$

Consequently, using the central limit theorem we can establish the following confidence interval for the population mean $\bar{X}_N$:

$$\text{CI}_N(1-\alpha) = \Bigg[ \bar{X}_n \pm \sqrt{\frac{N-n}{N}} \cdot \frac{t_{n-1, \alpha/2}}{\sqrt{n}} \cdot S_{N} \Bigg].$$

This is the form of the confidence interval that I find the most natural. However, with this form, you will notice that we use a finite population correction term that is different to your expression. The expression you are using occurs when we convert to the variance estimator $S_{N*}^2$ that does not use Bessel's correction (purportedly "the variance" of the population). In this case we have the equivalent expression:

$$\text{CI}_N(1-\alpha) = \Bigg[ \bar{X}_n \pm \sqrt{\frac{N-n}{N-1}} \cdot \frac{t_{n-1, \alpha/2}}{\sqrt{n}} \cdot S_{N*} \Bigg].$$

As you can see, framed in this latter form, the finite population correction term is the one in your question. You can see that the finite population correction term appears in the confidence interval formula in order to "correct" for the finite population. Taking $N \rightarrow \infty$ (so that the population of interest is the superpopulation) we get $FPC \rightarrow 1$, yielding the standard confidence interval for the mean parameter of a "large" population.

I hope this alternative presentation of the matter elucidates the correction term within the broader framework of the superpopulation model. I have always preferred this model of sampling theory, since it makes it simpler to distinguish between the finite population case and the infinite population case. As you can see, within this framework the correction term pops out fairly simply in the course of attempting to estimate the mean of the finite population.

Here is an alternative setup within the framework of the superpopulation model of sampling theory. It differs in notation and conception to classical sampling theory, but I think it is quite simple and intuitive.

Let $X_1,X_2,X_3,...$ be an exchangeable "superpopulation" of values. Take the first $N$ values to be the finite population of interest and the first $n \leqslant N$ values as a sample from this population. (The exchangeability of the superpopulation means that the sample is a simple-random-sample from the population.) Now, consider the mean-difference $\bar{X}_n - \bar{X}_N$ measuring the difference between the sample mean and population mean. This quantity can be written in the form:

$$\begin{align} \bar{X}_n - \bar{X}_N &= \frac{1}{n} \sum_{i=1}^n X_i - \frac{1}{N} \sum_{i=1}^N X_i \\[6pt] &= \Big( \frac{1}{n} - \frac{1}{N} \Big) \sum_{i=1}^n X_i - \frac{1}{N} \sum_{i=n+1}^N X_i \\[6pt] &= \frac{N-n}{nN} \sum_{i=1}^n X_i - \frac{1}{N} \sum_{i=n+1}^N X_i \\[6pt] &= \frac{1}{n} \Bigg[ \frac{N-n}{N} \sum_{i=1}^n X_i - \frac{n}{N} \sum_{i=n+1}^N X_i \Bigg]. \\[6pt] \end{align}$$

We clearly have $\mathbb{E}(\bar{X}_n - \bar{X}_N) = 0$, so we can use the sample mean as an unbiased estimator for the population mean. If we denote the variance of the superpopulation by $\sigma^2$ then our quantity has variance:

$$\begin{align} \mathbb{V}(\bar{X}_n - \bar{X}_N) &= \frac{1}{n^2} \Bigg[ \Big( \frac{N-n}{N} \Big)^2 \sum_{i=1}^n \mathbb{V}(X_i) + \Big( \frac{n}{N} \Big)^2 \sum_{i=n+1}^N \mathbb{V}(X_i) \Bigg] \\[6pt] &= \frac{1}{n^2} \Bigg[ \Big( \frac{N-n}{N} \Big)^2 n \sigma^2 + \Big( \frac{n}{N} \Big)^2 (N-n) \sigma^2 \Bigg] \\[6pt] &= \frac{1}{n^2 N^2} \Bigg[ (N-n)^2 n \sigma^2 + n^2 (N-n) \sigma^2 \Bigg] \\[6pt] &= \frac{1}{n^2 N^2} \cdot (N-n) N n \sigma^2 \\[6pt] &= \frac{N-n}{N} \cdot \frac{\sigma^2}{n}. \\[6pt] \end{align}$$

Suppose we let $S_N^2$ and $S_{N*}^2$ denote the variance values for the population, where the first uses Bessel's correction and the second does not (so we have $S_N^2 = \frac{N}{N-1} S_{N*}^2$). In classical sampling theory the latter quantity is considered to be "the variance" of the population. (Formally it is the variance of the empirical distribution of the population.) However, the first of these quantities is an unbiased estimator for the superpopulation variance, so we can estimate the variance of our mean-difference quantity by:

$$\begin{align} \hat{\mathbb{V}}(\bar{X}_n - \bar{X}_N) &= \frac{N-n}{N} \cdot \frac{S_N^2}{n}. \\[6pt] \end{align}$$

Consequently, using the central limit theorem we can establish the following confidence interval for the population mean $\bar{X}_N$:

$$\text{CI}_N(1-\alpha) = \Bigg[ \bar{X}_n \pm \sqrt{\frac{N-n}{N}} \cdot \frac{t_{n-1, \alpha/2}}{\sqrt{n}} \cdot S_{N} \Bigg].$$

This is the form of the confidence interval that I find the most natural. However, with this form, you will notice that we use a finite population correction term that is different to your expression. The expression you are using occurs when we convert to the variance estimator $S_{N*}^2$ that does not use Bessel's correction (purportedly "the variance" of the population). In this case we have the equivalent expression:

$$\text{CI}_N(1-\alpha) = \Bigg[ \bar{X}_n \pm \sqrt{\frac{N-n}{N-1}} \cdot \frac{t_{n-1, \alpha/2}}{\sqrt{n}} \cdot S_{N*} \Bigg].$$

As you can see, framed in this latter form, the finite population correction term is the one in your question. You can see that the finite population correction term appears in the confidence interval formula in order to "correct" for the finite population. Taking $N \rightarrow \infty$ (so that the population of interest is the superpopulation) we get $FPC \rightarrow 1$, yielding the standard confidence interval for the mean parameter of a "large" population.

Now, as to the "5% rule", that is an arbitrary rule, and I don't recommend it. In my view it is best to always include the FPC when you have a finite population. If the sample proportion is small then the FPC is close to one, so it does not change the interval much, but I find it silly to remove it. Practitioners who offer these rules-of-thumb evidently think that with an FPC close to one they should remove the term, but I see no sense in that; it is an approximation for approximation's sake.

I hope this alternative presentation of the matter elucidates the correction term within the broader framework of the superpopulation model. I have always preferred this model of sampling theory, since it makes it simpler to distinguish between the finite population case and the infinite population case. As you can see, within this framework the correction term pops out fairly simply in the course of attempting to estimate the mean of the finite population.

Here is an alternative setup within the framework of the superpopulation model of sampling theory. It differs in notation and conception to classical sampling theory, but I think is quite simple and intuitive.

Let $X_1,X_2,X_3,...$ be an exchangeable "superpopulation" of values. Take the first $N$ values to be the finite population of interest and the first $n \leqslant N$ values as a sample from this population. (The exchangeability of the superpopulation means that the sample is a simple-random-sample from the population.) Now, consider the mean-difference $\bar{X}_n - \bar{X}_N$ measuring the difference between the sample mean and population mean. This quantity can be written in the form:

$$\begin{align} \bar{X}_n - \bar{X}_N &= \frac{1}{n} \sum_{i=1}^n X_i - \frac{1}{N} \sum_{i=1}^N X_i \\[6pt] &= \Big( \frac{1}{n} - \frac{1}{N} \Big) \sum_{i=1}^n X_i - \frac{1}{N} \sum_{i=n+1}^N X_i \\[6pt] &= \frac{N-n}{nN} \sum_{i=1}^n X_i - \frac{1}{N} \sum_{i=n+1}^N X_i \\[6pt] &= \frac{1}{n} \Bigg[ \frac{N-n}{N} \sum_{i=1}^n X_i - \frac{n}{N} \sum_{i=n+1}^N X_i \Bigg]. \\[6pt] \end{align}$$

We clearly have $\mathbb{E}(\bar{X}_n - \bar{X}_N) = 0$, so we can use the sample mean as an unbiased estimator for the population mean. If we denote the variance of the superpopulation by $\sigma^2$ then our quantity has variance:

$$\begin{align} \mathbb{V}(\bar{X}_n - \bar{X}_N) &= \frac{1}{n^2} \Bigg[ \Big( \frac{N-n}{N} \Big)^2 \sum_{i=1}^n \mathbb{V}(X_i) + \Big( \frac{n}{N} \Big)^2 \sum_{i=n+1}^N \mathbb{V}(X_i) \Bigg] \\[6pt] &= \frac{1}{n^2} \Bigg[ \Big( \frac{N-n}{N} \Big)^2 n \sigma^2 + \Big( \frac{n}{N} \Big)^2 (N-n) \sigma^2 \Bigg] \\[6pt] &= \frac{1}{n^2 N^2} \Bigg[ (N-n)^2 n \sigma^2 + n^2 (N-n) \sigma^2 \Bigg] \\[6pt] &= \frac{1}{n^2 N^2} \cdot (N-n) N n \sigma^2 \\[6pt] &= \frac{N-n}{N} \cdot \frac{\sigma^2}{n}. \\[6pt] \end{align}$$

Suppose we let $S_N^2$ and $S_{N*}^2$ denote the variance values for the population, where the first uses Bessel's correction and the second does not (so we have $S_N^2 = \frac{N}{N-1} S_{N*}^2$). In classical sampling theory the latter quantity is considered to be "the variance" of the population. (Formally it is the variance of the empirical distribution of the population.) However, the first of these quantities is an unbiased estimator for the superpopulation variance, so we can estimate the variance of our mean-difference quantity by:

$$\begin{align} \hat{\mathbb{V}}(\bar{X}_n - \bar{X}_N) &= \frac{N-n}{N} \cdot \frac{S_N^2}{n} \\[6pt] &= \frac{N-n}{N} \cdot \frac{1}{n} \cdot \frac{1}{N-1} \sum_{i=1}^N (X_i-\bar{X}_N)^2 \\[6pt] &= \frac{N-n}{N-1} \cdot \frac{1}{n} \cdot \frac{1}{N} \sum_{i=1}^N (X_i-\bar{X}_N)^2 \\[6pt] &= \frac{N-n}{N-1} \cdot \frac{S_{N*}^2}{n} \\[6pt] &= FPC^2 \cdot \frac{S_{N*}^2}{n} . \\[6pt] \end{align}$$

Consequently, using the central limit theory we can establish the following confidence interval for the population mean $\bar{X}_N$:

$$\text{CI}_N(1-\alpha) = \Bigg[ \bar{X}_n \pm FPC \cdot \frac{t_{n-1, \alpha/2}}{\sqrt{n}} \cdot S_{N*} \Bigg].$$

You can see that the finite population correction term appears in the confidence interval formula in order to "correct" for the finite population. Taking $N \rightarrow \infty$ (so that the population of interest is the superpopulation) we get $FPC \rightarrow 1$, yielding the standard confidence interval for the mean parameter of a "large" population.

I hope this alternative presentation of the matter elucidates the correction term within the broader framework of the superpopulation model. I have always preferred this model of sampling theory, since it makes it simpler to distinguish between the finite population case and the infinite population case. As you can see, within this framework the correction term pops out fairly simply in the course of attempting to estimate the mean of the finite population.

Here is an alternative setup within the framework of the superpopulation model of sampling theory. It differs in notation and conception to classical sampling theory, but I think it is quite simple and intuitive.

Let $X_1,X_2,X_3,...$ be an exchangeable "superpopulation" of values. Take the first $N$ values to be the finite population of interest and the first $n \leqslant N$ values as a sample from this population. (The exchangeability of the superpopulation means that the sample is a simple-random-sample from the population.) Now, consider the mean-difference $\bar{X}_n - \bar{X}_N$ measuring the difference between the sample mean and population mean. This quantity can be written in the form:

$$\begin{align} \bar{X}_n - \bar{X}_N &= \frac{1}{n} \sum_{i=1}^n X_i - \frac{1}{N} \sum_{i=1}^N X_i \\[6pt] &= \Big( \frac{1}{n} - \frac{1}{N} \Big) \sum_{i=1}^n X_i - \frac{1}{N} \sum_{i=n+1}^N X_i \\[6pt] &= \frac{N-n}{nN} \sum_{i=1}^n X_i - \frac{1}{N} \sum_{i=n+1}^N X_i \\[6pt] &= \frac{1}{n} \Bigg[ \frac{N-n}{N} \sum_{i=1}^n X_i - \frac{n}{N} \sum_{i=n+1}^N X_i \Bigg]. \\[6pt] \end{align}$$

We clearly have $\mathbb{E}(\bar{X}_n - \bar{X}_N) = 0$, so we can use the sample mean as an unbiased estimator for the population mean. If we denote the variance of the superpopulation by $\sigma^2$ then our quantity has variance:

$$\begin{align} \mathbb{V}(\bar{X}_n - \bar{X}_N) &= \frac{1}{n^2} \Bigg[ \Big( \frac{N-n}{N} \Big)^2 \sum_{i=1}^n \mathbb{V}(X_i) + \Big( \frac{n}{N} \Big)^2 \sum_{i=n+1}^N \mathbb{V}(X_i) \Bigg] \\[6pt] &= \frac{1}{n^2} \Bigg[ \Big( \frac{N-n}{N} \Big)^2 n \sigma^2 + \Big( \frac{n}{N} \Big)^2 (N-n) \sigma^2 \Bigg] \\[6pt] &= \frac{1}{n^2 N^2} \Bigg[ (N-n)^2 n \sigma^2 + n^2 (N-n) \sigma^2 \Bigg] \\[6pt] &= \frac{1}{n^2 N^2} \cdot (N-n) N n \sigma^2 \\[6pt] &= \frac{N-n}{N} \cdot \frac{\sigma^2}{n}. \\[6pt] \end{align}$$

Suppose we let $S_N^2$ and $S_{N*}^2$ denote the variance values for the population, where the first uses Bessel's correction and the second does not (so we have $S_N^2 = \frac{N}{N-1} S_{N*}^2$). In classical sampling theory the latter quantity is considered to be "the variance" of the population. (Formally it is the variance of the empirical distribution of the population.) However, the first of these quantities is an unbiased estimator for the superpopulation variance, so we can estimate the variance of our mean-difference quantity by:

$$\begin{align} \hat{\mathbb{V}}(\bar{X}_n - \bar{X}_N) &= \frac{N-n}{N} \cdot \frac{S_N^2}{n}. \\[6pt] \end{align}$$

Consequently, using the central limit theorem we can establish the following confidence interval for the population mean $\bar{X}_N$:

$$\text{CI}_N(1-\alpha) = \Bigg[ \bar{X}_n \pm \sqrt{\frac{N-n}{N}} \cdot \frac{t_{n-1, \alpha/2}}{\sqrt{n}} \cdot S_{N} \Bigg].$$

This is the form of the confidence interval that I find the most natural. However, with this form, you will notice that we use a finite population correction term that is different to your expression. The expression you are using occurs when we convert to the variance estimator $S_{N*}^2$ that does not use Bessel's correction (purportedly "the variance" of the population). In this case we have the equivalent expression:

$$\text{CI}_N(1-\alpha) = \Bigg[ \bar{X}_n \pm \sqrt{\frac{N-n}{N-1}} \cdot \frac{t_{n-1, \alpha/2}}{\sqrt{n}} \cdot S_{N*} \Bigg].$$

As you can see, framed in this latter form, the finite population correction term is the one in your question. You can see that the finite population correction term appears in the confidence interval formula in order to "correct" for the finite population. Taking $N \rightarrow \infty$ (so that the population of interest is the superpopulation) we get $FPC \rightarrow 1$, yielding the standard confidence interval for the mean parameter of a "large" population.

I hope this alternative presentation of the matter elucidates the correction term within the broader framework of the superpopulation model. I have always preferred this model of sampling theory, since it makes it simpler to distinguish between the finite population case and the infinite population case. As you can see, within this framework the correction term pops out fairly simply in the course of attempting to estimate the mean of the finite population.