This deserves a simple derivation. The idea is to separate analysis of the population from analysis of the purely combinatorial process of sampling without replacement.
Let the population values be $(x_1,x_2,\ldots, x_N)$ with mean $\mu$ and variance $\sigma^2.$ We are interested in the standard error of the sample mean.
Let $I_j$ be the indicator of whether $x_j$ is in a random sample of size $K.$ These random variables are identically distributed (but not independent!) Bernoulli random variables and since they sum to $K,$ $\Pr(I_j=1) = K/N$ whence for $j=1,2,\ldots, N,$
$$\operatorname{Var}(I_j) = \frac{K}{N}\left(1 - \frac{K}{N}\right) = \frac{K(N-K)}{N^2}.$$
Moreover, since the $I_j$ sum to the constant random variable equal to $K,$
$$\begin{aligned} 0 &= \operatorname{Var}(K) = \operatorname{Var}\left(\sum_{j=1}^N I_j\right) = \sum_{j=1}^N \operatorname{Var}(I_j) + \sum_{i\ne j}^N \operatorname{Cov}(I_i,I_j). \end{aligned}$$
From the identical distributions of the $I_j$ all $N$ variances are equal to each other and all $N(N-1)$ covariances are equal to each other. Solving this equation yields
$$\operatorname{Cov}(I_i,I_j) = -\frac{1}{N-1}\operatorname{Var}(I_i) = -\frac{K(N-K)}{N^2(N-1)}.$$
That's the crux of the matter: sampling without replacement induces computable negative correlations among the indicators.
The rest is straightforward algebra. The sample mean is
$$\bar X = \frac{1}{K} \sum_{j=1}^N x_jI_j,$$
whence it has expectation $E[\bar X]=\mu$ (easily) and variance
$$\begin{aligned} \operatorname{Var}(\bar X) &= \frac{1}{K^2}\sum_{i=1}^N\sum_{j=1}^N x_ix_j\operatorname{Cov}(I_i,I_j) \\ &= \frac{1}{K^2}\sum_{j=1}^N x_j^2 \frac{K(N-K)}{N^2} - \frac{1}{K^2}\sum_{i\ne j}^N x_ix_j \frac{K(N-K)}{N^2(N-1)}\\ &= \frac{N-K}{K(N-1)}\,\frac{1}{N}\left(\sum_{j=1}^N x_j^2 - \frac{1}{N}\left(\sum_{j=1}^N x_i\right)^2\right)\\ &= \frac{N-K}{K(N-1)}\sigma^2. \end{aligned}$$
That's all there is to it. (Please notice that $\sigma^2$ is the actual population variance, not an estimator. It differs by using a fraction $1/N$ rather than $1/(N-1)$ in its definition.)
The usual "infinite" population formula for the standard error of $\bar X$ is $\sqrt{\sigma^2/K}.$ The foregoing result multiplies this by $\sqrt{(N-K)/(N-1)},$ the so-called "finite population correction factor" for random samples without replacement. A first-order approximation is
$$\sqrt{\frac{N-K}{N-1}} \ \approx\ 1 - \frac{K-1}{2(N-1)}\ \approx \ 1 - \frac{K}{N}.$$
Thus, the relative sample size $K/N$ governs the accuracy of the approximation. The 5% threshold merely assumes you want to estimate the standard error of the mean to within 5% accuracy.