Let's say I have taken a sample $S$ of a population. I am trying to figure out the population mean.
Because I have only made one sample, the best I can do is assume that $\bar S$ is the mean of the population. I can't get a confidence interval for the population mean, because I can't calculate the standard deviation of a single sample mean.
But I could break up my initial set $S$ into disjoint subsets $S_i$ such that $S=S_1 \cup S_2 \cup S_3 \ \cup ... \cup \ S_i$. Then, I pretend that each $S_i$ was a separate sample of the population. I estimate the population mean by averaging $\mu_i = \bar S_i$. I calculate the error of the mean by calculating the standard deviation of $\sigma_i = std(\mu_i)$.
I now have an estimate of the sample mean, and a confidence interval for it too, even though in reality I only sampled the population once. This isn't right, is it? Why/why not?
You can assume that the population is very large, and $S$ is very small, if that helps.