Let's say I have taken a sample $S$ of a population. I am trying to figure out the population mean.

Because I have only made one sample, the best I can do is assume that $\bar S$ is the mean of the population. I can't get a confidence interval for the population mean, because I can't calculate the standard deviation of a single sample mean.

But I could break up my initial set $S$ into disjoint subsets $S_i$ such that $S=S_1 \cup S_2 \cup S_3 \ \cup ... \cup \ S_i$. Then, I pretend that each $S_i$ was a separate sample of the population. I estimate the population mean by averaging $\mu_i = \bar S_i$. I calculate the error of the mean by calculating the standard deviation of $\sigma_i = std(\mu_i)$.

I now have an estimate of the sample mean, and a confidence interval for it too, even though in reality I only sampled the population once. This isn't right, is it? Why/why not?

You can assume that the population is very large, and $S$ is very small, if that helps.

  • $\begingroup$ "I can't get a confidence interval for the population mean, because I can't calculate the standard deviation of a single sample mean."-- you can't? I could understand saying you can't form a confidence interval unless the sample size is sufficiently large, but standard deviations of sample means (assuming independence) can be estimated from single samples. $\endgroup$ – Glen_b Apr 2 '14 at 8:27
  • $\begingroup$ @Glen_b so are you saying I can take $std(S)$ to be the error of the mean? $\endgroup$ – Superbest Apr 2 '14 at 8:36
  • $\begingroup$ I've made some edits to address the question of forming a confidence interval. $\endgroup$ – Glen_b Apr 2 '14 at 8:58

A brief exposition on standard errors of sample means

Consider a sample from some population (with mean $\mu$ and variance $\sigma^2$).

A random sample is taken from the population. It has sample mean $\bar x$ and sample variance $s^2_x$ (the usual $n-1$-denominator version).

$E(\bar X)=\mu$, and $E(s^2_x)=\sigma^2$.

Some basic facts about variance:

i) The variance of the sum of independent random variables is the sum of the variances.

ii) $\text{Var}(cX)=c^2\text{Var}(X)$

Hence $\text{Var}(\bar X)=\text{Var}(\frac{1}{n}\sum_i X_i)=\frac{1}{n^2}\text{Var}(\sum_i X_i)=\frac{1}{n}\sigma^2$

Hence $\text{sd}(\bar X)=\sqrt{\text{Var}(\bar X)}=\sqrt{\frac{1}{n}\sigma^2}=\sigma/\sqrt{n}$

Now we have an estimate of $\sigma^2$, which is $s_x^2$, so we can estimate $\text{sd}(\bar x)$ by $s_x/\sqrt{n}$

The standard deviation of the sample mean is called the standard error of the mean.

Now the actual issue is how to form a confidence interval for the population mean from a sample.

Even though we have an estimate of the standard error, this doesn't translate to a confidence interval.

In sufficiently large samples (as long as the required conditions apply), we can apply the central limit theorem and Slutsky's theorem (and a short argument) to form a confidence interval for the mean.

$\bar x\pm z_{1-\alpha/2}s/\sqrt{n}$

where $z_{1-\alpha/2}$ is the ${1-\alpha/2}$ percentile of a standard normal distribution.

If the population is close to normally distributed, we can form a small sample t-interval:

$\bar x\pm t_{1-\alpha/2}s/\sqrt{n}$

where the $t$ distribution used to compute the required percentile has $n-1$ degrees of freedom.

Otherwise, there are other means to generate intervals, such as bootstrapping (which is not terribly dissimilar to your idea, but makes fuller use of the data). This works reasonably well if the sample is not too small.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.