I am learning about confidence intervals, but don't think I understand them very welll.

Assume $$(\mu - \hat{\mu}) \sqrt{\frac{n}{\sigma(\mu)}}$$ is asymptotically standard normal. So I guess we can say that a 95 % CI is $\hat{\mu} \pm 1.96 \sqrt{\sigma(\mu)/n}$.

I am a bit confused with respect to this, since $\sigma$ is a function of $\mu$.

  1. Assume $\mu$ is unknown. How can we form a confidence interval given that $\sigma$ depends on $\mu$? I know we can just plug in $\hat{\mu}$ and pray it's close, but what if we don't want to do that? What are the alternatives, if any? What do people do in practice?

  2. Assume $\mu$ is known. What is the interpretation of $\hat{\mu} \pm 1.96 \sqrt{\sigma(\mu)/n}$? I mean, if I know what $\mu$ is, does it still make sense to talk about confidence intervals around $\mu$? Isn't a "100 %" confidence interval then $[\mu, \mu]$, if that makes sense?


1 Answer 1


The whole game is about learning something about $\mu$, so you're right in your first question: the starting point is that $\mu$ is unknown and you want to use data and an estimator to learn about it. Using a given dataset, you can form an estimator $\hat \mu$ to estimate the value of $\mu$.

Formally, we usually think about $\mu$ as being a parameter (for instance, the expectation of a random variable, or the coefficient in a linear regression), and the estimator $\hat \mu$ as being a random variable, which depends on the realisation of the data. For a given dataset, you will obtain a given estimate.

Let us assume (as you do) that $\hat \mu$ is normally distributed. Its expectation is equal to $\mu$ (which happens when the estimator is consistent). Expectation, in this case, means that if you were to observe not one dataset, but a large number of datasets, the average value of the $\hat \mu$ over these datasets would be equal to $\mu$.

The variance of $\hat \mu$ is a function of two quantities: $\sigma$ the asymptotic/underlying variance, and $n$ the number of observations. $\sigma$ essentially depends on the data generating process of the random variable (for instance, the variance of the underlying random variable). Variance of $\hat \mu$ means: how would my $\hat \mu$ vary if I computed it many times, on many datasets of size $n$? If $n$ was very large, all the $\hat \mu$ would be pretty close to each other (and pretty close to $\mu$).

Now that we have all this, let's answer your question. You're right that the formula of the variance (and CI) of $\hat \mu$ depends on $\sigma$ and we don't observe $\sigma$. What is usually done is to plug an estimator of $\sigma$ instead. In many cases, just like you can form an estimator of $\mu$, you can form an estimator of $\sigma$, that you call $\hat \sigma$, which can be computed as a function of the data.

For instance, the canonical problem is that you have a random variable $Y_i$ distributed in a $\mathcal N(\mu, \sigma^2)$. In this case, we can take $\hat \mu$ to be the average of the observed $Y_i$. An unbiased estimator of the variance $\sigma^2$ is then: $$ \hat \sigma = \frac{\sum_i Y_i^2}{n} - \hat \mu^2 $$ Note that $\hat \sigma$ depends on $n$ and on the observations of the dataset $\{Y_i\}$ and on the estimator $\hat \mu$ (which also depends on $\{Y_i\}$ and $n$), but not on $\mu$. $\hat \sigma$ is the quantity you will plug into your confidence interval.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.