Forming confidence interval using a sampling distribution

Question

I know the relation between the distribution of sample means (for a given sample size) and the population parameters, from which samples are taken.

My question is, how exactly do we use this relation to do interval estimation? More specifically, when we have a sample and attempt to make an inference about population from which sample is taken, we form the distribution of sample means as if the population mean is equal to mean of our sample, and then use this distribution as the probability density function of the population mean.

It seems right, intuitively, but I couldn't see a formal proof.

Edit: This question is because of a big confusion. I was taught that, X % confidence intervals are intervals, where parameter to be inferred lies with X % probability. As that statement is wrong, this question is also meaningless.

Answer 1

You seem to have a number of confusions in your question. I think it might be best to address those by explaining a way to construct confidence intervals, which might help to clarify some of the ideas.

There are a number of ways of constructing confidence intervals, but let me give you a basic (somewhat informal-and-handwavy) rundown of a standard approach.

Step 1: Form a pivotal quantity for the parameter in question. That is, form a statistic, a function of the parameter ($\theta$, say) and the data $\mathbf{x}$, $Q=g(\mathbf{x},\theta)$ whose distribution doesn't depend on $\theta$. We can specify an interval that would contain the desired fraction of random $Q_i$'s (has the desired coverage probability). So we could for example, write an inequality representing the interval along the lines of $l\leq Q \leq u$, which across many such random $Q$ values would contain the given proportion of them.

Step 2: Back out an interval in terms of $\theta$ (i.e. rearrange the inequality to put $\theta$ by itself in between the $\leq$ values). Everything else in the inequality is determined once the sample is observed. This will have the desired coverage for $\theta$.

Many of the common parametric intervals can be derived this way.

e.g. for i.i.d. data from a $N(\mu,\sigma^2)$, a pivotal quantity for $\mu$ is $Q = \frac{\bar{x}-\mu}{s/\sqrt{n}}$, since $Q$ has a standard t-distribution with $n-1$ d.f.