A confidence interval is an interval that covers an unknown parameter with $(1-\alpha)\%$ confidence. Confidence intervals are a frequentist concept. They are often confused with credible intervals which is the Bayesian analog.
A confidence interval is an interval that covers an unknown parameter of interest (e.g., the mean) with $(1-\alpha)\%$ confidence. Confidence intervals are a frequentist concept. A credible interval is a related concept in Bayesian statistics. People often incorrectly ascribe the meaning of credible intervals to confidence intervals.
In frequentist statistics, a confidence interval for a parameter, $\theta$, is an interval computed from a set of data whose distribution depends on that parameter in some way. The interval is computed such that, if the process of drawing a sample and computing the interval were repeated identically ad infinitum, the proportion of the intervals that included the true value of the parameter would converge to $(1-\alpha)\%$. This does not mean that the probability of a given interval including the true value of the parameter is $(1-\alpha)\%$. Each interval either does include the true value or it does not include the true value. The 'confidence' is a property of the procedure used to compute the interval and pertains to the theoretical infinite set of such intervals.
Some additional notes:
- The confidence interval is a function of the data, $X$. Since the data are conceptualized as a random sample from a population, confidence intervals are random variables (although the confidence interval computed on a particular set of data is a realization).
- Often one can only compute approximate confidence intervals, which may have the nominal coverage asymptotically.
- It may not be possible to compute any exact confidence interval that might otherwise be preferred if the data or the parameter can only take discrete or otherwise limited values.
- The same ideas can be applied to a set of parameters, e.g., $\vec{\theta} = [\mu\ \ \sigma^2]^T$. In that case, it is more correct to refer to the confidence region.
- In a regression context, the set of confidence intervals for all possible conditional means ($\mu_Y|X=x$) is called a confidence band.