# Unable to understand what confidence interval means

I seek to understand what confidence interval is with the aid of following example (which I know how to solve but do not understand the rationale behind it);

Suppose it is known that the weight of cement in packed bags is distributed normally with a standard deviation of 0.2 Kg. A sample of 25 bags is picked up at random and the mean weight of cement in these 25 bags is only 49.7 Kg. We want to find a 90% confidence interval for the mean weight of cement in filled bags.

The textbook from which this question comes explains confidence interval as follows;

The confidence level, therefore, may be defined as the probability that the interval estimate will contain the true value of the population parameter that is being estimated. If we say that a 95% confidence interval for the population mean is obtained by spanning 1.96 times the standard error of the mean on either side of the sample mean, we mean that we take a large number of samples of size n, say 1000, and obtain the interval estimates from each of these 1000 samples and then 95% of these interval estimates would contain the true population mean.

The textbook also states that;

It is to be noted that the true population mean is a constant and is not a variable. On the other hand, the interval that we specify is a random interval whose position depends on the sample mean.

Questions:

1. While solving the example will we assume that the estimate of true population mean is the mean of 1 sample we have picked i.e. 49.7 Kg or will the estimate of true population mean be one of the values in the interval we eventually calculate (49.6342, 49.7658)?

2. We say that probability that the sample picked at random from the population has mean ranging from (x - 1.645(sigma)/root(n) to (x + 1.645(sigma)/root(n) has probability of 90%. How does this translate to probability of picking up a sample which contains true population mean in the interval (x - 1.645(sigma)/root(n) to (x + 1.645(sigma)/root(n)?

If these questions don't make any sense, I will be grateful if the concept of confidence intervals could be explained with respect to the example above. There seem to be gaps in my knowledge.

• This is off the topic, but I can't resist saying that it feels very unlikely that a real-world process has a known standard deviation and unknown mean. If I have a sample of 25 observations I would use the observed standard deviation. Commented Feb 17 at 20:40
• @MichaelLew it would require a t-test to make the example rather than a z-test. That is more difficult as example. It is not very unlikely in a real-world. A case with known standard deviation could be when there is potentially a theoretical reasoning that the standard deviation is known, for example a physical or chemical experiment with known deviations. In the case of cement bags there might be historical knowledge about the variation in the cement bags. And the test is only to verify some drift of the mean, where it is known that a drift in variance is negligible. Commented Feb 18 at 1:23
• Also, the 0.2 kg might be the sample based estimate of the variance, but the example wishes to use a z-test approximation of a t-test. Commented Feb 18 at 1:27
• In education, we frequently teach and create examples which are simpler to work out than the student will face in the real world. The purpose is to give students a simpler process at first, then as they get familiar with and comfortable with it, refine the process to that which is more commonly used in the real world. It's called scaffolding. Every teacher I know of (US based, likely in Latin America also) has likely been trained in it Some processes are so complicated it take more than one semester to develop the skill to apply the real world process. Commented Feb 18 at 8:35
• Please include a proper reference when you are quoting from someone else's work. In this case it seems that the textbook you refer to is Six Sigma for Organizational Excellence: A Statistical Approach. Commented Feb 18 at 10:11

1. While solving the example will we assume that the estimate of true population mean is the mean of 1 sample we have picked i.e. 49.7 Kg or will the estimate of true population mean be one of the values in the interval we eventually calculate (49.6342, 49.7658)?

There is not 'the' estimate but there are several possible estimates. A common estimate is the population mean, 49.7, which is also known as the maximum likelihood estimate (given the assumed population properties of a Gaussian distribution).

To indicate not just the maximum, but also that multiple other values can be equally good candidates for an estimate we use a range or interval (typically the more precise and large the sample, the smaller this range can be made).

A confidence interval is a range of potential estimates for which the confidence is high. For any of the values in the range, if it would be the true value, then our observed sample mean of 49.7 wouldn't be an unlikely outlier and would be within at least 90% of the observations.

1. We say that probability that the sample picked at random from the population has mean ranging from (x - 1.645(sigma)/root(n) to (x + 1.645(sigma)/root(n) has probability of 90%. How does this translate to probability of picking up a sample which contains true population mean in the interval (x - 1.645(sigma)/root(n) to (x + 1.645(sigma)/root(n)?

The logic of a confidence interval is inversed. It relates to the probability of the data, given the predicted value (instead of the inverse 'the probability of the predicted value, given the data', which relates to a credibility interval).

The confidence interval can be seen as a range of population parameter values for which the data has a high probability (but as a fiducial probability based on p-values rather than a likelihood).

To construct a 90% confidence interval

• For each hypothetical population parameter you compute a range within which 90% of the observed samples would fall*.

• Following that you pick the hypothetical population parameters for which the observed sample is within their 90% range of observations.

Conditional on the true population parameter, in 90% of the time you will have observed a sample such that you picked that true population value as part of the confidence interval, in 10% of the time you will have observed a sample for which you compute an erroneous range.

See for instance the image from

The basic logic of constructing a confidence interval

and

Can we reject a null hypothesis with confidence intervals produced via sampling rather than the null hypothesis?

The idea behind a 90% confidence interval containing 90% of the time the true population is the conditioning on the true parameter instead of the observed experimental sample see: Why does a 95% Confidence Interval (CI) not imply a 95% chance of containing the mean?

* The computation of such 90% range can be done differently, and there is not a single 'the' confidence interval, instead there are many different ways to compute a confidence interval with different tradeoffs. See e.g. Asymmetric confidence intervals and What is a rigorous, mathematical way to obtain the shortest confidence interval given a confidence level?

For question 1, the estimate of the 49.7. An estimate is a number. The standard error of that estimate is a measure of how good the estimate is, given certain assumptions.

I'm not sure I understand your question 2, but I do know that the concept of confidence interval is very confusing and one must be careful. The definition given in the text that you quote is correct. Many people get this confused with the notion of a credible interval that comes from Bayesian statistics.

There has been a lot of words written in the literature about the meaning of a confidence interval and I have been scratching my head so many times about it. Whenever you think you get it, then you read someone else's definition and you are lost again. In the end, the best way for me was to simulate the meaning of a confidence interval (see figure below). And this is the best way since the confidence interval is a frequentist, i.e. a long run frequency, concept, which relies on assumptions for it to work the way it is intended.

In my own words, a confidence interval with a given probability, say 95%, will tell you that you can be 95% certain that this specific interval has coverage of the true population mean and in 5% of the time it does not cover the true population mean. So the probability statement is about the interval, i.e. the interval varies and the population parameter is fixed, or in other words, the interval jumps around the fixed population parameter in a predictiable fashion under repeated sampling (provided that the sampling process occurs at random and under the same conditions).

I try to avoid saying things like there is 95% probability that the true population mean is in that interval because such a statement usually implies that the population parameter is variable and the data is fixed (aka a Bayesian credible interval). However, for any given confidence interval the population parameter is either in the interval or it is not in the interval.

In my mind, the confusion arises since we generally assume that the population parameter is fixed. Then in this case saying that I can be 95% certain that this specific interval has coverage of the true population mean versus there is 95% probability that the true population mean is in that interval has somewhat the same meaning. Once I figured this out for myself, I made sure to explicitly attach the probability statement to the interval whenever talking about a confidence interval.

Here's the simulation (test your own definitions against this figure):

And the code to generate it (may not be the prettiest or the simplest piece of coding but it works):

# required package
library(tidyverse)
library(patchwork)

# create a simulation function for 95% confidence intervals and p-values
simulation <- function(n, mu, stdev) {
s <- rnorm(n, mu, stdev)
tibble(
N = length(s),
sample_mean = mean(s),
sample_sd = sd(s),
sample_se = sample_sd / sqrt(N),
confint_95 = sample_se * qt(0.975, N - 1),
t_statistic = (sample_mean - mu) / sample_se,
p_value = (1 - pt(abs(t_statistic), N - 1)) * 2
)
}

# specify parameters and sample size for each draw or experiment
n <- 10 # sample size
mu <- 0 # population mean
stdev <- 5 # population standard deviation

############ SIMULATION ##############

# set.seed to generate the exact same output
# remove it rerun it to see the stochastic nature of the process
set.seed(126)

# choose number of repetitions
sim <- 1:100

# rerun experiment under exact same conditions
map(sim, ~ simulation(n, mu, stdev)) %>%
bind_rows() %>%
mutate(experiment_id = 1:length(sim)) -> draws

# intervals that do not capture the true population mean are colored red (same as significant p-values)
draws %>%
mutate(
color_confint = if_else(
sample_mean - confint_95 > mu |
sample_mean + confint_95 < mu,
"red",
"black"
),
color_p = if_else(p_value <= 0.05, "red", "black")
) -> draws_colored

# check 95% confidence
draws_colored %>%
group_by(color_confint) %>%
tally() %>%
mutate(prop = n / sum(n) * 100)

# plot 95% confidence intervals
ggplot(data = draws_colored,
aes(x = sample_mean, y = experiment_id, color = color_confint)) + geom_point() +
geom_errorbarh(aes(xmax = sample_mean + confint_95, xmin = sample_mean - confint_95)) +
geom_vline(xintercept = mu) +
scale_colour_manual(name = "95% CI", values = c("black", "red")) +
labs(x = "Sample means", y = "Experiment ID") + theme(legend.position = "none") -> p1

# plot p-values
ggplot(data = draws_colored, aes(x = p_value, y = experiment_id, color = color_p)) + geom_point() +
geom_vline(xintercept = 0.05) +
scale_colour_manual(name = "p-values", values = c("black", "red")) +
labs(x = "p-values", y = "Experiment ID") + theme(legend.position = "none") -> p2

# OUTPUT WILL ALWAYS SLIGHLTY CHANGE SINCE NEW SAMPLES ARE BEING DRAWN RANDOMLY
# ON AVERGAE HOWEVER 5 OUT 100 CONFIDENCE INTERVALS WILL NOT CAPTURE THE TRUE POPULATION MEAN
# -> THE DEFINITION OF A 95% CONFIDENCE INTERVAL

# patch plots together (requires the patchwork package)
p <- p1 / p2

p