# Possible Paradox: Calculating a confidence interval with within-experiment error

This is a spinoff of

How to calculate the confidence interval of the mean of means?

and related to

When making inferences about group means, are credible Intervals sensitive to within-subject variance while confidence intervals are not?

Dataset 1 here is taken from the first link above. Dataset 2 has the approximately the same experimental means but different within experiment variance. My first question is:

1) How do I calculate a confidence interval for the overall mean for each of these data sets?

If I understand @Stéphane Laurent's answer in the two linked questions they should be the same.If that is true this goes strongly against all my scientific intuition and also appears to be a paradox.

2) How can it be that the confidence interval is apparently both sensitive to and not sensitive to within experiment error?

dataset 1:

  Experiment Value
1    34
1    41
1    39
2    45
2    51
2    52
3    29
3    31
3    35

structure(list(Experiment = structure(c(1L, 1L, 1L, 2L, 2L, 2L,
3L, 3L, 3L), .Label = c("1", "2", "3"), class = "factor"), Value = c(34,
41, 39, 45, 51, 52, 29, 31, 35)), .Names = c("Experiment", "Value"
), row.names = c(NA, -9L), class = "data.frame")


dataset2:

  Experiment    Value
1 38.20744
1 37.99410
1 37.96299
2 49.27085
2 49.40519
2 49.24894
3 31.81259
3 31.73708
3 31.73834

structure(list(Experiment = structure(c(1L, 1L, 1L, 2L, 2L, 2L,
3L, 3L, 3L), .Label = c("1", "2", "3"), class = "factor"), Value = c(38.2074373061779,
37.9941025108851, 37.9629896019425, 49.2708491636015, 49.4051867974062,
49.2489418702291, 31.8125943239769, 31.7370826901692, 31.7383364604132
)), .Names = c("Experiment", "Value"), row.names = c(NA, -9L), class = "data.frame")

• The answer is given by my answer in the first link: take the group means and draw a classical confidence interval for a Gaussian mean. So obivously CI are the same if group means are the same. Oct 24, 2013 at 6:13
• Right now I believe that a statement such as "In the long run, the between variance is a function of the within variance" is what I was looking for but I was asking the wrong questions and you were answering too mathematically for me to get. I need to think more on @John's answer below. Oct 24, 2013 at 6:59
• Sorry, I like to give less mathematical answers when needed, but I'm too busy. The important variance is $\sigma^2_b+\frac{\sigma^2_w}{J}$, which is the variance of the observed means. Here you're looking at one example only, but if you run simulations, the dispersion of the observed means increase with $\sigma^2_w$. Oct 24, 2013 at 11:43
• @Stéphane Your answers were good and I get it now. My remaining issue is that I am still not positive that it makes sense to assume within variation is related to between variation when analyzing a single data set. In the long run sure, but I am not sure it is useful to me to care about "the long run" as a researcher. Oct 24, 2013 at 21:21
• You have a sharp mind. I think I see why you are puzzled. Consider my notations here for the random one-way ANOVA model. The between-variance $\sigma^2_b$ is the variance of the theoretical means of the groups. It is not related to the within-variance $\sigma^2_w$. But the variance of the observed means $\bar y_{i\bullet}$ is $\sigma^2_b+\frac{\sigma^2_w}{J}$. You may think about this. Oct 24, 2013 at 22:25