How to calculate the confidence interval of the mean of means? Imagine that you repeat an experiment three times. In each experiment, you collect triplicate measurements. The triplicates tend to be fairly close together, compared to the differences among the three experimental means. Computing the grand mean is pretty easy. But how can one compute a confidence interval for the grand mean?
Sample data:
Experiment 1:   34, 41, 39
Experiment 2:   45, 51, 52
Experiment 3:   29, 31, 35
Assume that the replicate values within an experiment follow a Gaussian distribution, as does the mean values of each experiment. The SD of variation within an experiment is smaller than the SD among experimental means. Assume also that there is no ordering of the three values in each experiment. The left-to-right order of the three values in each row is entirely arbitrary.
The simple approach is to first compute the mean of each experiment: 38.0, 49.3, and 31.7, and then compute the mean, and its 95% confidence interval, of those three values. Using this method, the grand mean is 39.7 with the 95% confidence interval ranging from 17.4 to 61.9. 
The problem with that approach is that it totally ignores the variation among triplicates. I wonder if there isn't a good way to account for that variation. 
 A: This is a question of estimation within a linear mixed effects model.  The problem is that the variance of the grand mean is a weighted sum of two variance components which have to be separately estimated (via an ANOVA of the data).  The estimates have different degrees of freedom.  Therefore, although one can attempt to construct a confidence interval for the mean using the usual small-sample (Student t) formulas, it is unlikely to attain its nominal coverage because the deviations from the mean will not exactly follow a Student t distribution.
A recent (2010) article by Eva Jarosova, Estimation with the Linear Mixed Effects Model, discusses this issue.  (As of 2015 it no longer appears to be available on the Web.)  In the context of a "small" dataset (even so, about three times larger than this one), she uses simulation to evaluate two approximate CI calculations (the well-known Satterthwaite approximation and the "Kenward-Roger's method").  Her conclusions include

Simulation study revealed that quality of estimation of covariance parameters and consequently adjustment of confidence intervals in small samples can be quite poor.... A poor estimation may influence not only the true confidence level of conventional intervals but it can also make the adjustment impossible. It is obvious that even for balanced data three types of intervals [conventional, Satterthwaite, K-R] may differ
  substantially. When a striking difference between the conventional and the adjusted intervals is observed, standard errors of covariance parameter estimates should be checked.  On the other hand, when the differences between [the three] types of intervals are small, the adjustment seems to be unnecessary.

In short, a good approach seems to be


*

*Compute a conventional CI by using the estimates of variance components and pretending a t-distribution applies.

*Also compute at least one of the adjusted CIs.

*If the computations are "close," accept the conventional CI.  Otherwise, report that there are insufficient data to produce a reliable CI.
A: There is a natural exact confidence interval for the grandmean in the balanced random one-way ANOVA model $$(y_{ij} \mid \mu_i) \sim_{\text{iid}} {\cal N}(\mu_i, \sigma^2_w), \quad j=1,\ldots,J, 
\qquad 
\mu_i \sim_{\text{iid}} {\cal N}(\mu, \sigma^2_b), \quad i=1,\ldots,I.$$
Indeed, it is easy to check that the distribution of the observed means $\bar{y}_{i\bullet}$ is $\bar{y}_{i\bullet} \sim_{\text{iid}} {\cal N}(\mu, \tau^2)$ with $\tau^2=\sigma^2_b+\frac{\sigma^2_w}{J}$, 
and it is well known that the between sum of squares $SS_b$ has distribution $$SS_b \sim J\tau^2\chi^2_{I-1}$$ and is independent of the overall observed mean $$\bar y_{\bullet\bullet} \sim {\cal N}(\mu, \frac{\tau^2}{I})$$. 
Thus $$\frac{\bar y_{\bullet\bullet}  - \mu}{\frac{1}{\sqrt{I}}\sqrt{\frac{SS_b}{J(I-1)}}}$$ has a Student $t$ distribution with $I-1$ degrees of freedom, wherefrom it is easy to get an exact confidence interval about $\mu$.
Note that this confidence interval is nothing but the classical interval for a Gaussian mean by considering only the group means $\bar{y}_{i\bullet}$ as the observations. 
Thus the simple approach you mention:

The simple approach is to first compute the mean of each experiment:
  38.0, 49.3, and 31.7, and then compute the mean, and its 95% confidence interval, of those three values. Using this method, the
  grand mean is 39.7 with the 95% confidence interval ranging from 17.4
  to 61.9.

is right. And your intuition about the ignored variation:

The problem with that approach is that it totally ignores the
  variation among triplicates. I wonder if there isn't a good way to
  account for that variation.

is wrong. I also mention the correctness of such a simplification in https://stats.stackexchange.com/a/72578/8402
Update 12/04/2014
Some details are now written on my blog: Reducing a model to get confidence intervals.
A: You can't have one confidence interval that solves both of your problems.  You have to pick one.  You can either derive one from a mean square error term of within experiment variance that allows you to say something about how accurately you can estimate the values within experiment or you can do it between and it will be about between experiments.  If I just did the former I'd tend to want to plot it around 0 rather than around the grand mean because it doesn't tell you anything about the actual mean value, only about an effect (in this case 0).  Or you could just plot both and describe what they do.
You've got a handle on the between one.  For the within it's just like calculating the error term in an ANOVA to get an MSE to work with and from there the SE for the CI is just sqrt(MSE/n) (n = 3 in this case).
A: I think the CI for grand mean is too wide [17,62] even for the range of original data.
This experiments are VERY common in chemistry. For example, in certification of reference materials you have to pick up some bottles from whole lot in a random way, and you have to carry out replicate analysis on each bottles. How do you calculate the reference value and its uncertainty? There are a lot of way to do it, but the most sofisticated (and correct, I think) is applying meta-analysis or ML (Dersimonian-Laird, Vangel-Rukhin, etc)
What about bootstrap estimates?
