17
$\begingroup$

Suppose one has two independent samples from the same population, and different methods were used on the two samples to derive point estimate and confidence intervals. In trivial cases a sensible person would just pool the two samples and use one method to do the analysis, but let's suppose for the moment that different method has to be used due to limitation of one of the sample such as missing data. These two separate analyses would generate independent, equally valid estimates for the population attribute of interest. Intuitively I think there should be a way to properly combine these two estimates, both in terms of point estimate and confidence interval, resulting in a better estimation procedure. My question is what should be the best way to do it? I can imagine a weighted mean of some sort according to the information/sample size in each sample, but what about the confidence intervals?

$\endgroup$
9
$\begingroup$

You could do a pooled estimate as follows. You can then use the pooled estimates to generate a combined confidence interval. Specifically, let:

$\bar{x_1} \sim N(\mu,\frac{\sigma^2}{n_1})$

$\bar{x_2} \sim N(\mu,\frac{\sigma^2}{n_2})$

Using the confidence intervals for the two cases, you can re-construct the standard errors for the estimates and replace the above with:

$\bar{x_1} \sim N(\mu,SE_1)$

$\bar{x_2} \sim N(\mu,SE_2)$

A pooled estimate would be:

$\bar{x} = \frac{n_1 \bar{x_1} + n_2 \bar{x_2}}{n_1 + n_2}$

Thus,

$\bar{x} \sim N(\mu,\frac{{n_1}^2 SE_1 + {n_2}^2 SE_2}{(n_1+n_2)^2})=N(\mu,\frac{\sigma^2}{n_1+n_2})$

$\endgroup$
  • 1
    $\begingroup$ This approach would work if we assume that our CI are of the form $\hat{\beta}\pm Z_{\alpha}SE$. Unfortunately, sometimes asymmetric CI may be more sensible, for example the CI for a binomial proportion when it's close to 0. In that case pooling the SE like this may not help. $\endgroup$ – user1600 Oct 15 '10 at 20:52
  • $\begingroup$ @user1600 Good point. $\endgroup$ – user28 Oct 15 '10 at 22:14
  • $\begingroup$ This answer could be applied to any two distributions, it is just that the product of normals is a normal, giving a nice solution. MCMC simulation could be used with pairs of distributions without a closed form solution, using a Bayesian approach with one sample being the prior and the other the likelihood. $\endgroup$ – David LeBauer Oct 17 '10 at 4:38
  • $\begingroup$ If going back to confidence intervals from the pooled SE, what would the degrees of freedom for the T distribution be? Would this change if combining more than 2 confidence intervals? $\endgroup$ – DocBuckets May 23 '13 at 19:08
3
$\begingroup$

Sounds a lot like meta-analysis to me. Your assumption that the samples are from the same population means you can use fixed-effect meta-analysis (rather than random-effects meta-analysis). The generic inverse-variance method takes a set of independent estimates and their variances as input, so doesn't require the full data and works even if different estimators have been used for different samples. The combined estimate is then a weighted average of the separate estimates, weighting each estimate by the inverse of its variance. The variance of the combined estimate is the inverse of the sum of the weights (the inverses of the variances).

You want to work on a scale where the sampling distribution of the estimate is approximately normal, or at least a scale on which the confidence intervals are approximately symmetric, so a log transformed scale is usual for ratio estimates (risk ratios, odds ratios, rate ratios...). In other cases a variance-stabilising transformation would be useful, e.g. a square-root transformation for Poisson data, an arcsin-square-root transformation for binomial data, etc.

$\endgroup$
1
$\begingroup$

This is not unlike a stratified sample. So, pooling the samples for a point estimate and standard error seems like a reasonable approach. The two samples would be weighted by sample proportion.

$\endgroup$
0
$\begingroup$

See paper: K.M. Scott, X. Lu, C.M. Cavanaugh, J.S. Liu, Optimal methods for estimating kinetic isotope effects from different forms of the Rayleigh distillation equation, Geochimica et Cosmochimica Acta, Volume 68, Issue 3, 1 February 2004, Pages 433-442, ISSN 0016-7037, http://dx.doi.org/10.1016/S0016-7037(03)00459-9. (http://www.sciencedirect.com/science/article/pii/S0016703703004599)

$\endgroup$

We're looking for long answers that provide some explanation and context. Don't just give a one-line answer; explain why your answer is right, ideally with citations. Answers that don't include explanations may be removed.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.