1
$\begingroup$

I carried out a GEE Poisson regression on my dependent variable, number of days, and my independent variables are binary categories, including a high/low treatment indicator of interest. I obtained the following 95% confidence intervals for a high treatment group and low treatment group (the other binary covariates are held fixed):

High Treatment Group Point Estimate, (95% CI): 10.7099, (10.2836, 11.1540)

Low Treatment Group Point Estimate, (95% CI): 8.9673, (6.3788, 12.6062)

I noted that the two confidence intervals overlap, which leads me to believe that the two treatment groups means are not statistically significantly different.

However, I then calculated an estimate of the predicted group differences and the corresponding confidence interval (e.g. $\hat{\mu_{high}}-\hat{\mu_{low}}$), using the Delta method, and I got the following:

1.7426, (0.1714438, 3.313756).

Now, I noted that the the point estimate makes perfect sense as it is $10.7099-8.9673=1.7426$, but the confidence interval does not contain zero, which implies that the difference in means is statistically significantly different fro zero.

Why are the two conclusions different? Is the confidence made smaller in and therefore significant in the mean difference calculations because maybe, I'm taking advantage of the covariances between the estimates or is something else maybe going on here? Which one of the conclusions should I believe?

UPDATE: I've found some articles that talk about how overlapping confidence intervals don't necessarily imply that the differences between estimated means are not significant in a t-test. The made reference to the sample size being larger (and hence smaller confidence intervals) in the two-sided t-test. That doesn't seem to be going on here since I'm using regression. Can anyone help?

$\endgroup$
1
$\begingroup$

You should believe you second conclusion. As you stated you need to take into account the covariance between the two values. The hypothesis you are interested in testing is \begin{equation} H_0: \mu_{high} - \mu_{low} = \mu_{diff} = 0 \end{equation} So to construct a confidence interval for $\mu_{diff}$ we need the standard error of $\hat \mu_{diff}$, we have the variance is given by \begin{align*} Var(\hat \mu_{diff}) &= Var(\hat \mu_{high} - \hat \mu_{low}) \\ & = Var(\hat \mu_{high}) + Var(\hat \mu_{low}) - 2 \times Cov(\hat \mu_{high}, \hat \mu_{low}) \end{align*} You can take the square root to get the standard error and calculate the confidence interval from there to preform inference.

$\endgroup$
  • $\begingroup$ Thanks. So am I correct in that the difference essentially becomes "statistically significant" because we are subtracting off twice the covariance instead of independently examining the confidence intervals of each of the measurements? $\endgroup$ – StatsStudent Mar 25 '16 at 20:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.