6
$\begingroup$

I'm analyzing a dataset including 13000 students. Students are clustered into schools/grades. The ICC (intraclass correlation coefficient) shows that students in a same school are correlated. Therefore, I'd to take this clustering effect into account. One way is to run a linear regression and run the robust variance estimator on top of that to guard against getting biased estimates. Can we take the clustering effect into account with the sandwich estimator?. I think we cannot since the epsilon^2 matrix is still a diagonal matrix. What I think we should do is to run a GEE model with exchangeable varcov matrix for students in a same school and then we should run the robust varcov on top of the GEE model.

Anyways, assuming that sandwich estimator by itself can account for the school clustering effect, I've fitted two separate GEE models one with exchangeable varcov matrix and the other one with the robust variance estimator (also known as Huber-White, Sandwich Estimator, or empirical variance estimator). The point is under both models I get the same estimated variance per each covariate, but my GEE exchangeable estimates leads to much larger beta estimates that are also statistically significant whereas similar beta covariates are not significant in GEE with robust varcov estimator. I wonder why it happens? I use SAS 9.3 to fit my model (proc GENMOD):

Exchange:

repeated subject = SCHIID/type = EXCH;

Empirical Estimator:

repeated subject = SCHIID/ covb; 
$\endgroup$
  • $\begingroup$ I can't follow this question. Eg, "under both models I get the same estimated variance per each covariate", does that mean the beta estimates are the same, or the SE's are, or what? And, "in my GEE exchangeable estimates, I get much larger estimates... whereas similar covariates", does that mean the beta estimates are larger for the exchangeable covariance than HW? Also, what software are you using & what commands did you use? Can you copy & paste some of the output that confuses you into the question? $\endgroup$ – gung Jul 18 '12 at 23:14
  • $\begingroup$ Dear @gung: I edited the question a little bit based on your comment. Regarding your questions: 1) I get the same estimated variance ...==> same SE's per each Beta (better to say SE^2)! 2) Yes, my Beta estimates are larger under exchangeable...3) I use SAS 9.3, Proc GENMOD $\endgroup$ – Sam Jul 18 '12 at 23:19
  • $\begingroup$ Is the response discrete or continuous? $\endgroup$ – Macro Jul 18 '12 at 23:42
  • $\begingroup$ @marco: Continuous $\endgroup$ – Sam Jul 18 '12 at 23:52
  • $\begingroup$ @gung, I've edited my question again. $\endgroup$ – Sam Jul 19 '12 at 0:24
9
$\begingroup$

One way is to run a linear regression and run the robust variance estimator on top of that to guard against getting biased estimates.

An important point here is that having pockets of correlated data does not bias your estimates in a linear model - it results in having inflated standard errors. In a non-linear model (e.g. logistic regression), you can get biased estimates, since the population average effect is, in general, different from the individual specific effect, which is not the case with a linear model. More information on this distinction is in this answer

Can we take the clustering effect into account with the sandwich estimator?

From the title, I assume you're talking about using Huber White sandwich standard errors for your confidence intervals and $p$-values. These do impose a diagonal covariance matrix but are robust to the diagonal entries possibly being different - for that reason they were originally used when there is possible heteroskedasticity in your errors, which means that the error variance in non-constant. But, using a slight modification of the Huber-White standard errors where the "meat" of the sandwich is replaced with an empirical estimate of the covariance matrix within a cluster (still called Huber-White standard errors) provides inference that is robust to non-independence within a cluster (but not between clusters!) - this modification is described pretty clearly in a 2006 paper in The American Statistician called On The So-Called “Huber Sandwich Estimator” and “Robust Standard Errors” by David Freedman.

This procedure robust to non-independence within a cluster in the sense that they will still give you asymptotically unbiased inference (i.e. the confidence levels and $p$-values will be right) even if there is correlation within a cluster. I suspect this is what your code labeled 'Empirical Estimator' code is doing.

I've fitted two separate GEE models one with exchangeable varcov matrix and the other one with the robust variance estimator (also known as Huber-White, Sandwich Estimator, or empirical variance estimator). The point is under both models I get the same estimated variance per each covariate, but my GEE exchangeable estimates leads to much larger beta estimates that are also statistically significant whereas similar beta covariates are not significant in GEE with robust varcov estimator. I wonder why it happens?

In general, the GEE model solves the equation

$$ \sum_{i=1}^{n} \frac{ \partial \mu_i }{ \partial \beta } V(\alpha)^{-1} (y_i - \mu_i) = 0$$

as a function of the regression coefficients, $\beta$, where $\mu_i = x_i \beta$ is the expected values of the cluster $i$ response, $y_i$, given the predictors $x_i$ under the specified model. $V(\alpha)^{-1}$ is the "working" covariance matrix of the of the elements of cluster $i$. (note that $\mu_i=x_i \beta$ because we're dealing with a linear model but GEEs can more generally use a link function so that $\mu_i=g(x_i \beta))$

A key point here is that when you change the working covariance, you change the estimating equation, therefore the $\beta$ that solves it will be different. For example, if $V$ was $\sigma^2$ down the diagonal and $0$ off the diagonal and $\mu_i = x_i \beta$ as it does here, then the GEE estimator is the least squares estimator, which will not solve that equation in the exchangeable case. So it is no surprise that you're getting different parameter estimates. It may be a coincidence that you're getting the same standard errors.

In your situation, I'd suggest reporting the results that used the exchangeable covariance matrix since. While GEE-based inference in consistent even when you're misspecified the correlation structure, it is known that GEE estimators are more efficient when you use a more appropriate covariance structure and, if you have evidence that there are large intra-class correlations within a school, then the exchangeable correlation will probably provide a much closer approximation to the true association structure.

$\endgroup$
  • $\begingroup$ Thank you very much for your complete answer -very helpful. Suppose there is an exchangeable correlation between students in each school. Then we would expect to see a big difference between fitting a GEE with exchangeable varcov matrix v. a independence GEE. How about the other way, if there is no exchangeable correlation (suppose that students are independent), then we would expect both GEE exchangeable and GEE independent lead to the same result? With my actual data,GEE independent leads to an insignificant estimate for my covariate of interest while GEE exchangeable is signif. $\endgroup$ – Sam Jul 19 '12 at 1:34
  • $\begingroup$ @Sepehr, I'm not sure, since I don't think the estimates would always behave in a predictable way when you change the correlation structure (I could be wrong here). It's not clear to me from looking at the estimating equation that, for example, that if there were large exchangeable correlations that assuming independence would bias you in a particular direction. I do think that if there truly was independence, that you'd get pretty similar estimates from an Indep vs. Exchangeable correlation structure choice, at least in the linear case. $\endgroup$ – Macro Jul 19 '12 at 1:53
  • $\begingroup$ @Macro, +1 (as usual). Note that "since" is just hanging out at the end of the 1st sentence in the last paragraph. I don't know if you meant to add more to that sentence, or if it's just superfluous. Sepehr, note that specifying a covariance structure for the GEE just puts certain constraints on the variance-covariance matrix that gets used; it still estimates (covariance) parameters. W/ exchangeable, you tell it to fit $p(p+1)/2$ cov params (which eats a lot of df), but they can all be estimated as an identical value, whereas independent constrains it to estimate 1 cov param. $\endgroup$ – gung Jul 19 '12 at 2:45
  • 1
    $\begingroup$ (+1) Nice answer (and awesome new gravatar :-) $\endgroup$ – chl Jul 19 '12 at 22:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.