I'm currently working on some experimental data. Subjects are randomly assigned to one of two treatments. For each treatment I ran three sessions with 20 subjects each. In each session, participants were asked to make a sequence of decisions.
I estimated the effect of the treatment with a model that includes individual random effects. Afterwards, I estimated the cluster robust covariance matrix (CRCME): $$\hat{V}_{CRCME}=(X^{'}X)^{-1}\Bigg(\sum_{s=1}^{S}{X_{s}^{'}\hat{\epsilon}_{s}\hat{\epsilon}_{s}^{'}X_{s}}\Bigg)(X^{'}X)^{-1},$$ where $s$ defines the cluster. This procedure provides a consistent estimator of the covariance matrix which can typically be biased when the number of cluster is little. In my case I have only 6 clusters, so I decided to try the residual correction proposed by Bell and McCaffrey (2002). Instead of the OLS residuals $\hat{\epsilon}_{s}$, they propose $$\tilde{e}_{s}=\sqrt{(S-1)/S}[I_{N_{s}}-H_{ss}]^{-1}\hat{\epsilon}_{s},$$ where $H_{ss}=X_{s}(X^{'}X)^{-1}X_{s}^{'}$. Given that I estimated a random effects model, following McCaffrey, Bell and Botts (2002), I applied this correction on the transformed data: $X^{*}=W^{1/2}X$ and $\hat{\epsilon}^{*}=W^{1/2}\hat{\epsilon}$ where $W^{1/2}$ is the standard weight matrix used to estimate individual random effects models.
The standard errors I obtain with Bell and McCaffrey's correction are bigger than the cluster robust ones, except for the treatment variable and the intercept. Should I worry about that? The treatment variable is the main variable of interest. Do you have any idea about the possible reasons?
References:
- Bell, R.M., and D.F. McCaffrey (2002), "Bias Reduction in Standard Errors for Linear Regression with Multi-Stage Samples," Survey Methodology, 169-179.
- McCaffrey, D.F., Bell, R.M., and C.H. Botts (2001),
"Generalizations of bias Reduced Linearization," Proceedings of
the Survey Research Methods Section, American Statistical
Association
