Few-clusters bias correction for cluster robust covariance matrix in random effects model

Question

I'm currently working on some experimental data. Subjects are randomly assigned to one of two treatments. For each treatment I ran three sessions with 20 subjects each. In each session, participants were asked to make a sequence of decisions.

I estimated the effect of the treatment with a model that includes individual random effects. Afterwards, I estimated the cluster robust covariance matrix (CRCME): $$\hat{V}_{CRCME}=(X^{'}X)^{-1}\Bigg(\sum_{s=1}^{S}{X_{s}^{'}\hat{\epsilon}_{s}\hat{\epsilon}_{s}^{'}X_{s}}\Bigg)(X^{'}X)^{-1},$$ where $s$ defines the cluster. This procedure provides a consistent estimator of the covariance matrix which can typically be biased when the number of cluster is little. In my case I have only 6 clusters, so I decided to try the residual correction proposed by Bell and McCaffrey (2002). Instead of the OLS residuals $\hat{\epsilon}_{s}$, they propose $$\tilde{e}_{s}=\sqrt{(S-1)/S}[I_{N_{s}}-H_{ss}]^{-1}\hat{\epsilon}_{s},$$ where $H_{ss}=X_{s}(X^{'}X)^{-1}X_{s}^{'}$. Given that I estimated a random effects model, following McCaffrey, Bell and Botts (2002), I applied this correction on the transformed data: $X^{*}=W^{1/2}X$ and $\hat{\epsilon}^{*}=W^{1/2}\hat{\epsilon}$ where $W^{1/2}$ is the standard weight matrix used to estimate individual random effects models.

The standard errors I obtain with Bell and McCaffrey's correction are bigger than the cluster robust ones, except for the treatment variable and the intercept. Should I worry about that? The treatment variable is the main variable of interest. Do you have any idea about the possible reasons?

References:

• Bell, R.M., and D.F. McCaffrey (2002), "Bias Reduction in Standard Errors for Linear Regression with Multi-Stage Samples," Survey Methodology, 169-179.
• McCaffrey, D.F., Bell, R.M., and C.H. Botts (2001), "Generalizations of bias Reduced Linearization," Proceedings of the Survey Research Methods Section, American Statistical
  Association

Answer 1

If your randomization is at individual level within clusters, then the ICC of the treatment variable may well become negative. A positive ICC means that two units within cluster are more alike to one another than two units randomly taken across the population. A negative ICC means that units within cluster are more disperse than across the population. Negative ICCs are rare to come by in real life with natural data, but purposeful assignments may drive them there when you make sure that a variable varies as much within the cluster as it does across the population -- a random assignment is one good example. So it may not be so terribly surprising that the standard errors of the treatment variable improved with clustering.

Still, with only 6 clusters, and 20 subjects, I would not trust these standard errors, and would not trust the random effects model very much, either. I don't have any specific suggestions, though; your sample sizes just seem too small across the board.