5
$\begingroup$

I have data of operation success of many doctors. I estimated a regression using Stata with fixed effects on the individual doctors.

I first ran the regression using robust option. The resulted t value of estimates of individual doctors ranges from 2.17 to 6.14. Then I re-ran it using the vce(cluster doctor) option. I expected the standard errors would become large. However, I indeed got smaller std. error -- much smaller, for example, 1.04e-14. It's just too good to be true. Why is that? Any possible reason?

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ provide your code and output $\endgroup$
    – StasK
    Commented May 16, 2014 at 6:33

2 Answers 2

Reset to default
12
$\begingroup$

You have way overcorrected the individual doctor effects twice using methods that simply do not work together.

If your model is regress outcome i.doctor, vce(cluster doctor), then Stata should have complained that you've exhausted your degrees of freedom. xtreg may not be as smart, and may miss a perfect determination of the fixed effects. These 1e-14 standard errors should have been identically zero, and they are non-zero in practice due to rounding somewhere in the guts of fixed effect estimation. What happens here is this:

  1. cluster variance estimation works by summing up the cluster contributions, over clusters. However,
  2. by specifying doctors as fixed effects, you force the residuals for a given doctor to sum up to 0.
  3. regress knows how to determine this at the level of algebra. xtreg may not know enough of computational linear algebra to do this, though, and simply sums up the (numerical) zero contributions to produce the implausibly small standard errors that you see here.
Improve this answer
$\endgroup$
2
  • $\begingroup$ Thank both StasK and Dimitriy for the great help. To clarify, I did have other variables in my regression, i.e., " regress outcome weekly_dummies, regime, i.doctor, vce(cluster doctor)" in which weekly_dummies are dummies for weeks, and regime is my main variable of interest (There is a policy switch in my data). I didn't get the complain from regress as StasK advised. Furthermore, could you please clarify how to remedy my issue? I am still confused especially I used the similar specification on other data and there was no such an issue. Is this data specific? Many thanks again. $\endgroup$
    – user44968
    Commented May 17, 2014 at 16:47
  • $\begingroup$ @StasK, could we then use i.hh and robust s.e., instead of culster(hh)? (I am asking this to Dimitriy as well, above--I am trying my chances with this old but super-useful post) $\endgroup$
    – Fuca26
    Commented May 6, 2021 at 20:58
8
$\begingroup$

If I understand your problem, this can happen when the intra-cluster correlations are negative. See Stata FAQ for the therapist version with some intuition.

Edit:

I think Stas is right about the deeper issue. I was too hasty. Here's my attempt to replicate this with a dataset of pharmacy visits by 27,766 Vietnamese villagers that are nested in 5,740 households in 194 villages (data are from Cameron and Trivedi). I could not find a public dataset where the clustered errors were smaller, but I think this illustrates the main point. I will treat pharmacy visits as continuous, though they clearly are not.

First, we set up the data:

. use "http://cameron.econ.ucdavis.edu/mmabook/vietnam_ex2.dta", clear

. egen hh=group(lnhhinc)
(1 missing value generated)

. bys hh: gen person = _n

. xtset hh person
       panel variable:  hh (unbalanced)
        time variable:  person, 1 to 19
                delta:  1 unit

. xtdes

      hh:  1, 2, ..., 5740                                   n =       5740
  person:  1, 2, ..., 19                                     T =         19
           Delta(person) = 1 unit
           Span(person)  = 19 periods
           (hh*person uniquely identifies each observation)

Distribution of T_i:   min      5%     25%       50%       75%     95%     max
                         1       2       4         5         6       8      19

(snip)

Now for the FE regression of visits on days sick:

. xtreg PHARVIS ILLDAYS, fe

Fixed-effects (within) regression               Number of obs      =     27765
Group variable: hh                              Number of groups   =      5740

R-sq:  within  = 0.1145                         Obs per group: min =         1
       between = 0.1390                                        avg =       4.8
       overall = 0.1257                                        max =        19

                                                F(1,22024)         =   2848.23
corr(u_i, Xb)  = 0.0465                         Prob > F           =    0.0000

------------------------------------------------------------------------------
     PHARVIS |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
     ILLDAYS |   .0788618   .0014777    53.37   0.000     .0759654    .0817581
       _cons |   .2906284   .0077221    37.64   0.000     .2754925    .3057643
-------------+----------------------------------------------------------------
     sigma_u |  .85814688
     sigma_e |   1.085808
         rho |  .38447214   (fraction of variance due to u_i)
------------------------------------------------------------------------------
F test that all u_i=0:     F(5739, 22024) =     2.35         Prob > F = 0.0000

Clustering on the panel variable inflates the errors:

. xtreg PHARVIS ILLDAYS, fe vce(cluster hh)

Fixed-effects (within) regression               Number of obs      =     27765
Group variable: hh                              Number of groups   =      5740

R-sq:  within  = 0.1145                         Obs per group: min =         1
       between = 0.1390                                        avg =       4.8
       overall = 0.1257                                        max =        19

                                                F(1,5739)          =    464.54
corr(u_i, Xb)  = 0.0465                         Prob > F           =    0.0000

                                  (Std. Err. adjusted for 5740 clusters in hh)
------------------------------------------------------------------------------
             |               Robust
     PHARVIS |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
     ILLDAYS |   .0788618   .0036589    21.55   0.000     .0716889    .0860346
       _cons |   .2906284   .0102597    28.33   0.000     .2705154    .3107413
-------------+----------------------------------------------------------------
     sigma_u |  .85814688
     sigma_e |   1.085808
         rho |  .38447214   (fraction of variance due to u_i)
------------------------------------------------------------------------------

Now I try this a non-panel approach. I am using areg since Stata won't let me put in ~6K dummies.

. areg PHARVIS ILLDAYS, absorb(hh) vce(cluster hh)

Linear regression, absorbing indicators           Number of obs   =      27765
                                                  F(   1,   5739) =     368.52
                                                  Prob > F        =     0.0000
                                                  R-squared       =     0.4579
                                                  Adj R-squared   =     0.3166
                                                  Root MSE        =     1.0858

                                  (Std. Err. adjusted for 5740 clusters in hh)
------------------------------------------------------------------------------
             |               Robust
     PHARVIS |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
     ILLDAYS |   .0788618   .0041081    19.20   0.000     .0708084    .0869151
       _cons |   .2906284   .0115192    25.23   0.000     .2680464    .3132103
-------------+----------------------------------------------------------------
          hh |   absorbed                                    (5740 categories)

Unfortunately, areg obscures the thing you are interested in. If you use regress and limit the sample so the number of HHs is reasonable, you will get the tiny standard errors for clusters with only 1 villager. This makes sense since the residual for such observations will be exactly zero. Here's an example:

. reg PHARVIS ILLDAYS i.hh if inrange(hh,1,100), cluster(hh)

Linear regression                                      Number of obs =     219
                                                       F(  0,    99) =       .
                                                       Prob > F      =       .
                                                       R-squared     =  0.6473
                                                       Root MSE      =  .88177

                                   (Std. Err. adjusted for 100 clusters in hh)
------------------------------------------------------------------------------
             |               Robust
     PHARVIS |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
     ILLDAYS |   .0518095   .0314707     1.65   0.103    -.0106352    .1142542
             |
          hh |
          2  |         -1   1.84e-14 -5.4e+13   0.000           -1          -1
          3  |   .2590475   .1573536     1.65   0.103    -.0531762    .5712712
          4  |   .4662855   .2832365     1.65   0.103    -.0957171    1.028288
          5  |   2.129524   .0786768    27.07   0.000     1.973412    2.285636
          6  |          1   1.84e-14  5.4e+13   0.000            1           1
          7  |   -.585524   .2517657    -2.33   0.022    -1.085082   -.0859662
        (snip)....
        100  |  -.8359366   .0996573    -8.39   0.000    -1.033678   -.6381949
             |
       _cons |    .481905   .3147072     1.53   0.129    -.1425423    1.106352
------------------------------------------------------------------------------

Now I will cluster on the village, which inflates them some, as is expected, but still OK:

. reg PHARVIS ILLDAYS i.commune, cluster(commune)

Linear regression                                      Number of obs =   27765
                                                       F(  0,   193) =       .
                                                       Prob > F      =       .
                                                       R-squared     =  0.1814
                                                       Root MSE      =  1.1925

                              (Std. Err. adjusted for 194 clusters in commune)
------------------------------------------------------------------------------
             |               Robust
     PHARVIS |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
     ILLDAYS |   .0840634   .0056375    14.91   0.000     .0729444    .0951823
             |
     commune |
          2  |  -.1885549    .012027   -15.68   0.000    -.2122761   -.1648337
        (snip) ....
        191  |   .4646775   .0014571   318.91   0.000     .4618037    .4675514
        192  |  -.0020317   .0065782    -0.31   0.758    -.0150061    .0109427
        193  |  -.2444578   .0115522   -21.16   0.000    -.2672426   -.2216731
        194  |   .1917803   .0002288   838.33   0.000     .1913291    .1922315
             |
       _cons |   .4371527   .0200739    21.78   0.000     .3975602    .4767452
------------------------------------------------------------------------------

If I drop all other regressors and estimate something like Stas suggests, I get the zero standard errors on the commune dummies:

. reg PHARVIS i.commune, cluster(commune)

Linear regression                                      Number of obs =   27765
                                                       F(  0,   193) =       .
                                                       Prob > F      =       .
                                                       R-squared     =  0.0656
                                                       Root MSE      =   1.274

                              (Std. Err. adjusted for 194 clusters in commune)
------------------------------------------------------------------------------
             |               Robust
     PHARVIS |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
     commune |
          2  |  -.0092138   1.72e-14 -5.4e+11   0.000    -.0092138   -.0092138
          3  |  -.2910319   1.72e-14 -1.7e+13   0.000    -.2910319   -.2910319
          4  |  -.3957457   1.72e-14 -2.3e+13   0.000    -.3957457   -.3957457
          5  |  -.4244865   1.72e-14 -2.5e+13   0.000    -.4244865   -.4244865
        (snip) ....
        191  |   .4864051   1.72e-14  2.8e+13   0.000     .4864051    .4864051
        192  |  -.1001229   1.72e-14 -5.8e+12   0.000    -.1001229   -.1001229
        193  |   -.416719   1.72e-14 -2.4e+13   0.000     -.416719    -.416719
        194  |    .188369   1.72e-14  1.1e+13   0.000      .188369     .188369
             |
       _cons |   .7364865   1.72e-14  4.3e+13   0.000     .7364865    .7364865
------------------------------------------------------------------------------
Improve this answer
$\endgroup$
6
  • $\begingroup$ With negative ICCs, the standard errors can get a little bit smaller, but not numerically zero. $\endgroup$
    – StasK
    Commented May 16, 2014 at 6:34
  • $\begingroup$ I don't see why this answer was down voted. Obviously some respectable amount of work went into it. $\endgroup$
    – Andy
    Commented May 16, 2014 at 23:31
  • $\begingroup$ Dimitriy, thank you so much for the replicate. Could you please explain a bit on what you mean by "This makes sense since the residual for such observations will be exactly zero." in "reg PHARVIS ILLDAYS i.hh if inrange(hh,1,100), cluster(hh)" $\endgroup$
    – user44968
    Commented May 17, 2014 at 16:59
  • $\begingroup$ The hh dummy will be equal to the error for that singleton observation. You can see this clearly if you run a regression of y on x and a dummy for observation 7. The prediction will be equal to the actual. $\endgroup$
    – dimitriy
    Commented May 17, 2014 at 17:39
  • 2
    $\begingroup$ @Fuca26 That may fix one problem, but if there is some correlation in the errors for the same observation on different days, the robust SEs will be wrong. This is usually the case in social science settings. Moreover, the singleton dummy issue will persist. $\endgroup$
    – dimitriy
    Commented May 6, 2021 at 21:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.