2
$\begingroup$

I have a question about what the difference is in how Stata and R compute ANOVAs. I have run exactly the same ANOVA in both softwares, but curiously get a different F-statistics for one of the predictors. I´m not too familiar with Stata, but as far as I understood it, I do a Type 2 SS ANOVA for both.

To understand my output, this is my model:
Outcome variable is a continuous variable called vertrauen (=trust)
predictor 1 is a 2-level factor called trustee in R and Goodguy in Stata
predictor 2 is also a 2 level factor called Group in R and uw in Stata.

This is the R output:

 
>m2-lm(vertrauen~trustee*Group,data=RTG.UWD.short.50)
> Anova(m2,type="2")
>Anova Table (Type II tests)

>Response: vertrauen
>              Sum Sq Df F value    Pr(>F)      
>trustee       2.4928  1 24.5497    1.367e-05 ***  
>Group         0.0030  1  0.0292    0.8651      
>trustee:Group 0.1137  1  1.1200    0.2963      
>Residuals     4.0617 40                        
>  
>Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1  
>

This is the Stata output:

. anova vertrauen uw Goodguy uw#Goodguy

                         Number of obs =         44    R-squared     =  0.3912
                         Root MSE      =    .318658    Adj R-squared =  0.3455

                  Source | Partial SS         df         MS        F    Prob>F
              -----------+----------------------------------------------------
                   Model |  2.6095358          3   .86984526      8.57  0.0002
                         |
                      uw |  .00296733          1   .00296733      0.03  0.8651
                 Goodguy |  1.2981586          1   1.2981586     12.78  0.0009
              uw#Goodguy |  .11373073          1   .11373073      1.12  0.2963
                         |
                Residual |  4.0617062         40   .10154266  
              -----------+----------------------------------------------------
                   Total |   6.671242         43   .15514516  

As you can see, the F-statistics for the Group (UW) main effect and for the Group (UW) x trustee (Goodguy) interaction are the same, but for the trustee (Goodguy) main effect they differ. In R it´s almost twice as high as in Stata. I tried to change the order of the predictor and the reference levels, but that didn´t change my R output.

Does anyone know what causes the difference in the F-statistic here? I´m really puzzled about it. I expected it to be the same.

Here is the Stata output without the interaction:

. anova vertrauen uw Goodguy

                         Number of obs =         44    R-squared     =  0.3741
                         Root MSE      =    .319124    Adj R-squared =  0.3436

                  Source | Partial SS         df         MS        F    Prob>F
              -----------+----------------------------------------------------
                   Model |   2.495805          2   1.2479025     12.25  0.0001
                         |
                      uw |  .00296733          1   .00296733      0.03  0.8653
                 Goodguy |  2.4928377          1   2.4928377     24.48  0.0000
                         |
                Residual |   4.175437         41   .10183993  
              -----------+----------------------------------------------------
                   Total |   6.671242         43   .15514516  

And here is the R output without the interaction:

> m2.4-lm(vertrauen~trustee+Group,data=RTG.UWD.short.50)
> Anova(m2.4)
Anova Table (Type II tests)

Response: vertrauen
          Sum Sq Df F value    Pr(>F)    
trustee   2.4928  1 24.4780 1.328e-05 ***
Group     0.0030  1  0.0291    0.8653    
Residuals 4.1754 41                      
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
> 

It´s the same, thus it has to do something with how the two softwares incorporate the interaction term.

I also tried to manually compute the interaction term and found something interesting:

Here is the R output:

RTG.UWD.short.50$interaction-as.numeric(RTG.UWD.short.50$trustee)*as.numeric(RTG.UWD.short.50$Group)
> m2.7 Anova(m2.7)
Anova Table (Type II tests)

Response: vertrauen
            Sum Sq Df F value    Pr(>F)    
trustee     1.2982  1 12.7844 0.0009316 ***
Group       0.0030  1  0.0292 0.8651282    
interaction 0.1137  1  1.1200 0.2962617    
Residuals   4.0617 40                      
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
> 

And here is the Stata output:

. gen interaction=uw*Goodguy

. anova vertrauen uw Goodguy interaction

                         Number of obs =         44    R-squared     =  0.3912
                         Root MSE      =    .318658    Adj R-squared =  0.3455

                  Source | Partial SS         df         MS        F    Prob>F
             ------------+----------------------------------------------------
                   Model |  2.6095358          3   .86984526      8.57  0.0002
                         |
                      uw |   .0399785          1    .0399785      0.39  0.5339
                 Goodguy |  2.3984067          1   2.3984067     23.62  0.0000
             interaction |  .11373073          1   .11373073      1.12  0.2963
                         |
                Residual |  4.0617062         40   .10154266  
             ------------+----------------------------------------------------
                   Total |   6.671242         43   .15514516

Thus it seems that there is a difference in how R/ Stata computes the interactions. The R output of the manually computed interaction matches the automatically computed interaction output in Stata.

And finally the descriptives from R:

> describe(RTG.UWD.short.50$vertrauen)
RTG.UWD.short.50$vertrauen 
      n missing  unique    Info    Mean   
     44       0      43       1  0.5046
> describe(RTG.UWD.short.50$Group)
RTG.UWD.short.50$Group 
      n missing  unique 
     44       0       2 

1 (34, 77%), 2 (10, 23%) 
> describe(RTG.UWD.short.50$trustee)
RTG.UWD.short.50$trustee 
      n missing  unique 
     44       0       2 

bad (22, 50%), good (22, 50%) 

and from Stata:

. sum vertrauen uw Goodguy

    Variable |        Obs        Mean    Std. Dev.       Min        Max
-------------+---------------------------------------------------------
   vertrauen |         44    .5045969    .3938847    .000998          1
          uw |         44    .2272727    .4239151          0          1
     Goodguy |         44          .5    .5057805          0          1
$\endgroup$
  • 4
    $\begingroup$ There is no equivalent of if vertrauen>-.0001 in your R syntax so far as I can see. Show us the results of summarize uw Goodguy in Stata. Can you get the same results if you omit the interaction term in each case? $\endgroup$ – Nick Cox May 11 '16 at 16:48
  • $\begingroup$ Thanks for looking into this! Missing values of the vertrauen variable were coded as -.0001 in Stata, but I excluded them so that the Stata dataset matches exactly the R dataset. $\endgroup$ – LisaR May 12 '16 at 15:00
  • $\begingroup$ I included the outputs without the interaction term and they are identical. However if I manually compute the interaction terms, the R output matches the automatically computed interaction term output from Stata. But if I manually compute the interaction term in Stata, the output doesn´t match the R output. Any idea why that is? $\endgroup$ – LisaR May 12 '16 at 15:08
4
$\begingroup$

update: I´ve crossposted the question at statalist.org and got an answer there: http://www.statalist.org/forums/forum/general-stata-discussion/general/1348073-f-test-differences-stata-and-r

basically the problem was that the contrast settings differed between R and Stata & I did not compute the same SS type.

By default Stata computes type 3 SS, but I specified type 2 SS in R. But when computing type 3 SS in R, you should NOT use the default contrasts (contr.treatment), but instead use some orthogonal contrast (like contr.sum), see this link:
http://www.mail-archive.com/r-help@s.../msg69781.html

Thus when I did the ANOVA with type 3 SS and contr.sum, I got the same output as in Stata where I didn´t specify anything.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.