8
$\begingroup$

I have a question on how a statistician would normally interpret an anova output. Say I have anova output from R.

> summary(fitted_data)

Call:
lm(formula = V1 ~ V2)

Residuals:
     Min       1Q   Median       3Q      Max 
-2.74004 -0.33827  0.04062  0.44064  1.22737 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)  2.11405    0.32089   6.588  1.3e-09 ***
V2           0.03883    0.01277   3.040  0.00292 ** 
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 

Residual standard error: 0.6231 on 118 degrees of freedom
Multiple R-squared: 0.07262,    Adjusted R-squared: 0.06476 
F-statistic:  9.24 on 1 and 118 DF,  p-value: 0.002917 

> anova(fit)
Analysis of Variance Table

Response: V1
           Df Sum Sq Mean Sq F value   Pr(>F)   
V2          1  3.588  3.5878  9.2402 0.002917 **
Residuals 118 45.818  0.3883                    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 

From the above, I guess the most important value is Pr(>F), right? So this Pr, is less than 0.05 (95% level). How should my "explain" this? Do I explain it in "association", ie, V2 and V1 are associated (or not) ? or in terms of "significance"? I always felt that I couldn't understand when people say "This value is significant....". So what is "significant"? Is there a more intuitive form of explanation? like "I am 95% confident that ...." .

Also, is the Pr value the only important piece of information? or can i also look at residuals and the rest of the output to "explain" the result? thanks

$\endgroup$
2
$\begingroup$

From the above, i guess the most important value is Pr(>F), right?

Not to me. The idea that the size of the p-value is the most important thing in an ANOVA is pervasive but I think almost entirely misguided. For a start the p-value is a random quantity (moreso when the null is true, when it is uniformly distributed between 0 and 1). As such a lower p-value may not be particularly informative in any case, but even beyond the issue of the size of the p-value things like effect sizes are generally much more important.

You may like to read around a bit

Cohen, J. (1990). Things I have learned (so far), American Psychologist 45, 1304-1312.

Cohen, J. (1994). The earth is round (p < .05). American Psychologist, 49, 997-1003.

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1119478/

http://www.biostat.jhsph.edu/~cfrangak/cominte/goodmanvalues.pdf

http://en.wikipedia.org/wiki/Statistical_hypothesis_testing#Ongoing_Controversy

--

I didn't really address interpreting the output when a p-value is below $\alpha$. Without saying exactly what hypothesis is being considered, mentioning "significance" seems pointless. In that sense, then it would be preferable to mention the conclusion that results from the rejection of the null.

In the case you present, it's hard to interpret without context (I don't even know if V2 is categorical or continuous), but if V2 was continuous I might say something about concluding there's an association between V1 and V2. If V2 was categorical (0-1), I might say something about differences in mean V1 for the two categories, and so on.

Now some things NOT to say:

is less than 0.05 (95% level)

Never call p<0.05 "significant at the 95% level". That's wrong. Nor indeed should you call it 95% anything else.

like "I am 95% confident that ...." .

Never say that either. It's wrong.

$\endgroup$
  • $\begingroup$ hi thanks. I would look at these articles to understand p-values further. If p-values are not to be "trusted", in your opinion, which ones of the parameters should tell us more or less something about the relation between V1 and V2? R-squared? $\endgroup$ – dorothy Mar 10 '13 at 10:55
  • $\begingroup$ > If p-values are not to be "trusted" --- I wouldn't automatically say that either. I think you've gone too far the other way. It's not that they can't be 'trusted' (though if you use them wrongly they'll lead you astray sure enough). It's more that they - and hypothesis tests more generally - don't often tell you what you want them to. Effect sizes and confidence intervals are more relevant. $R^2$ isn't necessarily a very helpful measure either. $\endgroup$ – Glen_b Mar 10 '13 at 12:09
  • $\begingroup$ In short - p-values tell you something - they're just not, to my mind, generally the 'most important'. $\endgroup$ – Glen_b Mar 10 '13 at 12:44
  • $\begingroup$ oh, ok. I will look more in line line confidence interval and effect sizes to explain the results. Thanks very much. $\endgroup$ – dorothy Mar 10 '13 at 12:59
  • 1
    $\begingroup$ What is important mostly depends on what you're interested in finding out. For me it's usually the coefficients and their standard errors, and sometimes s. Sometimes the p-values are of interest to me as well. But other times I have interest in some particular part of the output. $\endgroup$ – Glen_b Mar 10 '13 at 13:39
1
$\begingroup$

The chunk of output I might look at first is this:

Multiple R-squared: 0.073,    Adjusted R-squared: 0.065
F-statistic:  9.24 on 1 and 118 DF,  p-value: 0.003

It tell you the overall model was significant (F(1,118) = 9.24, p= .003) And V1 is accounting for about 7% of the variance in V2.

The effect size (0.039) tells you that if V2 increases by 1, your model predicts V1 will increase (positive relationship) by ~ .04). The standard error on that estimate (0.013) indicates that (roughly), the 95% confidence interval of the effect is CI95 = [.0135, .064] (i.e., .039- 1.96*.013 to .039+ 1.96*.013)

The confidence interval doesn't include zero, which jives (as it must) with the p-value.

If you want anova output (as you state), you need to ask for that (not a regression summary, which is what summary() gives).

anova(), or, from the car package, Anova will give you this. Depending on your purposes, you may prefer car's Anova default output, which give the effect of each variable in your ANOVA as if it was entered last, so-called "type III sums of squares".

If we switch to a built-in example using Rs mtcars dataset of car miles per gallon and other data like weight and engine size, you can generate an Anova example:

m1 = lm(mpg ~ wt + disp + cyl+gear+am, data = mtcars);
Anova(m1)
|          | Sum Sq| Df| F value| Pr(>F) |
|:---------|------:|--:|-------:|-------:|
|wt        |  58.02|  1|    8.27|   0.01*|
|disp      |   1.53|  1|    0.22|   0.64 |
|cyl       |  57.59|  1|    8.21|   0.01*|
|gear      |   6.02|  1|    0.86|   0.36 |
|am        |   3.44|  1|    0.49|   0.49 |
|Residuals | 182.41| 26|        |        |

This suggests that vehicle weight and number of cylinders are significant factors in vehicle achieved miles per gallon. Of course all these variables are confounded in the cars dataset, showing we really need a theory of fuel consumption to make progress here.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.