1
$\begingroup$

This question is about the way to go with a very common statistical model: Continuos dependent and categorical independent variable. In my stats class I learned to use the ANOVA: check assumptions, conduct ANOVA, and if significant, do post-hoc tests. Now for the post-hoc tests I have seen several ways how people conduct them regarding p-value correction: One camp basically always applies some form of adjustment method, the second camp mostly states: "I am doing orthogonal contrasts, no adjustment needed".

anova(lm(Sepal.Length ~ Species, data = iris))
pairwise.t.test(iris$Sepal.Length, iris$Species)

Only later when I was introduced to R, I learned about linear models (LM). By now I much prefer those, easier to set up, easier to interpret, way more flexible.

summary(lm(Sepal.Length ~ Species, data = iris))

What I don't understand is why people do the extra step of conducting an ANOVA on top, when the LM already gives you everything that you need (in most cases). And you don't even have to conduct post-hoc tests, let alone think about p-value adjustments (I guess it's the orthogonal argument that strikes here again).

What is the way to go here? If one is fine with the contrasts the linear model provides you, is there a necessity to still conduct an ANOVA, or would you just interpret the parameter of the LM and leave it here?

And even if I need to compare every group vs every other group (or any other non-orthogonal contrasts) and have a more complicated model like a linear mixed model, can't I just estimate the model and conduct a general linear hypotheses test or a least square means test?

$\endgroup$
1
$\begingroup$

There are two different hypothesis being tested, when you do summary() :

Coefficients:
                  Estimate Std. Error t value Pr(>|t|)    
(Intercept)         5.0060     0.0728  68.762  < 2e-16 ***
Speciesversicolor   0.9300     0.1030   9.033 8.77e-16 ***
Speciesvirginica    1.5820     0.1030  15.366  < 2e-16 ***

You are testing the coefficients versicolor = 0 and virginia = 0 separately. If you look at the anova:

anova(lm(Sepal.Length ~ Species, data = iris))
Analysis of Variance Table

Response: Sepal.Length
           Df Sum Sq Mean Sq F value    Pr(>F)    
Species     2 63.212  31.606  119.26 < 2.2e-16 ***
Residuals 147 38.956   0.265  

You are looking at the variance collectively explained by Species (which includes coefficient versicolor and virginica). This is equivalent to testing the hypothesis versicolor = virginica = 0.

It depends on what is the reason behind testing. You are right, if we are good with the contrast, it's more informative to know the effect of each factor. And you can conduct post-hoc comparisons etc.

However the anova isn't just about p-values, we also want know much of the variation in the data is explained by the different factors. In this example you can see that the Species explains about 63.212/(63.212+38.956) = 0.61 of the variance, which is a lot. You don't get this from the p-value or the coefficients.

There are also instances where you want to learn the collective importance of the categorical variable as a whole. One loose example is measuring the response to a treatment over many timepoints, each modeled as a factor. We might be more interested in the effect of treatment, but it's good to check that the time point has an effect.

Other uses of anova also might include, testing the difference between nested models, you can check out Andrew Gelman's book, there's a chapter dedicated to anova.

$\endgroup$
3
  • $\begingroup$ well yes, the linear model does not test for a main effect. However, the R^2 of the linear regression model answers me the same question, as the explained variance as you calculated it. Though this is only possible because it has only one independent variable. It only represents the model as a whole, not a single predictor. I would not agree with your treatment over time example, I think a linear model provides a more detailed answer to interpret. It can indicate an effect regardless of time, or an interaction, or both. With the nested model comparison i totally agree. $\endgroup$
    – jde
    Apr 28 '20 at 20:30
  • $\begingroup$ i am really bowled over by the amount of statistical know-how for someone who "was introduced to R, I learned about linear models". So what I am supposed to do? delete my answer lol.. $\endgroup$
    – StupidWolf
    Apr 28 '20 at 20:42
  • 1
    $\begingroup$ No, not at all! I am thankful for you answer, it allows me to think again on your points and what my position was/is. I am certain that my question also wont have a true/false answer, but it should encourage a discussion. $\endgroup$
    – jde
    Apr 28 '20 at 20:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.