One-way ANOVA on data with two factors

Question

Assume there is no interaction effect in the data. However, it was gathered such that it had two factors. For example, let's say you look at running times of people from 3 different age categories. Further you make sure that in every age category, there is an equal number of male and females.

I am aware that this set-up asks for a two-way anova test. However, I wonder why is the p-value for age, so much larger in an one-way anova experiment, where gender is ignored. Is the test just less powerful if not all the factors are included? Or are any assumptions about the one-way anova design set-up violated? (The data samples are equal for each category and are also normally distributed.)

Answer 1

This difference is likely because you are estimating more effects in the 2-way ANOVA instance than in the one-way ANOVA instance. The equation for F is $$ F = \frac{\frac{SS_b}{df_b}}{\frac{SS_e}{df_e}} $$ Where $SS_b$ is the total variability between groups of interest and $SS_e$ is the leftover variability not explained by the differences you are interested in. $df_b$ is defined as the number of parameters that you are testing at a given time. In order to test all of the variability in a 3 level predictor, you would actually need to test two sets of differences, so $df_b$ = 2 in the case of testing the age category variable (in either the one-way or two way ANOVA model). We can think of this equation as a comparison of the amount of error each parameter explains in your actual model (i.e. overall, how much error are you explaining per degree of freedom lost) compared to the amount of error that would be explained by a typical remaining parameter on average ($\frac{SS_e}{df_e}$). In the one-way ANOVA example, you might get output like this:

> summary(aov(time~ageCat))
            Df Sum Sq Mean Sq F value   Pr(>F)    
ageCat       2  16.44   8.219   12.41 0.000151 ***
Residuals   27  17.88   0.662                     

We can see that the $SS_b$ for ageCat = 12.89 and that the $df_b$ = 2, this will be the same for both ANOVA models. What changes in the bottom part of the F equation. We see that the $df_w$ = 27 (this is calculated as # observations - # groups being compared, in this example 30 - 3 = 27). When I run the two-way ANOVA with the same data (including now gender balanced across levels of age) I get this output:

              Df Sum Sq Mean Sq F value   Pr(>F)    
ageCat         2 16.437   8.219  12.155 0.000226 ***
gender         1  0.640   0.640   0.947 0.340201    
ageCat:gender  2  1.015   0.508   0.751 0.482831    
Residuals     24 16.228   0.676    

Here, we can see that I have the same $SS_b$ and the same $df_b$ which makes sense because the total differences in time due to age have not changed. However, I'm including a lot more stuff in my model (i.e. I have more groups). Now instead of losing 3 df in my $df_e$ I lose 6 because I am now estimating differences between 6 different groups instead of 3. If estimating those additional differences was helpful in explaining the outcome, I would see that my $SS_e$ would also be smaller, but since gender and the interaction don't really explain any variability in the outcome(we can see this because the $SS_b$ for those predictors are really small), the bottom part of my F fraction is going to be a bigger number. If the bottom of the F is larger, the overall F will be smaller, and a smaller F = a larger p-value. My guess, in your case is that age doesn't really explain any of the variability in your outcome. Does that help?