# Why different results for the same factor once using Oneway and once Twoway ANOVA?

I am currently analysing data of eggs of laying hens that were fed with different forage. There were two forage in total, and eight group of laying hens. Four groups under forage 1 and four groups under forage 2.

I am using R and am performing some ANOVAs for different factors and dependant variables. Right now, the factor of interest is the eight groups of hens.

When I set the parameter "shell strength" (measured in N) as the dependant variable in an ANOVA test with factor groups, then I get two different p-values. With an Oneway ANOVA, I get a p-value of 0.03186 which is significant. On the countrary, with a Twoway ANOVA, I get a p-value of >0.05 which in my case is not significant.

I am quite a beginner in stats and also in R so I do not quite understand why the results differ. Maybe someone has an explanation for me? On researchgat.net, someone suggested to check the explained variance of both tests. In my case I get an explained variance of 0.69 for the Oneway and of 0.63 for the Twoway ANOVA. He then recommended to take the p-value of the test with the greater explained variance. What do you think?

Every response will be greatly appreciated! Best regards

Edit: Here is the code.

-BF stands for "egg strength", dependant variable

-Abteil stands for "group", there are 8 of them

# Oneway ANOVA
fac2 <- as.factor(sheet_all$$Abteil) # Factor group (8 levels) onewayANOVA <- aov(sheet_all$$BF ~ fac2)
summary(onewayANOVA)

# Twoway ANOVA
fac1 <- as.factor(sheet_all$$Verf) # Factor forage (2 levels) fac2 <- as.factor(sheet_all$$Abteil) # Factor group (8 levels)
twowayANOVA <- aov(sheet_all$BF ~ fac1*fac2) summary(twowayANOVA)  Here are both outputs I get. > fac2 <- as.factor(sheet_all$$Abteil) > onewayANOVA<- aov(sheet_all$$BF ~ fac2) > summary(onewayANOVA) Df Sum Sq Mean Sq F value Pr(>F) fac2 7 1109 158.50 2.221 0.0319 * Residuals 392 27978 71.37 --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 > fac1 <- as.factor(sheet_all$$Verf) > fac2 <- as.factor(sheet_all$$Abteil) > twowayANOVA <- aov(sheet_all$BF ~ fac1*fac2)
> summary(twowayANOVA)
Df Sum Sq Mean Sq F value Pr(>F)
fac1          1    365   364.8   5.111 0.0243 *
fac2          6    745   124.1   1.739 0.1107
Residuals   392  27978    71.4
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1


Dataframe that is being used:

> head(sheet_all)
# A tibble: 6 x 21
Stall Abteil Verf     LP Ei-Nb    EG    BF Dotterfarbe Flecken Eiklarhöhe
<chr>  <dbl> <chr> <dbl>   <dbl> <dbl> <dbl>       <dbl>   <dbl>      <dbl>
1 2.1.      11 K         9       1  71.7    43          11       0        9.6
2 2.1.      11 K         9       2  70.7    41          11       1        7
3 2.1.      11 K         9       3  73.5    57          11       0        9.6
4 2.1.      11 K         9       4  62.6    54          12       1        8.8
5 2.1.      11 K         9       5  72.6    47          10       0       10
6 2.1.      11 K         9       6  61.3    45          11       0        8.7
# ... with 11 more variables: Haugh_units <dbl>, Dottergewicht <dbl>,
#   Dotter_percent <dbl>, Schalengewicht <dbl>, Schalen_percent <dbl>,
#   Eiweissgewicht <dbl>, Weiss_percent <dbl>, Schalendicke1 <dbl>,
#   Schalendicke2 <dbl>, Schalendicke <dbl>, Date <dttm>

New contributor
arkadryyx is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.
• Hi, could you possibly add the code you used for your models in your question? That will be helpful – Lachlan yesterday
• @Lachlan Yes of course. Thank you! – arkadryyx yesterday
• There are a series of weird things going on here. For one, you've specified an interaction in your second model, but it isn't present in the results. Also, the residual degrees of freedom aren't changing between the models, when they really should be.. Can you run head() on your dataframe? e.g. head(sheet_all) and post the results? – Lachlan yesterday
• Yes, also for me as a beginner it surprised me that there was no interaction in the output of the two-way ANOVA. Now you can take a look at my dataframe. Hope you understand a bit of German ;-) – arkadryyx yesterday

## 1 Answer

The effect of a factor in one-way ANOVA and the principal effect of the same factor in a two-way ANOVA both try to answer the same question: whether the factor plays a significant role in the variations of the DV, regardless of the levels of any other factor or covariates. Factual support for this claim is that the numerator of the corresponding F ratios is the exact same in both cases, namely the mean square that quantifies the principal effect of your factor.

The only difference between the tests comes from the denominator of the F ratio, namely what is considered as the error in the model. In a one-way ANOVA, the residual error of the model is any variability in the DV that is not explained by the factor. In a two-way ANOVA, the residual error of the model is the variability in the DV that is not explained by any of the factors, or by their interaction. Therefore the sum of squares of the error (SSE) in a two-way ANOVA is systematically less than the SSE on a one-way ANOVA with only one of the factors, and in most cases this is enough to make the model more powerful to detect the effect of your main factor (compared to a smaller error, the same effect appears to be bigger and more significant).

Not always though, as there is a small caveat: adding a factor reduces the degree of freedom of the error. So, the resulting mean square may in fact be actually higher with two factors than it is with one (and also, the F distribution used to evaluate the significance of the model is going to be more demanding). When this happens, you can in general infer that the second factor is completely useless to try and explain the variations of the DV.

• Hi Arnaud, first thank you very much for your precisions. It made a lot of sense to me! You mentioned that the numerator for the F-ratio was the same in both cases, but I see once 158.50 (oneway) and once 124.1 (twoway). Or did I miss something? According to your last paragraph, I should then not take the p-value of the two-way ANOVA into consideration, since it is subject to changes in the degrees of freedom? Instead I should keep the p-value of the one-way ANOVA for fac2? – arkadryyx yesterday
• @arkadryyx I can see only one configuration where the numerator could change, that is when you have unbalanced data (not the same amount of subjects in all treatment groups) which would complicate everything as all would then depend on which kind of sum of squares you are using (Type I, II, III or otherwise). As for the last question in your comment, there are two approaches: either you choose a model and then stick to it no matter what, or you're in a flexible mindset and say "this model has shown us that such factor explains nothing and makes the model worse, so let's remove this factor". – Arnaud Mortier 23 hours ago
• @arkadryyx looking more closely at your table, I don't really understand how you could have a different degree of freedom for fac2 in 1-way and 2-way ANOVAs. The df of fac2 should stay the same and the df of error should decrease. – Arnaud Mortier 23 hours ago
• Indeed, I have unbalanced data. I did not mention it above because as a beginner I thought it did not matter. But it does apparently! This should explain the difference in numerator for fac2 between 1-way and 2-way ANOVAs, right? And then also the F-ratio and p-value that do not correspond with those of the 1-way ANOVA. Also thank you for the 2nd part of your comment about the different approachs regarding the models. I guess I will discuss it with my Prof and do what she suggests to do. – arkadryyx 22 hours ago
• @arkadryyx unbalanced data certainly brings a number of complications, but I can see no reason why it would make the degree of freedom of fac2 anything else than the number of levels minus 1, and neither can I see how the SSE could be the exact same in a one-way and two-way ANOVA, these are two complete mysteries in your table. – Arnaud Mortier 22 hours ago