My study design involves a control and 2 test groups plus some covariates. Each group consists of around 20 observations. In total I look at around 1,000 variables.
I created a linear model using the
lm() function in R including 2 covariates. After that I thought I would include another covariate because doing a PCA plot earlier showed a slight effect on that covariate. However, after adding this covariate to the model 50% of the significant hits are now different. I was actually assuming that it would pretty much identical as the effect was hardly seen in the PCA.
Could it be that I have overfitted the model? Or is the effect simple just not shown in the PCA plot but is there?
I just compared the two models using
anova(lm1, lm2) and the p-value is significant which I think means that the third covariate adds significant information to the model?
lm1 <- lm(var ~ factor_of_interest + cov1 + cov2) lm2 <- lm(var ~ factor_of_interest + cov1 + cov2 + cov3) anova(lm1, lm2)