Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 2 down vote favorite

My study design involves a control and 2 test groups plus some covariates. Each group consists of around 20 observations. In total I look at around 1,000 variables.

I created a linear model using the lm() function in R including 2 covariates. After that I thought I would include another covariate because doing a PCA plot earlier showed a slight effect on that covariate. However, after adding this covariate to the model 50% of the significant hits are now different. I was actually assuming that it would pretty much identical as the effect was hardly seen in the PCA.

Could it be that I have overfitted the model? Or is the effect simple just not shown in the PCA plot but is there?

I just compared the two models using anova(lm1, lm2) and the p-value is significant which I think means that the third covariate adds significant information to the model?

lm1 <- lm(var ~ factor_of_interest + cov1 + cov2)
lm2 <- lm(var ~ factor_of_interest + cov1 + cov2 + cov3)
anova(lm1, lm2)
share|improve this question
However, the result of anova(lm1, lm2) of course depends on which of the 1000 variables you look at. So how can I find out whether to include or exclude the 3rd covariate cov3? Best Jacky – jacky Jul 11 '12 at 9:11
Jacky, it appears your question came from SO (as user user969113) but you haven't registered your account here on CV, nor linked your two accounts. Please, see our FAQ on account registration. – chl♦ Jul 11 '12 at 17:04
What do you mean by significant hits? As this is not a nick seems to be suggesting you might be better off letting theory guide you in the choice of variables and to look for reasons why cov3 might be important. You could have cov3 be significant interms of its coefficient being different from 0 and not have it add much to the variance explained in the first PC. What varibles are you including inthe PCA? – Michael Chernick Jul 29 '12 at 12:59

1 Answer

up vote 1 down vote

From a purely statistical point of view, I think that if the cov3 adds significantly more information to your regression model, you have to mantain that covariate. Maybe it is useful for you to do a stepwise analysis, for example by the 'step' command that choose the model by computing the AIC.

However, the statistical choices should be done also considering theoretical aspects. Do theoretical assumptions about a possible role of cov3 as predictor exist?

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.