# Should covariates that are not statistically significant be 'kept in' when creating a model?

I have several covariates in my calculation for a model, and not all of them are statistically significant. Should I remove those that are not?

This question discusses the phenomenon, but does not answer my question: How to interpret non-significant effect of a covariate in ANCOVA?

There is nothing in the answer to that question that suggests that non-significant covariates be taken out, though, so right now I am inclined to believe that they should stay in. Before even reading that answer, I was thinking the same since a covariate can still explain some of the variance (and thus help the model) without necessarily explaining an amount beyond some threshold (the significance threshold, which I see as not applicable to covariates).

There is another question somewhere on CV for which the answer seems to imply that covariates should be kept in regardless of significance, but it is not clear on that. (I want to link to that question, but I was not able to track it down again just now.)

So... Should covariates that do not show as statistically significant be kept in the calculation for the model? (I have edited this question to clarify that covariates are never in the model output by the calculation anyway.)

To add complication, what if the covariates are statistically significant for some subsets of the data (subsets which have to be processed separately). I would default to keeping such a covariate, otherwise either different models would have to be used or you would have a statistically significant covariate missing in one of the cases. If you also have an answer for this split case, though, please mention it.

• Generally speaking, I would say that you should keep variables that are theoretically important or that have been significant in prior studies, even though your data do not support their effect. That being said, to get a more specific answer, I think you should add a couple of lines to explain your model and its purpose (e.g. identifying risk factors, making prediction, ...). – ocram Aug 3 '13 at 18:17
• I would say it depends. Tests are just indicators. If you believe that there should be a small dependence then think about keeping in the model. If you believe as well that the dependence should not be there then leave it out. – Bene Aug 3 '13 at 18:18
• OK, so you are both saying that non-significance does not dictate a covariate being removed from consideration, so you have both actually answered my question. I should actually rephrase my question to more clearly indicate that what I am asking is whether stastistical significance of a covariate is a necessary condition for keeping it ("Does non-significance of a covariate mean it should be removed..."), and I would accept either of your comments as answers. – A.M. Aug 3 '13 at 18:28
• Before I do that, though, I would like to make sure I am using the right terminology. Originally I wrote "kept in the model", but that did not seem right because covariates never appear in the model. I settled for "kept in the calculation for the model" (and "removed from consideration"), but is there a better way of saying this? What is the right term for what the covariate is being kept in or removed from? – A.M. Aug 3 '13 at 18:30
• You would need to validate the correct performance of such selection procedures. Others have failed. – Frank Harrell Aug 3 '13 at 22:59

You have gotten several good answers already. There are reasons to keep covariates and reasons to drop covariates. Statistical significance should not be a key factor, in the vast majority of cases.

1. Covariates may be of such substantive importance that they have to be there.
2. The effect size of a covariate may be high, even if it is not significant.
3. The covariate may affect other aspects of the model.
4. The covariate may be a part of how your hypothesis was worded.

If you are in a very exploratory mode and the covariate is not important in the literature and the effect size is small and the covariate has little effect on your model and the covariate was not in your hypothesis, then you could probably delete it just for simplicity.

• A very important but often neglected situation is covered by #4 here, but I will spell it out. Often -- indeed usually -- you should want to compare your results with those of previous workers with similar data. If others found particular covariates worth including in their models, you should want to compare your results with theirs, regardless of whether your covariates achieve (conventional) significance levels. Note that cases here can vary from reporting model(s) you decide are not (especially) good to reporting model(s) you decide are good. – Nick Cox Aug 4 '13 at 9:24
• I was definitely leaning toward 'keep in' (and not making much of p-value for covariates in the first place), but your answer makes a very nice checklist (well...two) for a minority to take out. The effect size is something I had not considered, and while I did consider hypotheses I very much like that you included it, for the reasons @NickCox mentioned and simply to discouraging fishing. – A.M. Aug 4 '13 at 14:21

The long answer is "yes". There are few reasons to remove insignificant predictors and many reasons not to. As far as interpreting them you do so ignoring the $P$-value just as you might interpret other predictors: with confidence intervals for effects over interesting ranges of the predictor.

• The long answer is "yes" ! +1 and a LOL. – Peter Flom - Reinstate Monica Aug 3 '13 at 19:44
• If not p-values, what are other reasons to remove predictors? You mention interpreting confidence intervals, but it seems like an "interesting range" would be zero, which means people would interpret CIs much like p-values (inclusion or exclusion of zero). – Mark White Jun 23 '17 at 16:54
• What are reasons to remove predictors when this distorts statistical properties? Not clear on your question and the "zero". – Frank Harrell Jun 23 '17 at 18:56

One useful insight is that there is really nothing specific about a covariate statistically speaking, see e.g. Help writing covariates into regression formula. Incidentally, it might explain why there is no covariate tag. Consequently, material here and elsewhere about non-significant terms in a linear model are relevant, as are the well known critics of stepwise regression, even if ANCOVA is not explicitly mentioned.

Generally speaking, it's a bad idea to select predictors based on significance alone. If for some reason you cannot specify the model in advance, you should consider other approaches but if you planned to include them in the first place, collected data accordingly and are not facing specific problems (e.g. collinearity), just keep them.

Regarding the reasons to keep them, the objections you came up with seem sound to me. Another reason would be that removing non-significant predictors biases inferences based on the model. Yet another way to look at all this is to ask what would be gained by removing these covariates after the fact.