# Preprocessing: Why do we remove constant/invariant predictors?

For a recent research project we used machine-learning. In the preprocessing phase we removed 2 predictors because they contained mostly uninformative 0 values (x1 = 100%, x2 = 99%). Is it the right terminology to call them invariant?

My main justification for the exclusion is, that it reduces the degrees of freedom and therefore the risk of overfitting.

I am looking for a paper / source to justify this procedure, besides the face validity of doing so. Do you have a recommendation for such a source?

Thanks to mkt:
I found what I was looking for in this book: Kuhn, M., & Johnson, K. (2013). Applied predictive modeling. Applied Predictive Modeling (Vol. 26). New York: Springer. https://doi.org/10.1007/978-1-4614-6849-3

• Suppose $x_2$ is an indicator of gender (0=male, 1=female) in an almost predominantly male population (99% zeros) and the response is whether a subject was ever pregnant. Would you still want to drop $x_2$ from the model?? – whuber Jul 16 '19 at 13:54
• Thanks for pointing this out. I think you are right, so of cause it is important to consider the data structure and the purpose of the analysis. In our case it is not a dummy coded variable. It is the probabilty of a feature is being active. So in the case of x2 there was only one unique value of 0.01, meaning a 1% chance that this feature was active. I think it is justified to drop it. – PythonBeginner Jul 16 '19 at 14:19
• That's fine--it was exactly my intention that you conclude judgment is needed. The point is that we cannot expect to find any authority to justify throwing out any explanatory variable based on how close it comes to being constant. Indeed, any source that might make a blanket statement of that nature should be discredited in light of the example I offered. – whuber Jul 16 '19 at 14:33