# Justification for feature selection by removing predictors with near zero variance

I have a large number of variables that I'm trying to reduce, and I've stumbled on Kuhn's (2008) suggestion to eliminate variables with zero or near-zero variance:

[Near-zero variance means that the] fraction of unique values over the sample size is low (say 10%) [...] [and the] ratio of the frequency of the most prevalent value to the frequency of the second most prevalent value is large (say around 20). If both of these criteria are true and the model in question is susceptible to this type of predictor, it may be advantageous to remove the variable from the model.

-- Kuhn, M., & Johnson, K. (2013). Applied predictive modeling, New York, NY: Springer.

This makes sense to me, it's been used in other studies, and it would do exactly what I'm hoping for with my data. However, as far as I can tell, Kuhn doesn't provide any justification (either theoretical or empirical) for using this technique, and I can't find any other literature that supports this.

Does anyone know of other sources that demonstrate why this technique works?

• What quantitative criterion do you propose to distinguish "near-zero" from "zero" variance? Although understanding the motivation behind Kuhn's suggestion, I confess to being extremely sceptical, in light of the fact that what matters is the relationship between each explanatory variable and the response variable (after controlling for the other explanatory variables). If tiny changes in one variable consistently are associated with appreciable changes in the response, then surely you would want to keep that variable! – whuber Apr 9 '15 at 21:26
• Zero-variance won't even run in most packages/would crash the model. Removing near-zero-variance predictors would also eliminate dummy variables and other binary predictors. this has 2 refs to alternative procedures r-bloggers.com/… – katya Apr 9 '15 at 21:30
• There may be that danger, yes--but eliminating the variable altogether is the wrong way to deal with it (in your hypothetical case) because you would then be removing perhaps the most important explanatory variable you have! – whuber Apr 16 '15 at 14:50
• Could you provide a full reference & perhaps a quote or summary of the recommendation? There's probably some sense in it when building a predictive model, but it has to be understood in a context-specific way. A range of temperatures in a sample from 25ºC to 40ºC might be considered a high variance in enzyme kinetics but a low variance in metallurgy. It comes down to not filling your model up with predictors that you'd expect to have a negligible relationship to the response over the range they take in your sample. – Scortchi Aug 21 '15 at 10:05
• @Scortchi Sorry for the delay; obviously haven't been here for a while. :) Here's the info: Near-zero equals: "The fraction of unique values over the sample size is low (say 10%)" and "[t]he ratio of the frequency of the most prevalent value to the frequency of the second most prevalent value is large (say around 20). If both of these criteria are true and the model in question is susceptible to this type of predictor, it may be advantageous to remove the variable from the model." Kuhn, M., & Johnson, K. (2013). Applied predictive modeling New York, NY: Springer. – Spencer Greenhalgh Dec 10 '15 at 2:29