What is the effect of low variance features on machine learning models intuitively it seems that low variance features are not useful and are just noise to the model. is it important to remove the features though? i.e., does the model performance improve significantly by removing irrelevant features?
I'm particularly interested in the effect on binary classification models.
 A: 
Intuitively it seems that low variance features are not useful and are just noise to the model.

This is folklore that is false in an essential way.  There are two intuitive reasons to doubt it:


*

*The variance of a feature is not unitless, by re-expressing, say, a length in meters, millimeters, or feet, you change the variance.  Any well founded model should not care.

*The variance of a feature ignores the relationship between the feature and the response, which is the focus of supervised learning models.  While the predictor may have small variation, the relationship between the predictor and response may be very powerful within that range.


With these points in mind, it's very easy to construct examples where a small variance feature dominates a large variance feature
> set.seed(154)
> x_1 <- rnorm(100, mean = 0, sd = .01)   # Low varaince
> x_2 <- rnorm(100, mean = 0, sd =  100)  # High varaince
> 
> y <- 100*x_1 + rnorm(100) 
> 
> lm(y ~ x_1 + x_2)

Call:
lm(formula = y ~ x_1 + x_2)

Coefficients:
(Intercept)          x_1          x_2  
 -0.2394094   95.5194309    0.0007392 

Any sensible measurement of feature "relevance" must take into account the relationship between a predictor and the thing being predicted.  The internal structure of the predictors themselves can tell you only very little.
