Add Absolute Value as a Feature In a machine learning context, does it make sense to have both a measurement and its absolute value transformation as features?
There are already ~120 features in this predictive model (an elastic net regression, which I inherited), so I think it would be good to try to reduce the dimensionality. Many of the features are simply absolute value transformations of other features, some of which are nearly always positive and others of which are more-or-less balanced around 0. I don't exactly have a statistical argument for my dissent, but intuitively I don't know what the model would be gaining by having those transformed features.
If my assumption is correct, a more formal explanation of why I should remove these transformed features would be appreciated!
 A: It is hard to say in general. It will in my opinion depend on the particular application. The absolute value transformation trivially adds nothing if the feature only have positive value but on the other hand introduces a non-linearity if the feature has both positive and negative values. Of course this non-linearity can be approximated by a flexible model but whether the model is sufficiently flexible depends. The picture displayes true observations $y_i$'s plotted as green and generated using 
$$y = a + \beta x + \lambda \lvert x \lvert + \epsilon$$
 
The blue dots are predicted values from model
$$ y = \alpha_1 + \alpha_2 x + + u$$
The red dots are predicted values from model similar to the true model and finally
the black are predicted values from a high order polynomial. 
Here is the code used to generate the plot
N <- 2000
x <- rnorm(N)
z <- abs(x)
a <- 1
b <- 0.5
e <- 0.2*rnorm(N)

# Simulate y as linear function 
y <- a + b*x - z + e

# Estimate model
model1 <- lm(y ~ x )
model2 <- lm(y ~ x + z)
model3 <- lm(y ~ x + I(x^2) + I(x^4)+ I(x^5)+ I(x^6) + I(x^7))


# plot the stuff
plot(x,model1$fitted,col="blue",ylab="y - and fitted")
points(x,y,col="green",pch=20)
points(x,model2$fitted,col="red")
points(x,model3$fitted,col="black",pch=20)

