# Feature expansion (multiplication) - What to do with higher correlations?

If I have a set of features $$\left \{x_1,x_2,x_3 \right \}$$ and I expand the feature set by multiplying all the features to have the following: $$\left \{x_1,x_2,x_3,x_1\times x_2,x_2\times x_3,x_1\times x_3 \right \}$$. Now, I find that two of my features $$\left \{x_1\times x_2, x_2\times x_3 \right\}$$ are highly correlated ($$>0.85$$ spearman/pearson). What should I do? Note that originally $$x_1, x_2$$ and $$x_3$$ are not correlated at all ($$<0.5$$)

And I am training Logistic Regression/SVM with these features, so correlation matters.

Thanks

• Please explain why correlation among the features matters at all! Your language of "training" suggests your objective is prediction rather than explanation; although correlation among predictors can affect an explanation, it has no effect on prediction.
– whuber
Feb 28 '20 at 0:46
• Yes, my objective is prediction. I will train/validate/test doing K-fold cross validation and so on. But my problem comes from the fact that high correlated features make the models (weights) unstable which is usually not recommended Feb 28 '20 at 14:49
• Sometimes instability is a problem, but it depends on the extent to which the explanatory variables may differ in future samples. Whenever they are substantially outside the range used in modeling, you're taking a risk. "Instability," understood in this relative sense, is no problem at all, because it inheres in all such predictive modeling efforts.
– whuber
Feb 28 '20 at 14:52