# Multiple regression - how to deal with mixed linear and non-linear variables

Say I have a bunch of explanatory variables to predict a continuous independent variable. Below, a simple toy example: I think it would be easiest to do a log-log transform and proceed with linear regression. Looks like that the explanatory variables are relatively highly correlated, and there might be a high collinearity between them. This, I would take care of later via e.g.,

• Lasso regression (or Ridge)
• feature selection algos
• Partial Least Squares
• Decision trees and feature importance
• Dimensionality reduction via principal component analysis

But back to the log-log-transform; now, the data looks like this: To me, it looks like that x2, x3, and x4 are now better suited for a linear regression. However, x1 does not look "very linear" anymore. How would I best deal with x1 before I proceed?

I'm not sure what your plots are of since the axes are not properly labeled. However, if your goal is simply to predict $y$ and get the best predictions are possible, and there is no need to interpret the coefficients, why not simply fit the the model with all your variables and then validate the data with a holdout sample. Select the one with the smallest predicted error. Don't worry about your model assumptions so much if your only goal is to predict. It would be good to get a better idea of exactly what you want to do with your model, however, before a "correct" answer could be given to your question.
• You could try using a quadratic term for $x_1$ (i.e. $x_1^2$). – StatsStudent Jan 31 '15 at 6:21
• BTW, $R^2$ can be improved by simply adding additional terms to your model. – StatsStudent Jan 31 '15 at 6:27
Actually the first plot didn't quite look linear in $x_1$ to me (though it's hard to tell from marginal plots anyway; you would get better information from partial plots).
If the relationship is not linear, you could consider either transforming $x_1$ (particularly in instances where that has a nice interpretation), or fitting a nonlinear form of relationship in $x_1$.