# Linear regression or polynomial regression? - Choice of features

I am trying to predict an output dependent variable $Y$ for a feature set of 4 independent variables - $X_1, \dots,X_4$ using regression. Here is a scatter plot of each of the dependent variables against the dependent variable $Y$.

Linear regression of the features $X_1, \dots, X_4$ was not able to predict $Y$ well. The $R^2$ of the predictions was just 0.1. I am thinking that a good fit might be obtained if I used more features which are polynomial (or some other function such as log/square root) of the input features instead of directly using the features. I tried fitting Y against second order polynomials such as $X_iX_j$ and $X_i^2$. But it doesn't seem to help.

I think that the shape of the scatter plots of $X_1$ and $X_2$ might be giving a clue. Based on the scatter plots here, can you give any recommendations about the list of features I can use to predict $Y$? How I can improve the prediction of $Y$? I have tried feature scaling and regularization. They don't seem to help.

I am using scikit-learn to fit and predict. Here is the sample code.

*X_train,X_test,y_train,y_test = train_test_split(X,y,test_size=0.3, random_state=1)
lin_reg1 = LinearRegression()
lin_reg1.fit(X_train,y_train)
print('The score is {}'.format(lin_reg1.score(X_test,y_test)))*

• Do you have to use linear regression? How does regressing using random forests perform? Or k-nn regression? Mar 11, 2018 at 1:03
• Regarding "Based on the scatter plots here, can you give any recommendations about the list of features I can use to predict $Y$", then it is hard to see a) whether there is any marginal dependence at all (it seems like this is not the case) and b) whether you just have a lots of data points. Plot a smooth instead. Mar 11, 2018 at 8:26
• By the way, you should not cross post. You have also posted this question on Stackexchange. Mar 11, 2018 at 8:28
• Thanks a lot for your reply, KirkDCO. I am not restricted to use only linear regression. I will try random forest and k-nn regression and update you. Thanks a lot for your suggestions. It really helps a ML newbie like me. Mar 11, 2018 at 11:26
• Thanks a lot for your feedback, Benjamin. I will try your suggestion and update you. And sorry about cross posting. Am new to stackexchange as well. Just trying to learn the ropes.. Mar 11, 2018 at 11:28