# Fitting a Multiple linear regression model to a set of data which was generated by a non-linear function plus an error term(normally distributed)

I am just looking for tips on where to start as I am completely lost with this. This is for a college project were I have been given a data set with 1 response variable(Y) and 2 predictor variables(X1,X2) with 1000 data points. I was told to try and create other variables such exp(X1), X1^2,etc to try and fit the model. However no matter how many polynomial,trigonometric, exponential or other functions I try and fit to the data I can't get the R^2 above 2%. I am not sure if it helps but all the data point Y,X1,X2 range from around(-4,+4). I have included a Matrix scatter plot of the data below. Any help would be appreciated.

edit: Just to clarify for the project I was told I needed to use a Linear Regression model, Thank you for the reply pvlkmrv,sorry I was unclear that was my fault also I have never used python so even if I was allowed to use A non-linear model I would be unable to use skikit.

Also some new info is I have tried using a Taylors series expansion with all terms up to the 11th power for it and the model still only has an R^2 of 22%(And thats with the 133 polynomial terms) which can be brought down to an R^2 of 11% with stepwise regression(This only has 19 terms) and for both these models the regression p value is 0.000. However once I remove one outlier(It has a y value of -9,I took this value out for the initial Matrix plot shown above) the R^2 goes down to 14% for the 133 term model which has a regression p value of 0.222 and using stepwise regression I get a 31 term model with an R^2 of 4% with a regression p value of 0.12. I presume this means I should get rid of this data point? Also when I remove the other outliers gotten from the boxplot of X1 and X2 the R^2 gets even worse as does the regression p value which leaves me kind of lost.

## 1 Answer

If the data was generated by a linear combination of nonlinear functions of the input data and you have no clue what those nonlinear functions are, then the solution is to not use a linear model.

Try a multi-layer perceptron (built in to scikit learn and easy to use) or projection pursuit regression (not built in to scikit learn).