I'm learning about logistics regression and gls regression. And I came across an interesting finding. I have 2 predictor variables ($x_1$ and $x_2$) and if I plot the response variable with $x_2$, there is a nice convex curve in the data. I decided to transform $x_2$ with ${x_2}^2$ (x2a) and add it to the data. So my new model will have 3 predictor variables.

I used 2 methods to build regression model:

gls(y~ x1 + x2 + x2a, data=sample)
gls(y~ x1 * x2 * x2a, data=sample)

The second model with interaction among the 3 predictors gives me the lowest mean square errors when vetted against the same dataset. What are some ways to explain this? Does interaction between predictors tend to produce a better result?


In R, when you use x1 * x2 in the formula, you are telling the regression to include these variables and the interactions between them. So your second model has the first model and the interaction term. If you want just the interaction term in the second model, you should do x1:x2 instead.

As you have it now, since the second model includes the first model, it has a lower MSE.

  • $\begingroup$ Thanks. Also I found if the more predictors I add, the better the model becomes based on the same data used to produce the model. When I plot the predictions, it's like a zigzag line. Adj r square becomes very close to 1 and MSE is almost 0. Would this be overfitting? $\endgroup$ – ajax2000 Mar 15 '17 at 0:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.