So after finalizing all the analysis based on the assumption that the relationships between my predicted variable and the predictors are linear (which was concluded based on looking at the scatter plots and 'fitting' a straight linear line which generally gathered the cases around it, my variances are also all homoscedastic as well), I discovered through what is called curve estimation that I could also fit a nonlinear line and I tried both quadratic and cubic. To my surprise, they both returned an R-square very close (a bit higher but I am aware about the possibility of over-fitting) than the original linear one but the F test value is always the highest for the linear model. What does this suggest?

  1. Does it mean that the relationships between my variables are not only linear? In other words, does this violate the assumption of multiple linear regression?
  2. Am I obligated to report the non-linear relationships?

Thank you.

  • $\begingroup$ As the order of polynomial grows the fitting curve gets more flexibility to meander, and therefore usually predict better and better data of a given dataset (sample) if there is some nonlinearity in it. It does not necessarily mean the dependency is that curvilinear in population. F tests for the situation in population; it is MSregr/MSerr. "Mean square" is SS/df. In models with constant term, df for MSregr is k-1, and df for MSerr is n-k. k is the number of parameters in the model. k=2 for linear, k=3 for quadratic, k=4 for cubic. $\endgroup$ – ttnphns Nov 10 '16 at 13:00
  • $\begingroup$ (cont.) Knowing it now, you can see youself why F test p-value may be less significant for cubic or quadratic than for linear despite the observed R-square was higher for them. You are not obliged to report non-linear relationships. Linear or nonlinear - is your choice, it is the model you decide to select. There, however, exist suggestions how to select the "best" (in some respect) model for a population. $\endgroup$ – ttnphns Nov 10 '16 at 13:00
  • $\begingroup$ Yes. But I am confused here, does this mean that ALL data sets can have different type of relationship fitting at the same time? What about my questions? Let's say all R-squares are around 20%. Can I confidently say that the assumption of linearity has been met? I am really confused here. $\endgroup$ – R. AS. Nov 10 '16 at 13:04
  • $\begingroup$ It is my choice to do a multiple linear regression analysis and so I don't desire any non-linearity (I am afraid that it may affect the results if other methods are followed though but all the r-squares were similar). $\endgroup$ – R. AS. Nov 10 '16 at 13:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.