I am working in some regression models to forecast opinions based on general demographic characteristics, and I'm not sure how to choose between linear regression and curve estimation (I'm using SPSS but could go for R). The data is really messy. I took off the outliers and it is possible to see some tendencies and correlations, but not enough to help me choose among linear or curved models visually. What criteria should I use? It is enough if I test different models and choose the model that fits better using the r-square? Or is there any method to choose between curved and linear models (cubic, exponential, linear, etc)?
$\begingroup$
$\endgroup$
1
-
1$\begingroup$ Related: stats.stackexchange.com/questions/60403/… stats.stackexchange.com/questions/74678/… $\endgroup$– kjetil b halvorsen ♦Commented Jun 4, 2015 at 12:53
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
3
There are a few things to look at, but the most enlightening (I find) is to plot the residuals against your result (you can also plot them against your argument). If they are uniform, centered on zero, and don't have an obvious pattern then you probably are using the right functional form - especially if the $R^2$ is large.
But it is an art, and you probably want to ask this on the Cross-Validated Stack Exchange, not here on Stack Overflow.
-
3$\begingroup$ This approach is likely to lead to over-fitting. It can be used to evaluate how good a given model is, but it should not be used to select a model -- unless you counterbalance this with a penalty for model complexity and have enough data as compared with the number of candidate models. $\endgroup$ Commented Jun 4, 2015 at 14:01
-
$\begingroup$ Thanks. Glad to see you guys don't totally disapprove of my advice :) $\endgroup$ Commented Jun 4, 2015 at 14:47
-
$\begingroup$ Please chime in with comments and advice, and even additional solutions. I have a lot to learn here too. $\endgroup$ Commented Jun 4, 2015 at 16:29