3
$\begingroup$

I have a set of inputs $x$ and noisy outputs $y$. I think that either $$y = a_0 + a_1 x$$ or $$y = a_0 + a_1 x + a_2 x^2.$$ How can I determine which model was more likely to have generated the data?

Could this be done by cross-validating ordinary least squares and lasso regression on these data, and then doing a paired hypothesis test to see which has the better median coefficient of determination?

|cite|improve this question
$\endgroup$
  • 2
    $\begingroup$ If you mean "Given the correct model is one of these two, which has a greater probability of having generated the data?", that's a question I'd approach from a Bayesian viewpoint. If you mean something other than that, I think you'd need to be more precise about what you actually mean by 'more likely' (I presume it's not intended to be formally a reference to likelihood). $\endgroup$ – Glen_b -Reinstate Monica Mar 19 '15 at 8:00
  • $\begingroup$ Yes, that is what I mean. How can I approach it this way? $\endgroup$ – rhombidodecahedron Mar 19 '15 at 21:54
3
$\begingroup$

More complicated model in this case is subset of other. You could easily test less restricted model versus more restricted one.

F-test and likelihood ratio test are based on this idea. You do not test then significance of parameter estimate of individual regressor.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ Isn't the less complicated model a subset of the other one, not vice versa? Or am I misunderstanding? The way I'm seeing it is MC = {1, x, x^2}; LC = {1, x}, therefore LC $\subset$ MC. $\endgroup$ – rhombidodecahedron Mar 19 '15 at 21:48
  • $\begingroup$ Do you calculate the F-test using the folds from cross validation? Or what? $\endgroup$ – rhombidodecahedron Mar 20 '15 at 0:05
  • $\begingroup$ @rhombidodecahedron I am sorry, of course less complicated is subset of more complicated one. I can also test model structure via classical test statistics when using all observations. $\endgroup$ – Analyst Mar 23 '15 at 8:25
3
$\begingroup$

Firstly, You can use cross-validation to opt for model complexity, which minimizes validation set error. You should also analysis model diagnostic plots for both models.

Also, in R, you can use anova() function to compare nested models. There are many other functions to compare models based on different criteria. Also, by visualizing data you can get a better understanding of the nature of relationship between the variables. Scatter plots should be the initial step before building regression model.

|cite|improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.