0
$\begingroup$

I've often seen the following approach to how people report parameters of best-fit curves: a correlation between X and Y is weak and/or non-significant, so because the data points seem to be placed on what appears to be a parabola, try a 2nd-order polynomial fit, and report those params if the fit is good enough.

I just wonder if it is correct for the curve type to be eye-balled and its parameters then computed. Isn't there a problem of multiple comparisons also apparent? Is there a systematic way to find the curve of best fit, one that eliminates the subjective "hunch" element, or is that element actually desirable?

$\endgroup$
2
$\begingroup$

There are a lot of issues here. The first two, and certainly not the least, are that higher-order polynomials will always fit at least as well as lower-order polynomials, and that data that lies perfectly on a parabola can still be significantly linearly correlated. These are some of many reasons that model fit alone is not a good model-selection criterion, and nor is correlation. Better methods of model selection include AIC, cross-validated prediction error, and Bayes factors. Which to use depends in part on what you want the model to do, because different purposes call for different models.

$\endgroup$
  • $\begingroup$ Just to make I understand your proposed answer. So even if one did a good job at eye-balling what the best-fit line would be from the first attempt (e.g, a 3rd-order polynomial), and finds that curve to fit well, this is still not the best to do it? Is it because doing the fit is itself one way of doing model selection, and model selection should instead be done with those other methods you mention? $\endgroup$ – z8080 Aug 12 '16 at 11:05
  • 1
    $\begingroup$ "So even if one did a good job at eye-balling what the best-fit line would be from the first attempt (e.g, a 3rd-order polynomial)" — A 3rd-order polynomial will always fit better than a parabola, and a parabola will always fit better than a line. That's what I meant by "higher-order polynomials will always fit at least as well as lower-order polynomials". So, if you use model fit as your criterion to decide whether a model is good, you'll always think that higher-order polynomials are better. $\endgroup$ – Kodiologist Aug 12 '16 at 14:30
  • $\begingroup$ Is that really true that N-poly always beats (N-1)-poly? If you have data points almost perfectly alligned on a line, how could a parabolic shape fit better than the straight line that connects these dots? $\endgroup$ – z8080 Aug 12 '16 at 14:32
  • 1
    $\begingroup$ Because a straight line is also a parabola: the models are nested. For a set of perfectly collinear points, the parabola will fit equally as well as a line rather than strictly better, but of course, real data is never perfectly collinear. $\endgroup$ – Kodiologist Aug 12 '16 at 14:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.