How can one objectively (read "algorithmically") select an appropriate model for doing a simple linear least-squares regression with two variables?
For example, say the data seem to show a quadratic trend, and a parabola is generated which fits the data quite well. How do we justify making this the regression? Or how do we eliminate the possibility of there existing a better model?
What I'm really worried about is this: we could just keep adding polynomial terms until we had a perfect fit for the data set (an interpolation of the points), with no error whatsoever. But this would be useless as far as predicting or extrapolating, because there would be no reason to think that the "model" was actually appropriate. So how does one balance the needs of accuracy and intuitive appeal?
(Also, please alert me if this has been asked before, I assumed it would have been but did not find anything.)