0
$\begingroup$

I have a question about using $R^2$ as a "best fit" technique for cross-sectional (not time series) type data...

Suppose you have a data set, and you're trying to fit a regression model to it. You try several types of models (classic linear, exponential, log-log, etc), and ultimately choose one with the highest $R^2$ value (unadjusted?).

Is this an appropriate way to select a regression model, or are there other, more appropriate ways to determine the model which best fits the data?

Thanks

$\endgroup$
  • $\begingroup$ In this case, you will pick a model with as many parameters as observations, leading to the optimal R squared of 1. By including many high order polynomials of the predictors and their interactions, this is easy to achieve. $\endgroup$ – Michael M Jul 14 '15 at 19:04
0
$\begingroup$

There is a much more appropriate way: the information-theoretic approach. Statistics such as AIC and BIC are distribution-independent and can be used to compare any number of candidate models. A great reference is Model Selection and Multimodel Inference by Kenneth Burnham.

I'll flesh this out a bit more: $R^2$ is useful in establishing how well a line fits data. It is defined as $$R^2:=1 - \frac{\text{Sum of vertical distances from the line of best fit}}{\text{Sum of horizontal distances from the line of best fit}}$$ This quantity works well in linear cases because it is bounded by 0 and 1; but for your purposes, e.g. exponential fits, there is no such guarantee of behavior of your data and so $R^2$ isn't such a good metric.

EDIT: See the answer in this post as to why $R^2$ is insufficient.

$\endgroup$
  • $\begingroup$ AIC and BIC are both likelihood-based, so what exactly do you mean when you say they're distribution independent? $\endgroup$ – dsaxton Jul 17 '15 at 1:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.