In the context of multiple linear regression, is it acceptable to use Pearson correlation to discriminate how well a model fits a data set?

Let's say that I have some experimental values that come from some function $y=w^Tx + \epsilon$. I made $N$ observations on $p>>N$ binary features, and now I would like to determine the weights (coefficients) associated with each binary feature. My goal is to recover some meaningful interpretation of the weights.

From what I have read, the default method of evaluating goodness of fit is Mean Squared Error (MSE) where I choose the model that has the lowest $\frac{(\hat{y}-y)^2}{N}$. However, I would like to know if I can use $corr(\hat{y},y)$ to determine the best model. My intuition is to select the model with $corr(\hat{y},y)$ closest to 1.

I understand that Pearson is scale invariant - which will lead to an unstable solution. For example, in a simpler context, if I were to compare the model $3X_1 + 2X_2 + 7X_3$, the Pearson correlation would evaluate the response from this model to be equivalent to the response of $6X_1 + 4X_2 + 14X_3$ However, if I am only concerned about capturing the ratio of the weights, is Pearson acceptable?

  • $\begingroup$ Welcome to our site! Could you edit your question to clarify some obscure points? First, in the context of multiple regression there are multiple variables and therefore many possible correlations: to which one(s) do you refer? Second, what do you mean by the weights? Incidentally, could you perhaps indicate why you think a scale-invariant measure could cause instability in the estimates of the model selection? It is hard to see how one implies the other. $\endgroup$ – whuber Apr 18 '14 at 1:50
  • 1
    $\begingroup$ Thanks whuber! The question has been updated with further clarification details. $\endgroup$ – Vincent Apr 18 '14 at 4:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.