9
$\begingroup$

I am training an artificial neural network (backpropagation, feed-forward) with non-normal distributed data. Beside the root mean squared error, literature does often suggest the Pearson correlation coefficient for evaluating the quality of the trained net. But, is the Pearson correlation coefficient reasonable, if the training data is not normally distributed? Would it not be more reasonable, to use a rank-based correlation measure, e.g. Spearman rho?

$\endgroup$
  • $\begingroup$ Can you elaborate on how exactly one would use it or give a reference? $\endgroup$ – bayerj Mar 2 '11 at 11:55
6
$\begingroup$

The Pearson correlation coefficient measures linear association. Being based on empirical second central moments, it is influenced by extreme values. Therefore:

  • Evidence of nonlinearity in a scatterplot of actual-vs-predicted values would suggest using an alternative such as the rank correlation (Spearman) coefficient;

    • If the relationship looks monotonic on average (as in the upper row of the illustration), a rank correlation coefficient will be effective;

    • Otherwise, the relationship is curvilinear (as in some examples from the lower row of the illustration, such as the leftmost or the middle u-shaped one) and likely any measure of correlation will be an inadequate description; using a rank correlation coefficient won't fix this.

  • The presence of outlying data in the scatterplot indicates the Pearson correlation coefficient may be overstating the strength of the linear relationship. It might or might not be correct; use it with due caution. The rank correlation coefficient might or might not be better, depending on how trustworthy the outlying values are.

Examples of scatterplots and their Pearson correlations

(Image copied from the Wikipedia article on Pearson product-moment correlation coefficient.)

$\endgroup$
  • $\begingroup$ Can you give some sources for your statement about using rank correlation to measure regression performance? $\endgroup$ – Simon Kuang Mar 8 '15 at 22:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.