Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
I am doing some Machine Learning experiments with Azure and the graphs that it gives me are measured in
Spearman Correlation vs Iteration Number (part of the machine learning)
However I was just in a talk that stated that Normalized Root mean SQUARED error is recommended instead of Spearman.
Can anyone explain the difference between these two metrics and why RMSE is "recommended"?
The choice will boil down to the problem at hand: are you looking to predict the rank order between dependent variables of certain items (for example, the rank order of stock returns) or are you predicting the actual value of the dependent variable (ie. the stock return).
Spearman correlation between two variables captures the correlation between the rank order of the two variables. So, if Spearman correlation is your performance metric, your model will be learning to find the order of items when sorted on the basis of the dependent variable. Because you consider only the order and not actual values, this has the benefit of not getting affected by outliers or noisiness in the data. For example, stock return data is quite noisy, so, fund managers predict rank order among stock returns rather than actual returns. They will use Spearman correlation as a performance metric for their models.
RMSE, on the other hand, is the root of the average squared error your model produces. This will directly try to match the dependent variables of the observations in the data. This is important if you are doing say acoustic modeling/ sound synthesis. You want to generate a sound which is as close to the real thing as possible.
