The Pearson's coefficient between two variables is quite high (r=.65). But when I rank the variable values and run a Spearman's correlation, the cofficient value is much lower (r=.30).
- What is the interpretation of this?
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.Sign up to join this community
If your data is normally distributed or uniformly distributed, I would think that Spearman's and Pearson's correlation should be fairly similar.
If they are giving very different results as in your case (.65 versus .30), my guess is that you have skewed data or outliers, and that outliers are leading Pearson's correlation to be larger than Spearman's correlation. I.e., very high values on X might co-occur with very high values on Y.
Also see these previous questions on differences between Spearman and Pearson's correlation:
The following is a simple simulation of how this might occur. Note that the case below involves a single outlier, but that you could produce similar effects with multiple outliers or skewed data.
# Set Seed of random number generator set.seed(4444) # Generate random data # First, create some normally distributed correlated data x1 <- rnorm(200) y1 <- rnorm(200) + .6 * x1 # Second, add a major outlier x2 <- c(x1, 14) y2 <- c(y1, 14) # Plot both data sets par(mfrow=c(2,2)) plot(x1, y1, main="Raw no outlier") plot(x2, y2, main="Raw with outlier") plot(rank(x1), rank(y1), main="Rank no outlier") plot(rank(x2), rank(y2), main="Rank with outlier") # Calculate correlations on both datasets round(cor(x1, y1, method="pearson"), 2) round(cor(x1, y1, method="spearman"), 2) round(cor(x2, y2, method="pearson"), 2) round(cor(x2, y2, method="spearman"), 2)
Which gives this output
 0.44  0.44  0.7  0.44
The correlation analysis shows that without the outlier Spearman and Pearson are quite similar, and with the rather extreme outlier, the correlation is quite different.
The plot below shows how treating the data as ranks removes the extreme influence of the outlier, thus leading Spearman to be similar both with and without the outlier whereas Pearson is quite different when the outlier is added. This highlights why Spearman is often called robust.