What is the explanation for obtaining a Pearson's correlation coefficient value that is significantly larger (a factor of ~2) than the Spearman's rank correlation coefficient value (on the same data)?
Doesn't this goes against the idea that Spearman's rank correlation coefficient, being the Pearson's correlation coefficient of the ranked data, can be seen as a generalization of Pearson's evaluation for monotonic dependences instead of linear ones? How can the correlation coefficient value for the monotonic dependence be smaller than that for a linear dependence only?
I was surprised to see that this was possible in a dataset with $N$~100 elements. I should add that the p-value associated to the Pearson's correlation coefficient is of 0.0 while that of Spearman's rank is of ~0.10.
Possible explanation:
This behaviour might be driven by the extreme values of the dataset. I compare the values of Pearson's c.c. ($\rho$) and Spearman's rank c.c. ($\rho_r$) after removal of these. I present the 2-sided p-values.
Full dataset: $\rho$ = 0.381 (p-value: 0.000), $\rho_r$ = 0.151 (p-value: 0.131)
One outlier removed: $\rho$ = 0.336 (p-value: 0.001), $\rho_r$ = 0.125 (p-value: 0.213)
Three outliers removed: $\rho$ = 0.167 (p-value: 0.100), $\rho_r$ = 0.076 (p-value: 0.459)
The remaining distribution (plotted) does not seem affected by the presence of outliers and yet is still exhibits the same behaviour. The full data is available here; note the outliers correspond to the first three rows.