# How much can the Pearson and Spearman correlation coefficients differ in a dataset? (edited)

Probably this has been asked and answered before, but I could not find an answer. It can also be, that I don't find an answer, because it is not a meaningful question.

Nevertheless, I would be interested in a set of data with Pearson correlation above 0.95 and Spearman correlation below 0.6.

• Are you asking if it's possible to have a set of data w/ those values, or have you seen one & want to know how to interpret that? Oct 4, 2019 at 19:40
• I have not seen one like that, I am asking whether this is possible or not. Oct 4, 2019 at 19:57
• Could you please explain what you mean by a "linear" or "monotonic" relationship? The reason why this needs elaboration is plain in some of the answers you are receiving, which exhibit scatterplots that are obviously neither linear nor monotonic. The issue is whether you are asking about data or a regression model.
– whuber
Oct 4, 2019 at 21:15
• @whuber I am not sure how to do this. I do not think it is appropriate to change the title, although as I explained in my comment to the answer by Alexis, it would have been better to ask "Can a relationship measured to be strongly linear but not strongly monotonic?" I think, the text of the question is clear. I was looking for an example set of data with very specific Pearson and Spearman correlation coefficients. The answers I get are to this specific question. The reason I did not accept either of them, because they all give different perspectives. Oct 4, 2019 at 21:28
• I did: it isn't clear what you mean by a "relationship" or even by "linear." Are you implicitly equating the meanings of "monotonic" and "linear" with the Spearman and Pearson correlation coefficients, respectively? If so, are you referring to the coefficients in the underlying model or in the data? The answers clearly use different senses of "linear" and "monotonic," which is why some of them disagree.
– whuber
Oct 4, 2019 at 21:36

Sure. We can achieve this result by adding a single extreme data point to an otherwise uncorrelated, and nonmonotonically related, set of data:

x <- c(rnorm(99), 100)
y <- c(rnorm(99), 100)

> cor(x,y,method="pearson")
[1] 0.990387
> cor(x,y,method="spearman")
[1] 0.02534653


Note that the first 99 values are uncorrelated with each other according to either definition of the term, but the 100th value is extreme. This causes the Pearson correlation coefficient to be large, but the Spearman correlation is less affected, as the rank of the largest value is 1 regardless of how large the value itself is.

Compare to a less extreme outlier:

x[100] <- 10
y[100] <- 10

> cor(x,y,method="pearson")
[1] 0.5148878
> cor(x,y,method="spearman")
[1] 0.02534653


And a plot:

• It’s possible to make this even weirder by giving a strong linear trend downward on that cluster of points. The Spearman will be strongly negative while Pearson is strongly positive!
– Dave
Oct 4, 2019 at 21:00
• Nice illustration of the leverage effect in regression/correlation. Oct 4, 2019 at 21:10

An (less extreme) example without outlier is

x <- 1:200
y <- c(rep(0, 50), rep(1, 100), rep(2, 50))  - 0.001 * x

plot(y ~ x, type = "s")
cor(x, y)               # 0.90
cor(x, y, method = "s") # 0.69


• This is not a linear relationship between $Y$ and $X$ without adding an additive interaction with at one additional variable creating the shifts in the intercept. Indeed the positive Pearson;s $r$ that must result from such a graph belies the negative association between $Y$ and $X$ which would result from conditioning on that/those other variables. Oct 9, 2019 at 16:03

Part I. No: Linearity is one example of monotonicity.

A monotonic relationship between $$Y$$ and $$X$$ means "$$Y$$ never decreases as $$X$$ increases, but only increases or remains constant" (for positive monotonicity), or "$$Y$$ never increases as $$X$$ increases, but only decreases or remains constant" (for negative monotonicity). A positive linear relationship between $$Y$$ and $$X$$ fits the former relationship, and a negative linear relationship between $$Y$$ and $$X$$ fits the latter.

While we cannot validly infer that "monotonicity implies linearity" in precisely the same way that we cannot infer that "food implies apples," we can infer that "linearity implies monotonicity" in the same way that "apples imply food."

Correspondingly, a lack of monotonicity implies a lack of linearity, but a lack of linearity does not imply a lack of monotonicity.

Part II. Pearson's $$r$$ is a fragile measure of linearity

Pearson's $$r$$ is subject to particular assumptions in order to produce estimates which have reliable interpretations. For example, violation of the homoscedasticity assumption (familiar from linear regression) can produce more or less any value of $$r$$ in uncorrelated data (i.e. in data with no linear, and no monotonic relation) with the simple addition of a high leverage data point which is (1) an extreme value in the $$X$$ dimension, and (2) has a value of $$Y$$ for that extreme value of $$X$$ including some non-zero slope on the linear regression between $$Y$$ and $$X$$. In this way, you can obtain an arbitrarily strong (and invalid) estimate of linearity, with an arbitrarily weak Spearman's $$r_{\text{S}}$$.

• Probably I was not careful enough phrasing the title. I was looking for an example that is given in the answer by @jbowman . Probably I should have asked "Can a relationship be measured to be much more linear than monotonic?" And apparently (contrary to my intuition) it can. Oct 4, 2019 at 20:22
• @FerencBeleznay Please see my update. Oct 4, 2019 at 20:26