# Pearson's or Spearman's correlation with non-normal data

I get this question frequently enough in my statistics consulting work, that I thought I'd post it here. I have an answer, which is posted below, but I was keen to hear what others have to say.

Question: If you have two variables that are not normally distributed, should you use Spearman's rho for the correlation?

• Why not calculate and report both (Pearson's r and Spearman's ρ)? Their difference (or lack thereof) will provide additional information.
– user14650
Sep 9, 2015 at 7:51
• A question comparing the distributional assumptions made when we test for significance a simple regression coefficient beta and when we test Pearson correlation coefficient (numerically eual to the beta) stats.stackexchange.com/q/181043/3277. Nov 29, 2015 at 9:36
• Pearson's correlation is linear, Spearman's is monotonic, so they're not normally for the same purpose. The Pearson coefficient doesn't need you to assume normality. There's a test for it that does assume normality, but you don't have only that option. Feb 24, 2020 at 8:03

Pearson's correlation is a measure of the linear relationship between two continuous random variables. It does not assume normality although it does assume finite variances and finite covariance. When the variables are bivariate normal, Pearson's correlation provides a complete description of the association.

Spearman's correlation applies to ranks and so provides a measure of a monotonic relationship between two continuous random variables. It is also useful with ordinal data and is robust to outliers (unlike Pearson's correlation).

The distribution of either correlation coefficient will depend on the underlying distribution, although both are asymptotically normal because of the central limit theorem.

• Pearson's $\rho$ does not assume normality, but is only an exhaustive measure of association if the joint distribution is multivariate normal. Given the confusion this distinction elicits, you might want to add it to your answer. Oct 19, 2010 at 7:42
• Is there a source that can be quoted to support the above statement (Person's r does not assume normality)? We're having the same argument in our department at the moment.
– user8891
Feb 1, 2012 at 14:52
• "When the variables are bivariate normal, Pearson's correlation provides a complete description of the association." And when the variables are NOT bivariate normal, how useful is Pearson's correlation? Sep 16, 2014 at 10:07
• This answer seems rather indirect. "When the variables are bivariate normal ..." And when not? This kind of explanation is why I never get statistics. "Rob, how do you like my new dress?" "The dark color emphasizes your light skin." "Sure, Rob, but do you like how it emphasisez my skin?" "Light skin is considered beautiful in many cultures." "I know, Rob, but do you like it?" "I think the dress is beautiful." "I think so, too, Rob, but is it beautiful on me?" "You always look beautiful to me, honey." sigh
– user14650
Aug 12, 2015 at 11:19
• No, we don't. It's quite possible to do inference for Pearson's correlation without assuming bivariate normality, in at least four different ways. (i) use asymptotic results -- already mentioned above; (ii) make some other parametric distributional assumption and derive or simulate the null distribution of the test statistic; (iii) use a permutation test; (iv) use a bootstrap test. There are probably other approaches Oct 6, 2019 at 3:46

Don't forget Kendall's tau! Roger Newson has argued for the superiority of Kendall's τa over Spearman's correlation rS as a rank-based measure of correlation in a paper whose full text is now freely available online:

Newson R. Parameters behind "nonparametric" statistics: Kendall's tau,Somers' D and median differences. Stata Journal 2002; 2(1):45-64.

He references (on p47) Kendall & Gibbons (1990) as arguing that "...confidence intervals for Spearman’s rS are less reliable and less interpretable than confidence intervals for Kendall’s τ-parameters, but the sample Spearman’s rS is much more easily calculated without a computer" (which is no longer of much importance of course).

Kendall, M. G. and J. D. Gibbons. 1990. Rank Correlation Methods. 5th ed. London: Griffin.

• I'm also a big fan of Kendall's tau. Pearson is far too sensitive to influential points/outliers for my taste, and while Spearman doesn't suffer from this problem, I personally find Kendall easier to understand, interpret and explain than Spearman. Of course, your mileage may vary. Oct 19, 2010 at 7:44
• My recollection from experience is that Kendall's tau still runs a lot slower (in R) than Spearman's. This can be important if your dataset is large. May 25, 2018 at 18:07

From an applied perspective, I am more concerned with choosing an approach that summarises the relationship between two variables in a way that aligns with my research question. I think that determining a method for getting accurate standard errors and p-values is a question that should come second. Even if you chose not to rely on asymptotics, there's always the option to bootstrap or change distributional assumptions.

As a general rule, I prefer Pearson's correlation because (a) it generally aligns more with my theoretical interests; (b) it enables more direct comparability of findings across studies, because most studies in my area report Pearson's correlation; and (c) in many settings there is minimal difference between Pearson and Spearman correlation coefficients.

However, there are situations where I think Pearson's correlation on raw variables is misleading.

• Outliers: Outliers can have great influence on Pearson's correlations. Many outliers in applied settings reflect measurement failures or other factors that the model is not intended to generalise to. One option is to remove such outliers. Univariate outliers do not exist with Spearman's rho because everything is converted to ranks. Thus, Spearman is more robust.
• Highly skewed variables: When correlating skewed variables, particularly highly skewed variables, a log or some other transformation often makes the underlying relationship between the two variables clearer (e.g., brain size by body weight of animals). In such settings it may be that the raw metric is not the most meaningful metric anyway. Spearman's rho has a similar effect to transformation by converting both variables to ranks. From this perspective, Spearman's rho can be seen as a quick-and-dirty approach (or more positively, it is less subjective) whereby you don't have to think about optimal transformations.

In both cases above, I would advise researchers to either consider adjustment strategies (e.g., transformations, outlier removal/adjustment) before applying Pearson's correlation or use Spearman's rho.

• The problem with transformation is that, in general, it also transforms the errors associated to each point, and thus the weight. And it doesn't solve the outlier's problem.
– skan
Mar 9, 2015 at 2:09
• The previous comment is puzzling. Transformation often tames outliers. Also, what to think about errors depends on the scale you choose for analysis. If a logarithmic scale makes sense, for example, additive errors on that scale often make sense too. Mar 15, 2020 at 10:19

Updated

The question asks us to choose between Pearson's and Spearman's method when normality is questioned. Restricted to this concern, I think the following paper should inform anyone's decision:

It's quite nice and provides a survey of the considerable literature, spanning decades, on this topic -- starting from Pearson's "mutilated and distorted surfaces" and robustness of distribution of $r$. At least part of the contradictory nature of the "facts" is that much of this work was done before the advent of computing power -- which complicated things because the type of non-normality had to be considered and was hard to examine without simulations.

Kowalski's analysis concludes that the distribution of $r$ is not robust in the presence of non-normality and recommends alternative procedures. The entire paper is quite informative and recommended reading, but skip to the very short conclusion at the end of the paper for a summary.

If asked to choose between one of Spearman and Pearson when normality is violated, the distribution free alternative is worth advocating, i.e. Spearman's method.

Previously ..

Spearman's correlation is a rank based correlation measure; it's non-parametric and does not rest upon an assumption of normality.

The sampling distribution for Pearson's correlation does assume normality; in particular this means that although you can compute it, conclusions based on significance testing may not be sound.

As Rob points out in the comments, with large sample this is not an issue. With small samples though, where normality is violated, Spearman's correlation should be preferred.

Update Mulling over the comments and the answers, it seems to me that this boils down to the usual non-parametric vs. parametric tests debate. Much of the literature, e.g. in biostatistics, doesn't deal with large samples. I'm generally not cavalier with relying on asymptotics. Perhaps it's justified in this case, but that's not readily apparent to me.

• No. Pearson's correlation does NOT assume normality. It is an estimate of the correlation between any two continuous random variables and is a consistent estimator under relatively general conditions. Even tests based on Pearson's correlation do not require normality if the samples are large enough because of the CLT. Oct 19, 2010 at 1:46
• I am under the impression that Pearson is defined as long as the underlying distributions have finite variances and covariances. So, normality is not required. If the underlying distributions are not normal then the test-statistic may have a different distribution but that is a secondary issue and not relevant to the question at hand. Is that not so?
– user28
Oct 19, 2010 at 1:47
• @Rob: Yes, we can always come up with workarounds to make things work out roughly the same. Simply to avoid Spearman's method -- which most non-statisticians can handle with a standard command. I guess my advice remains to use Spearman's method for small samples where normality is questionable. Not sure if that's in dispute here or not.
– ars
Oct 19, 2010 at 23:43
• @ars. I would use Spearman's if I was interested in monotonic rather than linear association, or if there were outliers or high levels of skewness. I would use Pearson's for linear relationships provided there are no outliers. I don't think the sample size is relevant in making the choice. Oct 20, 2010 at 0:32
• @Rob: OK, thanks for the discussion. I agree with the first part, but doubt the last, and would include that size only plays a role because normal asymptotics don't apply. For example, Kowalski 1972 has a pretty good survey of the history around this, and concludes that the Pearson's correlation is not as robust as thought. See: jstor.org/pss/2346598
– ars
Oct 20, 2010 at 1:00

I think these figures (of Gross-Error Sensitivity and Asymptotic Variance) and quotation from the below paper will make it a bit clear:  "The Kendall correlation measure is more robust and slightly more efficient than Spearman’s rank correlation, making it the preferable estimator from both perspectives."

Source: Croux, C. and Dehon, C. (2010). Influence functions of the Spearman and Kendall correlation measures. Statistical Methods and Applications, 19, 497-515.

Even though this is an age old question, I would like to contribute the (cool) observation that Pearson's $$\rho$$ is nothing but the slope of the trend line between $$Y$$ and $$X$$ after means have been removed and the scales are normalized for $$\sigma_Y$$, i.e. after removing means and normalizing for $$\sigma_Y$$, Pearson's $$r$$ is the least-squares solution of $$\hat Y=X\hat\beta$$ where $$\hat Y = Y / \sigma_Y$$.

This leads to a quite easy decision rule between the two: Plot $$Y$$ over the $$X$$ (simple scatter plot) and add a trend line. If the trend looks out of place then don't use Pearson's $$\rho$$. Bonus: you get to visualize your data, which is never a bad thing.

If you aren't comfortable with Pearson's $$\rho$$, then Spearman's rank makes this a bit better because it rescales both the x-axis and the y-axis in a non-linear way (rank encoding) and then fits the trend line in the embedded (transformed) space. In practice, this seems to work well and it does improve robustness towards outliers or skew as others have pointed out.

In theory, I do think Spearman's rank is a bit funny though because rank encoding is a transformation that maps real numbers onto a discrete sequence of numbers. Fitting a linear regression onto discrete numbers is non-sense (they are discrete), so what is happening is that we re-embedd the sequence into the real numbers again using their natural embedding and fit a regression in that space instead. Seems to work well enough in practice, but I do find it funny.

Instead of using Spearman's rank, it may be better to just commit to the rank encoding and go with Kendall's $$\tau$$ instead; even though we lose the relationship with Pearson's $$\rho$$.

#### Pearson's $$\rho$$ from Least-Squares

We can start with the desire to fit a linear regression model $$Y=X\hat\beta + b$$ on our observations using least-squares. Here $$X$$ is a vector of observations and $$Y$$ is another vector of matching observations. If we are happy to make the assumption that $$X$$ and $$Y$$ had their mean removed ($$\mu_X=\mu_Y=0$$, easy enough to do) then we can reduce the model to $$Y=X\hat\beta$$. For this, there exists a closed-form solution $$\hat\beta=(X^TX)^{-1}X^TY$$.

Under the vector notation $$\text{Cov}(X, Y) = E[XY]-E[X]E[Y] = E[XY] = X^TY$$ - we removed the means - and similarly $$\sigma_X = \text{Var}(X, X) = \text{Cov}(X, X) = X^TX$$. If we now rewrite $$\hat\beta$$ in terms of $$\text{Cov}$$ and $$\sigma_X$$ we get $$\hat\beta = \sigma_X^{-1}\text{Cov}(X,Y) = \frac{\text{Cov}(X,Y)}{\sigma_X}$$.

Plugging this back into the model and normalizing for $$\sigma_Y$$ resuls in $$Y/\sigma_Y = \frac{\text{Cov}(X,Y)}{\sigma_X\sigma_Y}X$$, where the slope is exactly Pearson's $$\rho$$. $$Y/\sigma_Y$$ is the expected rescaling of $$Y$$, since we are interested in a variance-normalized coefficient.

• Fascinating, I never realized this connection. How does Cov(X,Y) = E[XY] - E[X]E[Y]? Jul 22, 2021 at 5:32
• @saeranv It is one of the ways to define covariance (or follows quickly from your chosen definition): en.wikipedia.org/wiki/Covariance#Definition Jul 22, 2021 at 8:12
• thanks, so obvious, I should have worked it out for myself! I have another thought/question: I am trying to think of an intuitive reason for why $Y$ is equal to $Cov(X,Y)$ normalized by $\sigma_X$ but not $\sigma_Y$? Would it be accurate to say that regression is equal to the $X$ feature vectors shrunk by an "angle factor" between $X$ and $Y$ (since $X \cdot Y = cos(theta_{XY})$, and then scaled by the standard deviation of $Y$? Jul 23, 2021 at 19:58
• See stats.stackexchange.com/questions/70969 for a list of equivalent characterizations of correlation.
– whuber
May 2 at 21:45