155
$\begingroup$

I get this question frequently enough in my statistics consulting work, that I thought I'd post it here. I have an answer, which is posted below, but I was keen to hear what others have to say.

Question: If you have two variables that are not normally distributed, should you use Spearman's rho for the correlation?

$\endgroup$
3
  • 4
    $\begingroup$ Why not calculate and report both (Pearson's r and Spearman's ρ)? Their difference (or lack thereof) will provide additional information. $\endgroup$
    – user14650
    Commented Sep 9, 2015 at 7:51
  • $\begingroup$ A question comparing the distributional assumptions made when we test for significance a simple regression coefficient beta and when we test Pearson correlation coefficient (numerically eual to the beta) stats.stackexchange.com/q/181043/3277. $\endgroup$
    – ttnphns
    Commented Nov 29, 2015 at 9:36
  • 5
    $\begingroup$ Pearson's correlation is linear, Spearman's is monotonic, so they're not normally for the same purpose. The Pearson coefficient doesn't need you to assume normality. There's a test for it that does assume normality, but you don't have only that option. $\endgroup$
    – Glen_b
    Commented Feb 24, 2020 at 8:03

8 Answers 8

116
$\begingroup$

Pearson's correlation is a measure of the linear relationship between two continuous random variables. It does not assume normality although it does assume finite variances and finite covariance. When the variables are bivariate normal, Pearson's correlation provides a complete description of the association.

Spearman's correlation applies to ranks and so provides a measure of a monotonic relationship between two continuous random variables. It is also useful with ordinal data and is robust to outliers (unlike Pearson's correlation).

The distribution of either correlation coefficient will depend on the underlying distribution, although both are asymptotically normal because of the central limit theorem.

$\endgroup$
14
  • 19
    $\begingroup$ Pearson's $\rho$ does not assume normality, but is only an exhaustive measure of association if the joint distribution is multivariate normal. Given the confusion this distinction elicits, you might want to add it to your answer. $\endgroup$
    – user603
    Commented Oct 19, 2010 at 7:42
  • 9
    $\begingroup$ Is there a source that can be quoted to support the above statement (Person's r does not assume normality)? We're having the same argument in our department at the moment. $\endgroup$
    – user8891
    Commented Feb 1, 2012 at 14:52
  • 13
    $\begingroup$ "When the variables are bivariate normal, Pearson's correlation provides a complete description of the association." And when the variables are NOT bivariate normal, how useful is Pearson's correlation? $\endgroup$
    – landroni
    Commented Sep 16, 2014 at 10:07
  • 8
    $\begingroup$ This answer seems rather indirect. "When the variables are bivariate normal ..." And when not? This kind of explanation is why I never get statistics. "Rob, how do you like my new dress?" "The dark color emphasizes your light skin." "Sure, Rob, but do you like how it emphasisez my skin?" "Light skin is considered beautiful in many cultures." "I know, Rob, but do you like it?" "I think the dress is beautiful." "I think so, too, Rob, but is it beautiful on me?" "You always look beautiful to me, honey." sigh $\endgroup$
    – user14650
    Commented Aug 12, 2015 at 11:19
  • 5
    $\begingroup$ No, we don't. It's quite possible to do inference for Pearson's correlation without assuming bivariate normality, in at least four different ways. (i) use asymptotic results -- already mentioned above; (ii) make some other parametric distributional assumption and derive or simulate the null distribution of the test statistic; (iii) use a permutation test; (iv) use a bootstrap test. There are probably other approaches $\endgroup$
    – Glen_b
    Commented Oct 6, 2019 at 3:46
65
$\begingroup$

Don't forget Kendall's tau! Roger Newson has argued for the superiority of Kendall's τa over Spearman's correlation rS as a rank-based measure of correlation in a paper whose full text is now freely available online:

Newson R. Parameters behind "nonparametric" statistics: Kendall's tau,Somers' D and median differences. Stata Journal 2002; 2(1):45-64.

He references (on p47) Kendall & Gibbons (1990) as arguing that "...confidence intervals for Spearman’s rS are less reliable and less interpretable than confidence intervals for Kendall’s τ-parameters, but the sample Spearman’s rS is much more easily calculated without a computer" (which is no longer of much importance of course).

Kendall, M. G. and J. D. Gibbons. 1990. Rank Correlation Methods. 5th ed. London: Griffin.

$\endgroup$
2
  • 7
    $\begingroup$ I'm also a big fan of Kendall's tau. Pearson is far too sensitive to influential points/outliers for my taste, and while Spearman doesn't suffer from this problem, I personally find Kendall easier to understand, interpret and explain than Spearman. Of course, your mileage may vary. $\endgroup$ Commented Oct 19, 2010 at 7:44
  • 1
    $\begingroup$ My recollection from experience is that Kendall's tau still runs a lot slower (in R) than Spearman's. This can be important if your dataset is large. $\endgroup$ Commented May 25, 2018 at 18:07
49
$\begingroup$

From an applied perspective, I am more concerned with choosing an approach that summarises the relationship between two variables in a way that aligns with my research question. I think that determining a method for getting accurate standard errors and p-values is a question that should come second. Even if you chose not to rely on asymptotics, there's always the option to bootstrap or change distributional assumptions.

As a general rule, I prefer Pearson's correlation because (a) it generally aligns more with my theoretical interests; (b) it enables more direct comparability of findings across studies, because most studies in my area report Pearson's correlation; and (c) in many settings there is minimal difference between Pearson and Spearman correlation coefficients.

However, there are situations where I think Pearson's correlation on raw variables is misleading.

  • Outliers: Outliers can have great influence on Pearson's correlations. Many outliers in applied settings reflect measurement failures or other factors that the model is not intended to generalise to. One option is to remove such outliers. Univariate outliers do not exist with Spearman's rho because everything is converted to ranks. Thus, Spearman is more robust.
  • Highly skewed variables: When correlating skewed variables, particularly highly skewed variables, a log or some other transformation often makes the underlying relationship between the two variables clearer (e.g., brain size by body weight of animals). In such settings it may be that the raw metric is not the most meaningful metric anyway. Spearman's rho has a similar effect to transformation by converting both variables to ranks. From this perspective, Spearman's rho can be seen as a quick-and-dirty approach (or more positively, it is less subjective) whereby you don't have to think about optimal transformations.

In both cases above, I would advise researchers to either consider adjustment strategies (e.g., transformations, outlier removal/adjustment) before applying Pearson's correlation or use Spearman's rho.

$\endgroup$
2
  • $\begingroup$ The problem with transformation is that, in general, it also transforms the errors associated to each point, and thus the weight. And it doesn't solve the outlier's problem. $\endgroup$
    – skan
    Commented Mar 9, 2015 at 2:09
  • 1
    $\begingroup$ The previous comment is puzzling. Transformation often tames outliers. Also, what to think about errors depends on the scale you choose for analysis. If a logarithmic scale makes sense, for example, additive errors on that scale often make sense too. $\endgroup$
    – Nick Cox
    Commented Mar 15, 2020 at 10:19
15
$\begingroup$

Updated

The question asks us to choose between Pearson's and Spearman's method when normality is questioned. Restricted to this concern, I think the following paper should inform anyone's decision:

It's quite nice and provides a survey of the considerable literature, spanning decades, on this topic -- starting from Pearson's "mutilated and distorted surfaces" and robustness of distribution of $r$. At least part of the contradictory nature of the "facts" is that much of this work was done before the advent of computing power -- which complicated things because the type of non-normality had to be considered and was hard to examine without simulations.

Kowalski's analysis concludes that the distribution of $r$ is not robust in the presence of non-normality and recommends alternative procedures. The entire paper is quite informative and recommended reading, but skip to the very short conclusion at the end of the paper for a summary.

If asked to choose between one of Spearman and Pearson when normality is violated, the distribution free alternative is worth advocating, i.e. Spearman's method.


Previously ..

Spearman's correlation is a rank based correlation measure; it's non-parametric and does not rest upon an assumption of normality.

The sampling distribution for Pearson's correlation does assume normality; in particular this means that although you can compute it, conclusions based on significance testing may not be sound.

As Rob points out in the comments, with large sample this is not an issue. With small samples though, where normality is violated, Spearman's correlation should be preferred.

Update Mulling over the comments and the answers, it seems to me that this boils down to the usual non-parametric vs. parametric tests debate. Much of the literature, e.g. in biostatistics, doesn't deal with large samples. I'm generally not cavalier with relying on asymptotics. Perhaps it's justified in this case, but that's not readily apparent to me.

$\endgroup$
10
  • 3
    $\begingroup$ No. Pearson's correlation does NOT assume normality. It is an estimate of the correlation between any two continuous random variables and is a consistent estimator under relatively general conditions. Even tests based on Pearson's correlation do not require normality if the samples are large enough because of the CLT. $\endgroup$ Commented Oct 19, 2010 at 1:46
  • 2
    $\begingroup$ I am under the impression that Pearson is defined as long as the underlying distributions have finite variances and covariances. So, normality is not required. If the underlying distributions are not normal then the test-statistic may have a different distribution but that is a secondary issue and not relevant to the question at hand. Is that not so? $\endgroup$
    – user28
    Commented Oct 19, 2010 at 1:47
  • 2
    $\begingroup$ @Rob: Yes, we can always come up with workarounds to make things work out roughly the same. Simply to avoid Spearman's method -- which most non-statisticians can handle with a standard command. I guess my advice remains to use Spearman's method for small samples where normality is questionable. Not sure if that's in dispute here or not. $\endgroup$
    – ars
    Commented Oct 19, 2010 at 23:43
  • 3
    $\begingroup$ @ars. I would use Spearman's if I was interested in monotonic rather than linear association, or if there were outliers or high levels of skewness. I would use Pearson's for linear relationships provided there are no outliers. I don't think the sample size is relevant in making the choice. $\endgroup$ Commented Oct 20, 2010 at 0:32
  • 5
    $\begingroup$ @Rob: OK, thanks for the discussion. I agree with the first part, but doubt the last, and would include that size only plays a role because normal asymptotics don't apply. For example, Kowalski 1972 has a pretty good survey of the history around this, and concludes that the Pearson's correlation is not as robust as thought. See: jstor.org/pss/2346598 $\endgroup$
    – ars
    Commented Oct 20, 2010 at 1:00
6
$\begingroup$

I think these figures (of Gross-Error Sensitivity and Asymptotic Variance) and quotation from the below paper will make it a bit clear:

enter image description here

enter image description here

"The Kendall correlation measure is more robust and slightly more efficient than Spearman’s rank correlation, making it the preferable estimator from both perspectives."

Source: Croux, C. and Dehon, C. (2010). Influence functions of the Spearman and Kendall correlation measures. Statistical Methods and Applications, 19, 497-515.

$\endgroup$
5
$\begingroup$

Even though this is an age old question, I would like to contribute the (cool) observation that Pearson's $\rho$ is nothing but the slope of the trend line between $Y$ and $X$ after means have been removed and the scales are normalized for $\sigma_Y$, i.e. after removing means and normalizing for $\sigma_Y$, Pearson's $r$ is the least-squares solution of $\hat Y=X\hat\beta$ where $\hat Y = Y / \sigma_Y$.

This leads to a quite easy decision rule between the two: Plot $Y$ over the $X$ (simple scatter plot) and add a trend line. If the trend looks out of place then don't use Pearson's $\rho$. Bonus: you get to visualize your data, which is never a bad thing.

If you aren't comfortable with Pearson's $\rho$, then Spearman's rank makes this a bit better because it rescales both the x-axis and the y-axis in a non-linear way (rank encoding) and then fits the trend line in the embedded (transformed) space. In practice, this seems to work well and it does improve robustness towards outliers or skew as others have pointed out.

In theory, I do think Spearman's rank is a bit funny though because rank encoding is a transformation that maps real numbers onto a discrete sequence of numbers. Fitting a linear regression onto discrete numbers is non-sense (they are discrete), so what is happening is that we re-embedd the sequence into the real numbers again using their natural embedding and fit a regression in that space instead. Seems to work well enough in practice, but I do find it funny.

Instead of using Spearman's rank, it may be better to just commit to the rank encoding and go with Kendall's $\tau$ instead; even though we lose the relationship with Pearson's $\rho$.


Pearson's $\rho$ from Least-Squares

We can start with the desire to fit a linear regression model $Y=X\hat\beta + b$ on our observations using least-squares. Here $X$ is a vector of observations and $Y$ is another vector of matching observations. If we are happy to make the assumption that $X$ and $Y$ had their mean removed ($\mu_X=\mu_Y=0$, easy enough to do) then we can reduce the model to $Y=X\hat\beta$. For this, there exists a closed-form solution $\hat\beta=(X^TX)^{-1}X^TY$.

Under the vector notation $\text{Cov}(X, Y) = E[XY]-E[X]E[Y] = E[XY] = X^TY$ - we removed the means - and similarly $\sigma_X = \text{Var}(X, X) = \text{Cov}(X, X) = X^TX$. If we now rewrite $\hat\beta$ in terms of $\text{Cov}$ and $\sigma_X$ we get $\hat\beta = \sigma_X^{-1}\text{Cov}(X,Y) = \frac{\text{Cov}(X,Y)}{\sigma_X}$.

Plugging this back into the model and normalizing for $\sigma_Y$ resuls in $Y/\sigma_Y = \frac{\text{Cov}(X,Y)}{\sigma_X\sigma_Y}X$, where the slope is exactly Pearson's $\rho$. $Y/\sigma_Y$ is the expected rescaling of $Y$, since we are interested in a variance-normalized coefficient.

$\endgroup$
4
  • 1
    $\begingroup$ Fascinating, I never realized this connection. How does Cov(X,Y) = E[XY] - E[X]E[Y]? $\endgroup$
    – saeranv
    Commented Jul 22, 2021 at 5:32
  • 1
    $\begingroup$ @saeranv It is one of the ways to define covariance (or follows quickly from your chosen definition): en.wikipedia.org/wiki/Covariance#Definition $\endgroup$ Commented Jul 22, 2021 at 8:12
  • $\begingroup$ thanks, so obvious, I should have worked it out for myself! I have another thought/question: I am trying to think of an intuitive reason for why $Y$ is equal to $Cov(X,Y)$ normalized by $\sigma_X$ but not $\sigma_Y$? Would it be accurate to say that regression is equal to the $X$ feature vectors shrunk by an "angle factor" between $X$ and $Y$ (since $X \cdot Y = cos(theta_{XY})$, and then scaled by the standard deviation of $Y$? $\endgroup$
    – saeranv
    Commented Jul 23, 2021 at 19:58
  • 1
    $\begingroup$ See stats.stackexchange.com/questions/70969 for a list of equivalent characterizations of correlation. $\endgroup$
    – whuber
    Commented May 2, 2023 at 21:45
3
$\begingroup$

In order to add something that hasn't been said yet in 14 years but in my opinion should be said, it is in my view generally wrong to say that "model assumptions have to be fulfilled" in order to apply a model-based statistical model. Statistical models are idealisations and their job is not to be "true" in reality; reality doesn't normally behave like formal models. The idea that something that supposedly "assumes" normality cannot be applied if data are not normally distributed is wrong. If this were true, no model-based statistical method could ever be applied, and neither nonparametric methods as long as they still involve formal model assumptions such as independence.

Model assumptions mean that a method can be shown to perform in a desirable or optimal manner in a certain idealised situation. Model-based statements such as about inference (probabilities for type I or type II error, confidence interval coverage probabilities, Bayesian posteriors) can be used as quantitative orientations, but do not refer to what is true in reality, rather to formal models of it.

Of course one can hope that these results give a good and useful orientation at least in situations in which data look similar to data having been generated by the assumed model, and one may be more skeptical if they don't. This makes some sense, but formalising and proving/checking such an intuition in a mathematical way isn't straightforward. It can be done in more than one way, and results are often ambiguous (for example from simulations in which methods are applied to data generated from models that deviate from the assumptions), i.e., the intuition may often but not always hold.

The thing that is most important is ultimately to understand what the considered statistics are actually doing to the data, and to know in what cases it can be misleading. Also it is important to take into account to what extent inference is of interest, and what the consequences can be. If statistical inference is not of interest, computing statistics such as Pearson correlation is possible without any assumptions, but the question is still relevant to what extent the value tells you what you want to know.

Other answers say much about the details (such as that Pearson correlation is strongly affected by outliers, Spearman less so) and I won't repeat things here, but note that Pearson correlation does not "assume" linearity either (standard inference based on it does; but bootstrap and permutation tests are possible that don't). It is true that a value of 1 or -1 is equivalent to perfect linearity, and Pearson correlation can be interpreted as slope of a standardised data summary line. Testing whether this is zero (using for example a permutation test) is not the worst thing to do even when interested in testing independence against not necessarily linear monotonicity. There are better alternatives in many specific situations (with outliers Spearman will normally be better), however a Pearson correlation larger than zero, say, means that there is a tendency of one variable to be larger/smaller that its expected value when the other one is (also measuring to what extent amounts of deviation from the mean in the two variables are proportional in a standardised manner), and this is often a sensible thing to be interested in, whether a relationship is really linear or not.

$\endgroup$
2
$\begingroup$

Please be aware that "if inferences are drawn about the relationship (e.g. we set a null hypothesis that r = 0; [no correlation]), then the Pearson's correlation coefficient assumes that the joint distribution of X and Y is ‘bivariate normal’. (...) Bivariate normality implies that both X and Y must come from normal distributions, but the reverse is not necessarily true" (Source).

So in case of significance testing you must care about normality. At least in my field no one just calculates Pearson's correlation without testing against the Null, so IMO for the most situations the answer is yes, (bivariate) normality is an assumption and if this is violated you can either use Spearman or maybe alternatives as mentioned in other answers.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.