# How robust is Pearson's correlation coefficient to violations of normality?

Data for certain kinds of variables tends to be non-normal when measured in particular populations (e.g. levels of depression in a population of people with Major Depressive Disorder). Given that Pearson's assumes normality, how robust is the test statistic under conditions of non-normality?

I have a number of variables that I would like correlation coefficients for, but Z-skewness for some of those variables is significant at p<.001 (and that's for a relatively small sample). I've tried some transformations, but the improvements in the distributions are only marginal at best.

Am I going to have to stick with non-parametric analyses? And not just for correlations, but for other types of analysis as well?

• Wait, Pearson's correlation coefficient assumes normality? I don't think it does, and I have been using it on non-normal data. It's just not robust to some things which happen more often in some non-normal situations, but there are plenty of non-normal situations where I see no problem with using Pearson's correlation coefficient. – Douglas Zare Sep 29 '12 at 21:04
• That Pearson's correlation assumes normality is what many stats texts claim. I've heard elsewhere that normality is a needless assumption for Pearson's r. When I run the analyses, both Pearson's and Spearman's produce relatively similar results. – Archaeopteryx Sep 30 '12 at 2:31
• Spearman's rank correlation coefficient is Pearson's correlation coefficient applied to the non-normal rankings. I still don't know in what sense you believe Pearson's requires normality. Perhaps you can say a few extra things in case you are using it on a multivariate normal distribution. – Douglas Zare Sep 30 '12 at 4:11
• I'm just using it for simple bivariate correlations. I'm not sure why it is claimed that normality is required. The stats texts I've read always list normality as an assumption of the Pearson's correlation and advise to use Spearman's for conditions in which non-normality holds. – Archaeopteryx Sep 30 '12 at 4:33

Short answer: Very non-robust. The correlation is a measure of linear dependence, and when one variable can’t be written as a linear function of the other (and still have the given marginal distribution), you can’t have perfect (positive or negative) correlation. In fact, the possible correlations values can be severely restricted.

The problem is that while the population correlation is always between $-1$ and $1$, the exact range attainable heavily depends on the marginal distributions. A quick proof and demonstration:

# Attainable range of the correlation

If $(X,Y)$ has the distribution function $H$ and marginal distribution functions $F$ and $G$, there exists some rather nice upper and lower bounds for $H$, $$H_-(x,y) \leq H(x,y) \leq H_+(x,y),$$ called Fréchet bounds. These are \begin{aligned} H_-(x,y) &= \max(F(x) + G(y)-1, 0)\\ H_+(x,y) &= \min(F(x), G(y)). \end{aligned} (Try to prove it; it’s not very difficult.)

The bounds are themselves distribution functions. Let $U$ have a uniform distribution. The upper bound is the distribution function of $(X,Y)=(F^-(U), G^-(U))$ and the lower bound is the distribution function of $(F^-(-U), G^-(1-U))$.

Now, using this variant on the formula for the covariance, $$\mathop{\textrm{Cov}}(X,Y)=\iint H(x,y)-F(x)G(y) \mathop{\mathrm d\!}x \mathop{\mathrm d\!}y,$$ we see that we obtain the maximum and minimum correlation when $H$ is equal to $H_+$ and $H_-$, respectively, i.e., when $Y$ is a (positively or negatively, respectively) monotone function of $X$.

# Examples

Here are a few examples (without proofs):

1. When $X$ and $Y$ are normally distributed, we obtain the maximum and minimum when $(X,Y)$ has the usual bivariate normal distribution where $Y$ is written as a linear function of $X$. That is, we get the maximum for $$Y=\mu_Y+\sigma_Y \frac{X-\mu_X}{\sigma_X}.$$ Here the bounds are (of course) $-1$ and $1$, no matter what means and variances $X$ and $Y$ have.

2. When $X$ and $Y$ have lognormal distributions, the lower bound is never attainable, as that would imply that $Y$ could be written $Y=a-bX$ for some $a$ and positive $b$, and $Y$ can never be negative. There exists (slightly ugly) formulas for the exact bounds, but let me just give a special case. When $X$ and $Y$ have standard lognormal distributions (meaning that when exponentiated, they are standard normal), the attainable range is $[-1/e, 1]\approx [-0.37, 1]$. (In general, the upper bound is also restricted.)

3. When $X$ has a standard normal distribution and $Y$ has a standard lognormal distribution, the correlation bounds are $$\pm \frac{1}{\sqrt{e-1}} \approx 0.76.$$

Note that all bounds are for the population correlation. The sample correlation can easily extend outside the bounds, especially for small samples (quick example: sample size of 2).

# Estimating the correlation bounds

It’s actually quite easy to estimate the upper and lower bounds on the correlation if you can simulate from the marginal distributions. For the last example above, we can use this R code:

> n = 10^5      # Sample size: 100,000 observations
> x = rnorm(n)  # From the standard normal distribution
> y = rlnorm(n) # From the standard lognormal distribution
>
> # Estimated maximum correlation
> cor( sort(x), sort(y) )
0.772
>
> # Estimated minimum correlation
> cor( sort(x), sort(y, decreasing=TRUE) )
−0.769


If we only have actual data and don’t know the marginal distributions, we can still use the above method. It’s not a problem that the variables are dependent as long as the observations pairs are dependent. But it helps to have many observation pairs.

# Transforming the data

It is of course possible to transform the data to be (marginally) normally distributed and then calculate the correlation on the transformed data. The problem is one of interpretability. (And why use the normal distribution instead of any other distribution where $Y$ can be a linear function of $X$?) For data that are bivariate normally distributed, the correlation has a nice interpretation (its square is the variance of one variable explained by the other). This is not the case here.

What you’re really doing here is creating a new measure of dependence that does not depend on the marginal distributions; i.e., you are creating a copula-based measure of dependence. There already exists several such measure, Spearman’s ρ and Kendall’s τ being the most well-known. (If you’re really interested in dependence concepts, it’s not a bad idea to look into copulas.)

# In conclusion

Some final thoughts and advice: Just looking at the correlation has one big problem: It makes you stop thinking. Looking at scatter plots, on the other hand, often makes you start thinking. My main advice would therefore be to examine scatter plots and try to model dependence explicitly.

That said, if you need a simple correlation-like measure, I would just use Spearman’s ρ (and associated confidence interval and tests). Its range is not restricted. But be very aware of non-monotone dependence. The Wikipedia article on correlation has a couple of nice plots illustrating potential problems.

• +1 This very nice contribution clearly addresses several recurring issues associated with correlations. I especially appreciate the remarks in the first concluding paragraph about stopping/starting thinking. – whuber Jan 26 '14 at 14:13
• Would the non-robustness remain even asymptotically? If so, is the wiki incorrect in saying that "[The Student's t distribution for a simple transformation of r] also holds approximately even if the observed values are non-normal, provided sample sizes are not very small"? – max Apr 30 '16 at 8:13

What do the distributions of these variables look like (beyond being skewed)? If the only non-normality is skewness, then a transformation of some sort must help. But if these variables have a lot of lumping, then no transformation will bring them to normality. If the variable isn't continuous, the same is true.

How robust is correlation to violations? Take a look at the Anscombe Quartet. It illustrates several problems quite well.

As for other types of analysis, it depends on the analysis. If the skewed variables are independent variables in a regression, for example, there may not be a problem at all - you need to look at the residuals.

• Some of the variables also have problems with kurtosis, but skewness is the biggest problem. I've tried square root and log transformations on the problem variables, but they don't improve by much. In fact, the distributions seem to look almost exactly the same, but with greater pile-up of scores. – Archaeopteryx Sep 29 '12 at 14:53
• That seems very odd. Can you post the mean, median, skewness, kurtosis of the variable in question? Or (even better) a density plot of it? – Peter Flom - Reinstate Monica Sep 29 '12 at 15:14
• Regardless of whether the distribution of (X ,Y) is bivariate normal or not the Pearson correlation is a measure of degree of linearity. The probability distribution for the sample estimate will depend on normality. – Michael R. Chernick Sep 29 '12 at 16:23
• Those variables are not very skew. You can leave them as is. – Peter Flom - Reinstate Monica Sep 29 '12 at 17:06
• Don't worry about significance here. Typically, skew and kurtosis that is < -2 or > 2 is regarded as perhaps needing transformation. Better yet is to look at graphs e.g. quantile normal plot and density plot w/kernel to see what's going on. – Peter Flom - Reinstate Monica Sep 30 '12 at 11:43