27
$\begingroup$

Apparently Pearson's correlation coefficient is parametric and Spearman's rho is non-parametric.

I'm having trouble understanding this. As I understand it Pearson is computed as $$ r_{xy} = \frac{cov(X,Y)}{\sigma_x\sigma_y} $$ and Spearman is computed in the same way, except we substitute all values with their ranks.

Wikipedia says

The difference between parametric model and non-parametric model is that the former has a fixed number of parameters, while the latter grows the number of parameters with the amount of training data.

But I do not see any parameters except for the samples themselves. Some say that parametric tests assume normal distributions and go on to say that Pearson does assume normal distributed data, but I fail to see why Pearson would require that.

So my question is what do parametric and non-parametric mean in the context of statistics? And how do Pearson and Spearman fit in there?

Improve this question
$\endgroup$
6
  • 8
    $\begingroup$ It's a good question and there is an awful lot of misinformation out there. For example, the equation of parametric tests and assuming normal distributions is unfortunately a frequent confusion, whereby many textbook writers, course teachers and internet posters just copy from others who are as or more confused. $\endgroup$
    – Nick Cox
    Commented Mar 17, 2015 at 11:28
  • 7
    $\begingroup$ Perhaps the simplest positive resolution of the question is this: yes, Spearman's correlation is a parameter to be estimated quantifying strength of a relationship and so resembles Pearson (at root, it's the same idea, as you point out); but no, Spearman's correlation is not a parameter that features in a distribution, whereas Pearson's is a parameter in a bivariate normal distribution (a historic but now downplayed interpretation of what you are doing when you do correlation). It's a fine distinction, to be understood by seeing that the word "parameter" has multiple senses. $\endgroup$
    – Nick Cox
    Commented Mar 17, 2015 at 11:28
  • $\begingroup$ @NickCox, why don't you post that as an answer. $\endgroup$
    – Richard Hardy
    Commented Mar 17, 2015 at 11:37
  • 5
    $\begingroup$ The point about normality of distribution only really bites when you want to do significance tests with correlation. If you use correlations only as descriptive measures, non-normality need not be a barrier to using correlations. Correlations can even be a little useful with two binary variables so long as both do vary. You still need to watch out for the effects of outliers, etc., etc. $\endgroup$
    – Nick Cox
    Commented Mar 17, 2015 at 11:37
  • 1
    $\begingroup$ Since it hasn't seem to have been clearly said yet, I would like to emphasize that no statistic is "parametric." That's like saying numbers are tasty: the adjective simply does not apply to the noun. Statistical models can be parametric (as indicated by the Wikipedia quotation), as well as the tests and procedures that are based on them. The Spearman and Pearson statistics can be used in both parametric and non-parametric settings. More on this at stats.stackexchange.com/questions/67204. What makes a model parametric is its state space. $\endgroup$
    – whuber ♦
    Commented Mar 21, 2015 at 0:05

3 Answers 3

Reset to default
17
$\begingroup$

The problem is that "nonparametric" really has two distinct meanings these days. The definition in Wikipedia applies to things like nonparametric curve fitting, eg via splines or local regression. The other meaning, which is older, is more along the lines of "distribution-free" -- that is, techniques that can be applied regardless of the assumed distribution of the data. The latter is the one that applies to Spearman's rho, since the rank-transformation implies it will give the same result no matter what your original distribution was.

Improve this answer
$\endgroup$
5
  • 2
    $\begingroup$ Nonparametric does have two meanings, but the comment in wikipedia really applies to both. In nonparametric regression it refers to the relationship not being finite-parametric. In the 'distribution-free'-side of things it refers to distributional models not being finite-parametric. $\endgroup$
    – Glen_b
    Commented Mar 20, 2015 at 10:43
  • 1
    $\begingroup$ Hm, that's quote from Wikipedia isn't me. Someone else has added it. $\endgroup$
    – Hong Ooi
    Commented Mar 20, 2015 at 15:55
  • 2
    $\begingroup$ The main edit - which I believe is incorrect in one detail and doesn't add anything especially useful - came up for review since it was made by a low=rep user, and was rejected by one person, but then was auto-accepted when a third person tried to edit to improve it (they may not have realized that this would be a consequence). I'm going to roll that edit back to your original. You can do that any time there's an edit you don't like. $\endgroup$
    – Glen_b
    Commented Mar 20, 2015 at 22:26
  • $\begingroup$ Now rolled back to your original post, since I think it changed your post too much without seeking your agreement and doesn't sound like you agree with it. If there was anything you liked about it, click the "edited ... ago" link above my name and copy what parts you like from what was there before, then edit and paste it in. $\endgroup$
    – Glen_b
    Commented Mar 20, 2015 at 22:32
  • $\begingroup$ When is it justified to use Spearman? How can Pearson help when you use Spearman? $\endgroup$
    – Léo Léopold Hertz 준영
    Commented Nov 11, 2016 at 8:52
3
$\begingroup$

I think the only reason why Pearson's correlation coefficient would be called parametric is because you can use it to estimate the parameters of the multivariate normal distribution. for instance, bivariate normal distribution has 5 parameters: two means, two variances and the correlation coefficient. The latter can be estimated with Pearson correlation coefficient.

Otherwise, you're absolutely right, in order to compute Pearson $\rho$ you don't need to make any distributional assumptions. It's just when you assume normal distribution, the Pearson correlation has additional meanings as opposed to Spearman or Kendall.

Improve this answer
$\endgroup$
3
  • $\begingroup$ isn't pearson's correlation coefficient parameter in the sense that you have to assume normality to test its significance? that is, it doesn't assume normality as a statistic, but you assume that the data is normal when computing the distribution of the sample correlation coefficient and test it? this is an honest question, I could be 100% wrong. $\endgroup$
    – mugen
    Commented Mar 18, 2015 at 1:12
  • $\begingroup$ Can you explain please if you do any distribution assumptions in sperman and kendall? $\endgroup$
    – Léo Léopold Hertz 준영
    Commented Nov 11, 2016 at 8:46
  • $\begingroup$ @mugen you don't have to assume normality to test the significance of a Pearson correlation; a common test of a Pearson correlation does so. You could make a different parametric assumption and come up with a different test ... or indeed, one could perform a permutation test of the null that the population Pearson correlation is zero, resulting in a nonparametric test. $\endgroup$
    – Glen_b
    Commented Dec 9, 2016 at 7:25
1
$\begingroup$

Simplest answer I think is that Spearmen's rho test uses ordinal data (numbers that can be ranked but don't tell you anything about the interval between the numbers e.g. 3 flavours of ice cream are ranked 1, 2 and 3 but this only tells you which flavour was preferred not how much by). Ordinal data cannot be used in parametric tests.

Pearson's r test uses interval or ratio data (numbers that have fixed intervals e.g. seconds, kg, mm). 1mm is not only smaller than 5mm but you know exactly how much by. this type of data can be used in a parametric test.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ Certainly it's possible to use parametric models - and hence parametric tests - with ordinal data. One need simply propose a distribution for this variable with a finite - and fixed - number of parameters, and some suitable hypothesis in relation to those parameters and voila, a parametric test exists. The Pearson correlation calculated in situations where one or both the variables have two categories (labelled with two different numbers, typically 0/1) result in commonly used measures of association for those situations. $\endgroup$
    – Glen_b
    Commented Dec 9, 2016 at 4:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.