# In Spearman's rank correlation: 4 out of 26 observations totally change the conclusion, what should I do?

After I remove these 4 observations, the results change from weak and not statistically significant to the opposite (rho=0.559 with p<0.01). Should I just remove them or what should I do? Although the Spearman correlation is robust against outliers, can I call these observations outliers? If so, how should I deal with them?

I am calculating the correlation coefficients by using this Excel file, which is taken from this website. Is it accurate?

Here are the data:

X:    6 51  111 123 157 168 195 195 200 205 214 222 229 234 241 256 314 340 352 368 427
431 442 473 501 523 994
Y:    2 16  12  41  28  11  36  13  16  17  15  18  19  33  75  12  75  22  50  5   26
45 14  9   27  9   98

• The file you point to gives a strong and significant result ($\hat\rho = -0.763,\ p=.00023$), which is contrary to your statement. Which data do you consider to be the "outliers" and how did you select them? I cannot see any possible way that removing four of those points would produce a positive correlation coefficient.
– whuber
Feb 13, 2014 at 22:33
• I am not using their data, I am using their file to analyse my own data. can I post my data here? Feb 13, 2014 at 22:37
• 6,2; 51,16; 111,12; 123,41; 157,28; 168,11; 195,36; 195,13; 200,16; 205,17; 214,15; 222,18; 229,19; 234,33; 241,75; 256,12; 314,75; 340,22; 352,50; 368,5; 427,26; 431,45; 442,14; 473,9; 501,27; 523,9; 994,98; Feb 13, 2014 at 22:48
• With 26 points, and the ability to choose 4, you could radically alter the measure of an effect (and often, the conclusion of a test) for almost any statistic. Feb 14, 2014 at 0:13
• This phenomenon has little or nothing to do with Spearman's $\rho$: given any dataset of 26 cases that exhibits any variation at all, I can substantially change the analysis if you allow me to remove four of them at will. Spearman's $\rho$ is robust against wild changes in a few of the values in the dataset, but not against motivated deletion of data!
– whuber
Feb 14, 2014 at 14:48

The phenomenon you observe is caused by the use of Spearman's $\rho$ coefficients, which has the undesirable property that it is easily swayed by the presence in the sample of even a few observations that happen to depart from the bivariate ellipse fitting of the bulk of the data.

The general solution to this problem is to replace the Spearman $\rho$ by a robust alternative. In a nutshell, robust estimation procedures were designed to fit a model that is cannot be easily swayed by a handful of non-conforming data-points.

A popular such algorithm is FastMCD [1,2]. The aim of FMCD is to find the model that best fits the central observations in your dataset. Observations that are far from the FMCD fit are flagged as outliers. They do not part take in the estimation of the coefficients fitted by FMCD. They should be modelled separately from the remaining bulk of the data: their very distance from the FMCD fit makes them unduly influential on any classical statistical analysis that would include them alongside the rest of the data-set.

Now, I turn to your specific problem. Here is the code to run FMCD on your dataset:

X <- c(6, 51,  111, 123, 157, 168, 195, 195, 200, 205, 214, 222, 229, 234, 241, 256,
314, 340, 352, 368, 427, 431, 442, 473, 501, 523, 994)
Y <- c(2, 16,  12,  41,  28,  11,  36,  13, 16, 17, 15, 18, 19, 33, 75, 12,
75, 22, 50, 5,  26, 45, 14, 9, 27, 9, 98)
library("rrcov")
library("fBasics")
a1 <- CovMcd(cbind(X, Y), nsamp="exact")
#Spearman rho on the observation not classified as outliers:
spearmanTest(X[a1@raw.wt==1], Y[a1@raw.wt==1])


These observations row index: (2,3,6,8,9,10,11,12,13,16,18,20,23,24,26) are, in the sense of FMCD, those most central point in your dataset. Furthermore, the following observations: (4,7,15,17,19,22,27) have been identified by FMCD to be highly influential: they stand so far out from the model fitting the bulk of the data that they, if left unchecked, will (either alone, or together) single handedly sway the resulting fit. Finally, the Spearman $\rho$ of the observations not flagged as outliers by FMCD is now $\approx0.07$ (p=0.77).

In short, the main advantage of the FMCD fit lies in its resilience (the technical term is robustness) to the contamination of the sample by a few inconsistent data points. For example, it would be very difficult to find 4 points such that by removing them will significantly alter the solution found by FMCD.

# EDIT:

As suggested by @amoeba I'm illustrating the resilience of the FMCD to the removal of 4 data points from the set (those with indexes 20 23 24 26, these were picked by @amoeba, see comments below). Looking at the estimation results obtained re-running FMCD on this amputated data-set, 14 of the 16 observations not classified as outliers by the FMCD fit of the original data-set (and not subsequently removed from the data-set) still are classified as such. The Spearman $\rho$ of the good data went from $0.07$ (p=0.77) on the original data to $0.21$ (p=0.43) on the data set with the four observations selected by @amoeba removed (so the conclusions of the analysis remain the same).

@whuber suggested plotting a scatterplot of the rank of the data to better show the effect of the outliers. The observations flagged as outliers by FMCD are shown as red dots.

@Thomas Speidel. Spearman's $\rho$ has good robustness property, so long as the rate of contamination of the sample is smaller than 5%. For a more in depth (including formal) study of these issues, see [ 3]:

1. P. J. Rousseeuw and K. van Driessen (1999) A fast algorithm for the minimum covariance determinant estimator. Technometrics 41, 212–223.
2. Ricardo A. Maronna, Douglas R. Martin, Victor J. Yohai (2006). Robust Statistics: Theory and Methods (Wiley Series in Probability and Statistics).
3. Christophe Croux & Catherine Dehon, 2010. "Influence functions of the Spearman and Kendall correlation measures," Statistical Methods and Applications, vol. 19(4), pp 497--515.
• Isn't Spearman's correlation supposed to be robust already? Are you sure that the red ellipse on your plot corresponds to Spearman's (rank) correlation, and not to the Pearson's one? Feb 14, 2014 at 16:02
• @amoeba: if you can sway it by removing 4 points, it ain't robust. As written above, the red ellipse in the plot corresponds to the MLE fit. Feb 14, 2014 at 16:03
• Why not just plot the scatterplot of the ranks of the data? Its correlation coefficient is what Spearman's $\rho$ measures.
– whuber
Feb 14, 2014 at 17:06
• whuber, amoeba: I've edited the post to re-center it on the Spearman $\rho$. Hopefully, it now better addresses the question of the O.P. @amoeba, the p-vals are n.s. in both cases. Feb 14, 2014 at 17:44
• I thought the whole point of using ranks (Spearman) was to increase robustness. Also, this could be a good exploratory tool to identify suspicious observations. The next step is to use substantive knowledge to investigate those points before a call can be made on modelling things separately, exclude them or use other robust methods. For some robust suggestions, I higly suggest Randy Wilcox "Fundamentals of Modern Statistical Methods". Feb 14, 2014 at 17:45