Maybe this question is naive, but:
If linear regression is closely related to Pearson's correlation coefficient, are there any regression techniques closely related to Kendall's and Spearman's correlation coefficients?
$\begingroup$
$\endgroup$
3
-
3$\begingroup$ As a simple example where you have one explanatory and a dependent variable: A linear regression of the ranks of $x$ and $y$ would yield Spearman's correlation coefficient as regression coefficient. And in this case, $x$ and $y$ are interchangeable in the regression. $\endgroup$– COOLSerdashCommented Jul 20, 2013 at 9:28
-
2$\begingroup$ Just a few thoughts. Kendall's $\tau$ and Spearman's $\rho$ are both correlation coefficients based on ranks. The sought after relationship between $x$ and $y$ would then need to involve their ranks. However, computing the ranks introduces dependence between the observations, which in turn imposes dependence between the error terms, eliminating linear regression. However, in a different setting, modeling the dependence structure between $x$ and $y$ with copulas would make a link with Kendall's $\tau$ and/or Spearman's $\rho$ possible, depending on the choice of copula. $\endgroup$– QuantIbexCommented Jul 20, 2013 at 10:35
-
1$\begingroup$ @QuantIbex does that dependence necessarily imply $E[\varepsilon_i\varepsilon_j]\neq 0$? $\endgroup$– shadowtalkerCommented Jul 31, 2014 at 3:07
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
2
There's a very straightforward means by which to use almost any correlation measure to fit linear regressions, and which reproduces least squares when you use the Pearson correlation.
Consider that if the slope of a relationship is $\beta$, the correlation between $y-\beta x$ and $x$ should be expected to be $0$.
Indeed, if it were anything other than $0$, there'd be some uncaptured linear relationship - which is what the correlation measure would be picking up.
We might therefore estimate the slope by finding the slope, $\tilde{\beta}$ that makes the sample correlation between $y-\tilde{\beta} x$ and $x$ be $0$. In many cases -- e.g. when using rank-based measures -- the correlation will be a step-function of the value of the slope estimate, so there may be an interval where it's zero. In that case we normally define the sample estimate to be the center of the interval. Often the step function jumps from above zero to below zero at some point, and in that case the estimate is at the jump point.
This definition works, for example, with all manner of rank based and robust correlations. It can also be used to obtain an interval for the slope (in the usual manner - by finding the slopes that mark the border between just significant correlations and just insignificant correlations).
This only defines the slope, of course; once the slope is estimated, the intercept can be based on a suitable location estimate computed on the residuals $y-\tilde{\beta}x$. With the rank-based correlations the median is a common choice, but there are many other suitable choices.
Here's the correlation plotted against the slope for the car data in R:
The Pearson correlation crosses 0 at the least squares slope, 3.932
The Kendall correlation crosses 0 at the Theil-Sen slope, 3.667
The Spearman correlation crosses 0 giving a "Spearman-line" slope of 3.714
Those are the three slope estimates for our example. Now we need intercepts. For simplicity I'll just use the mean residual for the first intercept and the median for the other two (it doesn't matter very much in this case):
intercept
Pearson: -17.573 *
Kendall: -15.667
Spearman: -16.285
*(the small difference from least squares is due to rounding error in the slope estimate; no doubt there's similar rounding error in the other estimates)
The corresponding fitted lines (using the same color scheme as above) are:
Edit: By comparison, the quadrant-correlation slope is 3.333
Both the Kendall correlation and Spearman correlation slopes are substantially more robust to influential outliers than least squares. See here for a dramatic example in the case of the Kendall.
-
$\begingroup$ (+1) Great explanation! Is there any reason why Kendall seems to be more preferred to Spearman in this context (at least judging from the fact that Kendall correlation corresponds to a slope estimator that has a name, Theil-Sen, whereas Spearman one does not)? $\endgroup$– amoebaCommented Jul 31, 2014 at 10:45
-
5$\begingroup$ There are a number of reasons why this seems to be the case. First is that the Theil-Sen line has a simply-described estimator (median of the pairwise slopes), which the Spearman lacks; in small samples it's very suitable for hand calculation. The Kendall correlation approaches normality faster and is more mathematically tractable. See also here and here. $\endgroup$– Glen_bCommented Jul 31, 2014 at 11:35
$\begingroup$
$\endgroup$
The proportional odds (PO) model generalizes Wilcoxon and Kruskal-Wallis tests. Spearman's correlation when $X$ is binary is the Wilcoxon test statistic simply translated. So you could say that the PO model is a unifying method. Since the PO model can have as many intercepts as there are unique values of $Y$ (less one), it handles both ordinal and continuous $Y$.
The numerator of the score $\chi^2$ statistic in the PO model is exactly the Wilcoxon statistic.
The PO model is a special case of a more general family of cumulative probability (some call cumulative link) models including the probit, proportional hazards, and complementary log-log models. For a case study see Chapter 15 of my Handouts.
answered Jul 20, 2013 at 12:42
$\begingroup$
$\endgroup$
0
Aaron Han (1987 in econometrics) proposed the Maximum Rank Correlation estimator that fits regression models by maximizing tau. Dougherty and Thomas (2012 in the psychology literature) recently proposed a very similar algorithm. There is an abundance of work on the MRC illustrating its properties.
Aaron K. Han, Non-parametric analysis of a generalized regression model: The maximum rank correlation estimator, Journal of Econometrics, Volume 35, Issues 2–3, July 1987, Pages 303-316, ISSN 0304-4076, http://dx.doi.org/10.1016/0304-4076(87)90030-3. (http://www.sciencedirect.com/science/article/pii/0304407687900303)
Dougherty, M. R., & Thomas, R. P. (2012). Robust decision making in a nonlinear world. Psychological review, 119 (2), 321. Retrieved from http://damlab.umd.edu/pdf%20articles/DoughertyThomas2012Rev.pdf.
