2
$\begingroup$

Could one explain the value of correlation in statistics?

If regression is a so advanced technique, why would we even need correlation? Could we also use theoretically use regression for measuring a degree of relationship between two variables? What is so special about correlation? Just a convenient, standardized statistic for showing degree of relationship between two variables?

$\endgroup$
1
  • 1
    $\begingroup$ One place to look for answers is in the literature on observational studies, which often implies all variables are measured with appreciable error. Regression doesn't directly handle that situation. Much can be learned from the questions asked by such disciplines, which include psychology and economics. $\endgroup$
    – whuber
    Jan 8 at 17:21

2 Answers 2

2
$\begingroup$

You’re right: correlation drops out of regression.

If we have just two variables, $X$ and $Y$, we get $cor(X,Y)$ form the square root of the regression $R^2$ and the sign of the slope coefficient.

However, correlation gives an intuition about what a strength of relationship means (particularly since the word has entered into common English), and it skips regression issues that beginning students are unlikely to follow, such as matrix algebra and minimizing the sum of squares (calculus). We don’t need to go that deep to teach correlation to middle school students, who will follow correlation but perhaps not the math of OLS.

Note, however, that an awful lot drops out of regression. Even a t-test can be formulated as a regression on a binary variable, so correlation is not unique in this regard.

(And Spearman correlation drops out of ordinal regression, and the $\chi^2$ test drops out of logistic regression. My professor wasn’t kidding when we said that basically everything in statistics is regression!)

$\endgroup$
4
  • $\begingroup$ There are some set-ups (e.g.) in genetics where correlations are deducible from theory and so empirical correlations are the needed checks on theory, hence directly interesting and informative. There are also many situations where measurement is so arbitrary that regressions aren't really much more informative than correlations. There is much more greyness about the second case than about the first. Further, sometimes one scans lots of correlations to see which regressions are worth looking at in detail; there are many pitfalls there but it can be a practical strategy. $\endgroup$
    – Nick Cox
    Jan 8 at 15:40
  • $\begingroup$ I find correlation difficult to explain to a nontechnical audience without use of regression. Though there are at least "Thirteen Ways to Look at the Correlation Coefficient" (Rodgers & Nicewander, 1988), I find most of them inaccessible for people without a background in mathematics. $\endgroup$ Jan 8 at 16:43
  • $\begingroup$ ("And Spearman correlation drops out of ordinal regression, and the χ2 test drops out of logistic regression." It is not appropriate to assert that these two are drop outs. Correlation theory is based on variance -covariance modeling and not on the regression. $\endgroup$
    – user10619
    Jan 8 at 19:17
  • 2
    $\begingroup$ @SubhashC.Davar The OLS estimate for the slope coefficient is based on those same variances and covariances. $\endgroup$
    – Dave
    Jan 8 at 19:58
1
$\begingroup$

Pearson correlation is

$$ r_{x,y} = \frac{s_{x, y}}{s_x s_y} $$

where $s_x$, $s_y$ are standard deviations and $s_{x, y}$ is covariance. You can estimate the parameter of simple linear regression as

$$ \hat\beta = \frac{s_{x, y}}{s_x^2} = r_{x,y} \frac{s_y}{s_x} $$

Correlation is the standardized regression coefficient so that it ranges from -1 to 1. It's not really about one being better, more advanced, or replacing the other. They are just differently scaled, so in some scenarios, you could prefer one and in some the other. You can also just use the covariance that appears in both.

$\endgroup$
1
  • 1
    $\begingroup$ How does this answer the question of the "value of correlation"? "Because it is bounded" scarcely sounds like a useful criterion for interpretability. Indeed, one could argue the opposite: correlation can be difficult to interpret precisely because it is bounded and therefore not on an interval or ratio scale. It is something more complex than either. $\endgroup$
    – whuber
    Jan 8 at 17:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.