105
$\begingroup$

In particular, I am referring to the Pearson product-moment correlation coefficient.

Improve this question
$\endgroup$
1

9 Answers 9

Reset to default
126
$\begingroup$

What's the difference between the correlation between $X$ and $Y$ and a linear regression predicting $Y$ from $X$?

First, some similarities:

  • the standardised regression coefficient is the same as Pearson's correlation coefficient
  • The square of Pearson's correlation coefficient is the same as the $R^2$ in simple linear regression
  • The sign of the unstandardized coefficient (i.e., whether it is positive or negative) will the same as the sign of the correlation coefficient.
  • Neither simple linear regression nor correlation answer questions of causality directly. This point is important, because I've met people that think that simple regression can magically allow an inference that $X$ causes $Y$.
  • Standard tests of the null hypothesis (i.e., "correlation = 0" or, equivalently, "slope = 0" for the regression in either order), such as carried out by lm and cor.test in R, will yield identical p-values.

Second, some differences:

  • The regression equation (i.e., $a + bX$) can be used to make predictions on $Y$ based on values of $X$
  • While correlation typically refers to the linear relationship, it can refer to other forms of dependence, such as polynomial or truly nonlinear relationships
  • While correlation typically refers to Pearson's correlation coefficient, there are other types of correlation, such as Spearman's.
  • The correlation between X and Y is the same as the correlation between Y and X. In contrast, the unstandardized coefficient typically changes when moving from a model predicting Y from X to a model predicting X from Y.
Improve this answer
$\endgroup$
5
  • $\begingroup$ Hi Jeromy, thank you for your explaination, but I still have a question here: What if I don not need to make predictions and just want to know how close two variable are and in which direction/strength? Is there still a different using these two technique? $\endgroup$
    – yue86231
    Commented Jun 15, 2014 at 22:15
  • 3
    $\begingroup$ @yue86231 Then it sounds like a measure of correlation would be more appropriate. $\endgroup$
    – Jeromy Anglim
    Commented Jun 16, 2014 at 2:40
  • 9
    $\begingroup$ (+1) To the similarities it might be useful to add that standard tests of the hypothesis "correlation=0" or, equivalently, "slope=0" (for the regression in either order), such as carried out by lm and cor.test in R, will yield identical p-values. $\endgroup$
    – whuber ♦
    Commented Jan 19, 2017 at 0:02
  • 4
    $\begingroup$ I agree that the suggestion from @whuber should be added, but at a very basic level I think it is worth pointing out that the sign of the regression slope and the correlation coefficient are equal. This is probably one of the first things most people learn about the relationship between correlation and a "line of best fit" (even if they don't call it "regression" yet) but I think it's worth noting. To the differences, the fact that you get the same answer correlation X with Y or vice versa, but that the regression of Y on X is different to that of X on Y, might also merit a mention. $\endgroup$
    – Silverfish
    Commented Jul 27, 2017 at 23:50
  • 2
    $\begingroup$ @whuber and silverfish: 5 years later, I've added your points :-) $\endgroup$
    – Jeromy Anglim
    Commented Nov 16, 2022 at 1:20
37
$\begingroup$

Here is an answer I posted on the graphpad.com website:

Correlation and linear regression are not the same. Consider these differences:

  • Correlation quantifies the degree to which two variables are related. Correlation does not fit a line through the data.
  • With correlation you don't have to think about cause and effect. You simply quantify how well two variables relate to each other. With regression, you do have to think about cause and effect as the regression line is determined as the best way to predict Y from X.
  • With correlation, it doesn't matter which of the two variables you call "X" and which you call "Y". You'll get the same correlation coefficient if you swap the two. With linear regression, the decision of which variable you call "X" and which you call "Y" matters a lot, as you'll get a different best-fit line if you swap the two. The line that best predicts Y from X is not the same as the line that predicts X from Y (unless you have perfect data with no scatter.)
  • Correlation is almost always used when you measure both variables. It rarely is appropriate when one variable is something you experimentally manipulate. With linear regression, the X variable is usually something you experimentally manipulate (time, concentration...) and the Y variable is something you measure.
Improve this answer
$\endgroup$
2
  • 14
    $\begingroup$ "the best way to predict Y from X" has nothing to do with cause and effect: X could be the cause of Y or vice versa. One can reason from causes to effects (deduction) or from effects to causes (abduction). $\endgroup$
    – Neil G
    Commented Nov 20, 2013 at 7:07
  • 5
    $\begingroup$ "you'll get a different best-fit line if you swap the two" is a little misleading; the standardized slopes will be the same in both cases. $\endgroup$
    – xenocyon
    Commented Jun 1, 2017 at 0:36
27
$\begingroup$

In the single predictor case of linear regression, the standardized slope has the same value as the correlation coefficient. The advantage of the linear regression is that the relationship can be described in such a way that you can predict (based on the relationship between the two variables) the score on the predicted variable given any particular value of the predictor variable. In particular one piece of information a linear regression gives you that a correlation does not is the intercept, the value on the predicted variable when the predictor is 0.

In short - they produce identical results computationally, but there are more elements which are capable of interpretation in the simple linear regression. If you are interested in simply characterizing the magnitude of the relationship between two variables, use correlation - if you are interested in predicting or explaining your results in terms of particular values you probably want regression.

Improve this answer
$\endgroup$
4
  • $\begingroup$ "In particular one piece of information a linear regression gives you that a correlation does not is the intercept"... Very much of difference! $\endgroup$
    – Learner
    Commented Feb 17, 2016 at 8:42
  • $\begingroup$ Well, looking back on that, it is only true that the regression provides an intercept is because it is the default for many stats packages to do so. One could easily calculate a regression without an intercept. $\endgroup$
    – russellpierce
    Commented Feb 18, 2016 at 18:50
  • $\begingroup$ Yes, one could easily calculate a regression without an intercept but it would seldom be meaningful: stats.stackexchange.com/questions/102709/… $\endgroup$
    – kjetil b halvorsen ♦
    Commented Oct 10, 2017 at 12:21
  • 1
    $\begingroup$ @kjetilbhalvorsen Except as in the case I've described when you are fitting a standardized slope. The intercept term in a standardized regression equation is always 0. Why? Because both the IV and DVs have been standardized to unit scores - as a result the intercept is definitionally 0. Exactly the kind of case you describe in your answer. (the equivalent to standardizing the the IV and the DV). When both IV and DV have been standardized to 0, the intercept is definitionally 0. $\endgroup$
    – russellpierce
    Commented Oct 13, 2017 at 15:27
17
$\begingroup$

All of the given answers so far provide important insights but it should not be forgotten that you can transform the parameters of one into the other:

Regression: $y = mx + b$

Connection between regression parameters and correlation, covariance, variance, standard deviation and means: $$m = \frac{Cov(y, x)}{Var(x)} = \frac{Cor(y, x) \cdot Sd(y)}{Sd(x)}$$ $$b = \bar{y}-m\bar{x}$$

So you can transform both into each other by scaling and shifting their parameters.

An example in R:

y <- c(4.17, 5.58, 5.18, 6.11, 4.50, 4.61, 5.17, 4.53, 5.33, 5.14)
x <- c(4.81, 4.17, 4.41, 3.59, 5.87, 3.83, 6.03, 4.89, 4.32, 4.69)
lm(y ~ x)
## 
## Call:
## lm(formula = y ~ x)
## 
## Coefficients:
## (Intercept)            x  
##      6.5992      -0.3362
(m <- cov(y, x) / var(x)) # slope of regression
## [1] -0.3362361
cor(y, x) * sd(y) / sd(x) # the same with correlation
## [1] -0.3362361
mean(y) - m*mean(x)       # intercept
## [1] 6.599196
Improve this answer
$\endgroup$
11
$\begingroup$

Correlation analysis only quantifies the relation between two variables ignoring which is dependent variable and which is independent. But before appliyng regression you have to calrify that impact of which variable you want to check on the other variable.

Improve this answer
$\endgroup$
3
$\begingroup$

From correlation we can only get an index describing the linear relationship between two variables; in regression we can predict the relationship between more than two variables and can use it to identify which variables x can predict the outcome variable y.

Improve this answer
$\endgroup$
0
3
$\begingroup$

Quoting Altman DG, "Practical statistics for medical research" Chapman & Hall, 1991, page 321: "Correlation reduces a set of data to a single number that bears no direct relation to the actual data. Regression is a much more useful method, with results which are clearly related to the measurement obtained. The strength of the relation is explicit, and uncertainty can be seen clearly from confidence intervals or prediction intervals"

Improve this answer
$\endgroup$
1
  • 3
    $\begingroup$ Although I am sympathetic with Altman--regression methods often are more suitable than correlation in many cases--this quotation is setting up a straw man argument. In OLS regression the information produced is equivalent to that afforded by the information that goes into a correlation calculation (all first and second bivariate moments and their standard errors) and the correlation coefficient provides the same information as the regression slope. The two approaches differ somewhat in the underlying data models they assume and in their interpretation, but not in the ways claimed by Altman. $\endgroup$
    – whuber ♦
    Commented Aug 20, 2014 at 13:20
1
$\begingroup$

The regression analysis is a technique to study the cause of effect of a relation between two variables. whereas, The correlation analysis is a technique to study the quantifies the relation between two variables.

Improve this answer
$\endgroup$
1
  • 7
    $\begingroup$ Welcome to CV! Given that there are so many answers to this question already, do you want to have a look at them & see if yours adds anything new? If you've more to say, you can edit it to do so. $\endgroup$
    – Scortchi - Reinstate Monica ♦
    Commented Oct 22, 2014 at 17:08
-1
$\begingroup$

Correlation is an index (just one number) of the strength of a relationship. Regression is an analysis (estimation of parameters of a model and statistical test of their significance) of the adequacy of a particular functional relationship. The size of the correlation is related to how accurate the predictions of the regression will be.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ No it's not. Correlation gives us a bounded relationship but it doesn't relate to how accurate the predictions could be. R2 gives that. $\endgroup$
    – SmallChess
    Commented Aug 6, 2015 at 15:58

Not the answer you're looking for? Browse other questions tagged or ask your own question.