A correlation test and a simple linear regression test both answer the same question, which is

Is there a good chance that my data comes from a population where the correlation is positive rather than negative / where the regression line has a positive slant rather than negative?

Both are based on the Least Squares method, and output the same p-value. They're basically the same test with a slightly different wording of the interpretation (also apart from the fact that Linear Regression allows you to make predictions which correlation alone can't).

I would therefore expect similar assumptions in both tests. Yet normality of variables is a standard requirement in the correlation test, and is not required in the linear regression test.

Why is that?

  • $\begingroup$ Note that the linear regression test assumes the $x$ to be fixed, so you're right that the requirements are not the same (which is remarkable enough),but it's not the case that the linear regression test requires less. It requires something that looks different at first sight. Oh, and the regression test assumes the errors to be normal (unless asymptotic arguments are used), though not both variables. $\endgroup$ Feb 10 '20 at 15:24
  • $\begingroup$ @Lewian Yes, it is remarkable that the regression line is not the same if $x$ and $y$ are switched, although the $R^2$ will be the same. So you're saying that somehow the assumptions are actually equivalent for the two tests? $\endgroup$ Feb 10 '20 at 15:40
  • $\begingroup$ No, they're certainly not. Nice question. $\endgroup$ Feb 10 '20 at 15:54

It depends which "tests" you are using, and what theory was used originally to develop them. I think a few people find it a bit confusing that you can start off in different places, using very different theories and yet end up with a set of working / fitting computations that look exactly the same.

For linear regression fitted by Ordinary Least Squares (OLS) the Gauss Markov theorem tells us (for example) that you get the Best Linear Unbiased Estimators as a simple geometric feature of the data. The only assumptions you need to make are that the errors are zero mean, constant variance and independent. I think the Central Limit theorem then lets you develop an estimator for your slope, which has a t-distribution and doesn't require a Normality assumption on the errors. You could however ignore the whole OLS aspect of regression and develop a maximum likelihood estimator, which would require that you also assume that the errors were Normal (zero mean, constant variance and independent). Interestingly, you solve the likelihood equations the same way you solve the OLS problem. So I suppose you could make either set of assumptions (with or without Normality), depending which theory you wanted to believe, but you'd still compute the answer the same way. Offhand, I don't know how the correlation "test" was developed, but I guess the theoretical framework used to develop the test you are using required Normality assumptions.

  • $\begingroup$ Thank you for your answer. I realize that I could have been more precise, so I edited the question to indicate that indeed I am considering both tests based on the OLS method. $\endgroup$ Feb 10 '20 at 15:06
  • $\begingroup$ As far as I know, the t-distributions of the test statistics for both the standard correlation test under bivariate normal assumption, and the test for testing a simple linear regression coefficient to be zero under normal assumption for errors only given fixed $x$-values are both precise, so the answer to the original question has nothing to do with the central limit theorem, I'd think. $\endgroup$ Feb 10 '20 at 15:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.