Is it correct to claim that a correlation between two variables indicates a potential main effect in explaining one of the variables?

  • 1
    $\begingroup$ Could you be more specific? What do you mean by "potential main effect"? What would it mean that correlation between age and salasy would be equal to 0.55 ? $\endgroup$ – Tim Jan 23 '18 at 14:22
  • $\begingroup$ @Tim I actually worked on the potential solution, could you unclose the question? $\endgroup$ – son520804 Jan 23 '18 at 14:34
  • $\begingroup$ @son520804 ok, but I don't think this is answerable in such form, since the question is very unclear. $\endgroup$ – Tim Jan 23 '18 at 14:38
  • $\begingroup$ @Tim I actually agree on that but I know the feeling that the person who asks the question may feel really confused with the concept too. It was the same when I was in undergrad. $\endgroup$ – son520804 Jan 23 '18 at 14:40

Well, correlation is a wide topic. Generally, we define correlation of two random variables X and Y as

$$\rho = corr(X,Y) = \frac{cov(X,Y)}{\sqrt{Var(X}\sqrt{Var(Y)}}$$ By Cauchy-Schwarz inequality, $cov(X,Y) = E\Big[(X-E(X))(Y-E(Y))\Big] \le \sqrt{Var(X)Var(Y)}=\sqrt{E\Big[(X-E(X))^2\Big]E\Big[(Y-E(Y))^2\Big]}$, so, $-1\le\rho \le 1$.

In linear model, we say that the two predictors (for example we use education level and working experience to predict the response salary) are positively correlated if $\rho > 0$, while negatively correlated if $\rho < 0$.

Special Case 1: If the correlation is 0, we may say the predictors are uncorrelated, but be very careful that uncorrelated does not necessarily means independent. However, for normally distributed data (linear regression assumes the error term is of $N(0,\sigma^2)$ distribution), it has the nice property that uncorrelated implies independent.

Special Case 2: If the correlation is 1 or -1, then we say the two variables are perfectly correlated, so we can say that $X$ is a linear combination of $Y$, i.e., $X= \lambda Y$ for some real value $\lambda$.

Finally, for edification only, for data reduction, we typically look for the variables having great correlation in the correlation matrix, since it may represent high collinearity. This is the starting point to do PCA for example.

So, you may consult books, Wiki, or other posts for more deduction of proofs and specific information. Hope it helps.

  • $\begingroup$ With respect to your example, does a correlation between working experience and salary thus indicate that working experience may predict salary? $\endgroup$ – TimH Jan 23 '18 at 14:53
  • $\begingroup$ I said education level and working experience predict the salary, so the correlation here stands for the correlation between working experience and education level. If correlation is 0, then this two predictors are independent (assume to be normal), and if correlation is 1 or -1, then this two predictors can be combined as 1 predictor. $\endgroup$ – son520804 Jan 23 '18 at 14:57
  • $\begingroup$ I get that point. What does a correlation between the predictor and outcome tell me though? $\endgroup$ – TimH Jan 23 '18 at 15:00
  • $\begingroup$ Good point. If you are performing linear regression, sometimes you may not specify which variable being the outcome, so it is still possible to model the correlation for all variables. The same interpretation above applies as well. $\endgroup$ – son520804 Jan 23 '18 at 15:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.