2
$\begingroup$

I am trying to interpret the sign of my 5 x-variables against y-variable. The sign of some coefficients in the regression output (command: reg) are different than the signs under correlation matrix (command: correlate). Which one defines the relation sign/direction?

$\endgroup$

marked as duplicate by gung regression Aug 21 '15 at 20:37

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

  • $\begingroup$ Can you please add the statements as a code block. Would help in understanding the question better. $\endgroup$ – Dawny33 Aug 21 '15 at 17:11
  • $\begingroup$ In the regression output under the column Coef, it shows -58.17107 for x1. When I ran correlate command to get the correlation matrix, I get 0.8592 correlation value between x1 and y. This difference in signs is confusing. $\endgroup$ – Asaad Aug 21 '15 at 17:49
  • 2
    $\begingroup$ Do a search on Simpson's Paradox. This is a fairly famous statistical concern. $\endgroup$ – DWin Aug 21 '15 at 19:02
  • $\begingroup$ This page is a good introduction to these issues in the context of logistic regression; the essential issues in terms of independent variables are the same in linear regression. $\endgroup$ – EdM Aug 21 '15 at 20:14
  • 1
    $\begingroup$ I believe you will get the understanding you need in the linked thread. Please read it. If you still have a question afterwards, come back here & edit your Q to state what you've learned & what you still don't understand. Then we can provide the information you need without simply duplicating material elsewhere that already didn't help you. $\endgroup$ – gung Aug 21 '15 at 20:40
3
$\begingroup$

The correlation coefficient is the unconditional simple relationship between Y and x1 . The regression coefficient is the conditional impact on Y GIVEN x2,x3,x4,and x5. They would have the same sign if x1,x2,x3,x4,x5 were orthogonal (uncorrelated) with each other. As @Whuber correctly reflected, if they are not orthogonal they may or may not have signs that are the same.

EXPANDED ANSWER TO MAKE IT MORE CONSTRUCTIVE:

Consider the following data set where the simple correlation between Y and X1 is positive enter image description here Let is analyze Y as a function of X1 and X2enter image description here . First separately then together. The regression coefficient of the unconditional relationship between Y and M_X1 is .463 ( same sign as the simple correlation ) while the conditional impact of M_X1 given M_X2 is -1.222 (different sign ). As others have pointed out this is sometimes referred to as " the expected sign fallacy ', "Simpson's paradox" , "Ecological fallacy " etc. In summary conditional analysis (multiple regression) is often different from unconditional analysis (simple regression) because the predictor variables are not independent of each other.

$\endgroup$
  • $\begingroup$ This is very helpful. However, the correlation values are different we I run it for all or individually. I mean correlation values between command: correlate y x1 x2 x3 x4 x5 and command: correlate y with each x individually. $\endgroup$ – Asaad Aug 22 '15 at 10:03
  • $\begingroup$ Multivariate regression coefficient: indicates the change in the dependent variable associated with a one-unit increase in the independent variable in question holding constant the other independent variables in the equation.If the independent variable affects one of the other independent variables the resultant value of the dependent variable Y might be different. $\endgroup$ – IrishStat Aug 22 '15 at 11:55
  • $\begingroup$ If you like my answer then up-vote it and select it as your answer. In this way new readers will play close attention to my answer and consequently I will have helped to save the planet because your question is "everybody's question" and is not a repeat AFAI am concerned.. $\endgroup$ – IrishStat Aug 22 '15 at 13:53
  • $\begingroup$ Great explanation! An eye-opener! Thank you! $\endgroup$ – Antoni Parellada Aug 24 '15 at 17:31

Not the answer you're looking for? Browse other questions tagged or ask your own question.