I am trying to interpret the sign of my 5 x-variables against y-variable. The sign of some coefficients in the regression output (command: reg) are different than the signs under correlation matrix (command: correlate). Which one defines the relation sign/direction?
5
-
$\begingroup$ Can you please add the statements as a code block. Would help in understanding the question better. $\endgroup$– Dawny33Commented Aug 21, 2015 at 17:11
-
$\begingroup$ In the regression output under the column Coef, it shows -58.17107 for x1. When I ran correlate command to get the correlation matrix, I get 0.8592 correlation value between x1 and y. This difference in signs is confusing. $\endgroup$– AsaadCommented Aug 21, 2015 at 17:49
-
2$\begingroup$ Do a search on Simpson's Paradox. This is a fairly famous statistical concern. $\endgroup$– DWinCommented Aug 21, 2015 at 19:02
-
$\begingroup$ This page is a good introduction to these issues in the context of logistic regression; the essential issues in terms of independent variables are the same in linear regression. $\endgroup$– EdMCommented Aug 21, 2015 at 20:14
-
1$\begingroup$ I believe you will get the understanding you need in the linked thread. Please read it. If you still have a question afterwards, come back here & edit your Q to state what you've learned & what you still don't understand. Then we can provide the information you need without simply duplicating material elsewhere that already didn't help you. $\endgroup$– gung - Reinstate MonicaCommented Aug 21, 2015 at 20:40
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
4
The correlation coefficient is the unconditional simple relationship between Y and x1 . The regression coefficient is the conditional impact on Y GIVEN x2,x3,x4,and x5. They would have the same sign if x1,x2,x3,x4,x5 were orthogonal (uncorrelated) with each other. As @Whuber correctly reflected, if they are not orthogonal they may or may not have signs that are the same.
EXPANDED ANSWER TO MAKE IT MORE CONSTRUCTIVE:
Consider the following data set where the simple correlation between Y and X1 is positive 
Let is analyze Y as a function of X1 and X2
. First separately then together. The regression coefficient of the unconditional relationship between Y and M_X1 is .463 ( same sign as the simple correlation ) while the conditional impact of M_X1 given M_X2 is -1.222 (different sign ). As others have pointed out this is sometimes referred to as "
the expected sign fallacy ', "Simpson's paradox" , "Ecological fallacy " etc. In summary conditional analysis (multiple regression) is often different from unconditional analysis (simple regression) because the predictor variables are not independent of each other.
-
$\begingroup$ This is very helpful. However, the correlation values are different we I run it for all or individually. I mean correlation values between command: correlate y x1 x2 x3 x4 x5 and command: correlate y with each x individually. $\endgroup$– AsaadCommented Aug 22, 2015 at 10:03
-
$\begingroup$ Multivariate regression coefficient: indicates the change in the dependent variable associated with a one-unit increase in the independent variable in question holding constant the other independent variables in the equation.If the independent variable affects one of the other independent variables the resultant value of the dependent variable Y might be different. $\endgroup$ Commented Aug 22, 2015 at 11:55
-
$\begingroup$ If you like my answer then up-vote it and select it as your answer. In this way new readers will play close attention to my answer and consequently I will have helped to save the planet because your question is "everybody's question" and is not a repeat AFAI am concerned.. $\endgroup$ Commented Aug 22, 2015 at 13:53
-
$\begingroup$ Great explanation! An eye-opener! Thank you! $\endgroup$ Commented Aug 24, 2015 at 17:31