If we apply linear regression on a data which has a BINARY(0,1) dependant variable, the very important assumption of "constant variance" of the dependant variable across independant variables is violated. Can anyone explain how ?
$\begingroup$
$\endgroup$
8
-
3$\begingroup$ Linear regression with a binary dependent variable? That is asking for trouble in so many ways beyond the constancy of variance. Can you elaborate why you would want to do this instead of (e.g.) logistic regression? $\endgroup$– Nick SabbeCommented Mar 14, 2013 at 13:22
-
$\begingroup$ What's the variance of a Bernoulli($\pi$) random variable? $\endgroup$– Glen_bCommented Mar 14, 2013 at 22:59
-
$\begingroup$ @glen_b : its p(1-p) where p is the probability that your event is 1 $\endgroup$– MukulCommented Mar 15, 2013 at 4:31
-
$\begingroup$ @NickSabbe : I am trying to understand why do we NOT use linear for binary dependant. As per text books its not wise to use linear for some violations. So I was trying to understand individually what all assumptions are violated $\endgroup$– MukulCommented Mar 15, 2013 at 4:34
-
$\begingroup$ So what happens to $p(1-p)$ when $p$ is near 0 or 1 as compared to when it's near the middle? It changes. The variance is not constant. Since the $p$ changes with the IVs, so does the variance. $\endgroup$– Glen_bCommented Mar 15, 2013 at 4:42
|
Show 3 more comments
1 Answer
$\begingroup$
$\endgroup$
4
You may try to visualize the idea by making a residual plot of the said regression:
The constant variance assumption is that the expected dependent variable conditioned on all independent variables is constant. If we, very roughly, slice up the fitted values into chunks, and calculate the variance of the residual within each, they should be roughly similar. From the above plot you can see that it's not always true for binary outcome (red fonts indicate the variance within each segment.)
Not always because sometimes the situation is not that bad, especially if the independent variable is not very predictive (aka, there is a good deal of overlapping in the groups outcome = 1 and outcome = 0.) However, the normality assumption will still be violated.
-
$\begingroup$ you just +1 your count of number of your fans. This is the precise answer I was looking for. So this exactly in line with your previous answer to my old question. So we check for constant variance within subsamples of the data by considering the segments(roughly) as mentioned above in your red fonts. That is how we look for constant variance. Hope my understanding is correct $\endgroup$– MukulCommented Mar 14, 2013 at 16:01
-
$\begingroup$ can you explain why do we have two distinct lines above. Is it because one line corresponds to the residuals calculated for 1's and the other is for 0's. And also as per your last comment. The height of the Red arrows (previous post) is a proxy for varaince. Right ? $\endgroup$– MukulCommented Mar 14, 2013 at 16:11
-
$\begingroup$ @Mukul For me, this would serve just to understand the concept. In real life, I have never segregated the data and check the variance chunk by chunk (too many tests, and the cutting of the sample can also be controversial.) You can simply plot residual$^2$ against the predicted value to observe if there are any violation of constant variance. $\endgroup$ Commented Mar 14, 2013 at 17:37
-
$\begingroup$ @Mukul Yes, correct. One line is for the outcome = 0 and the other one is outcome = 1. This is, however, only true if you have one predictor. As you add more predictors, the picture can change to either smearing out (with continuous predictor) or having many straight lines (with categorical predictor.) As for the red arrows, yes you're correct. They show the range of the error, which is not variance. Please consider that as a simplified representation. $\endgroup$ Commented Mar 14, 2013 at 17:37