2
$\begingroup$

I was learning the mechanics of omitted variable bias in the context of linear regression. I built the following simple model with R:

x_female = runif(100, 1.1, 1.4)
x_male = runif(100, 2.8, 3.1)
y_female = -5 + 5 * x_female + rnorm(100, 0, 0.3)
y_male = -10 + 5 * x_male + rnorm(100, 0, 0.3)
data = data.frame(edu = c(x_female, x_male),
                  income = c(y_female, y_male), 
                  male = c(rep(0, 100), rep(1, 100)))

The corresponding scatterplot looks like this:enter image description here

My question is: since gender has a positive correlation with education (male population are better educated), and since gender has a positive effect on income, according to what I read, omitting gender should introduce a positive bias on the estimate of the coefficient of income. However, this isn't the case here - apparently the red/blue regression line (which controls for gender) is steeper than the green regression line (which doesn't). Is there something wrong with my understanding of omitted variable bias?

$\endgroup$

1 Answer 1

1
$\begingroup$

The observed difference after grouping the regressions can be either positive or negative. If you think of it as two groups of values binned to only have one filtered number of responses, almost necessarily that implies that the relationship within groups will either be more positive or more negative and will vary based off by-group differences unless the groups are almost exactly alike.

Remember that the intercept is a conditional mean and that a categorical regression with no continuous variables simply adjusts this mean to represent the mean DV of each group. With continuous variables added, these are the same thing but adjustments based on continuous variables. In your case its very clear. Your blue group has a mean that is much higher than the pink group, and so the distance between group regression lines is quite extreme. But the relationship within those groups is fairly similar. If we try to "smooth" over those differences by just including the continuous predictor, we get a line that adjusts these differences as best as possible to get the best fitting line, which is somewhere in between and thus less "strong" or "positive." Another way to think of it is that the regression line draws a line between the means (the centers) of these two centroids, hence the line will adjust when possible to make the line more/less magnified. The opposite can be true when your blue group has a much lower mean than the pink group, which will make a less extreme overall relationship if the groupings are not made.

All in all, this is not surprising. Grouping variables has a number of influences on the regression fits, and your plot shows a clear example of where the relationship is downward biased when one doesn't consider important by-group differences in the mean.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.