4
$\begingroup$

Assume I am estimating a simple cross-sectional regression model. What happens if a large portion of the cross sections consists of zeros only? That is, both the dependent and the independent variables are zero.

Do these cross-sections even have an impact on the estimation at all? Or are they excluded from the estimation.

Improve this question
$\endgroup$
6
  • $\begingroup$ Thinking about it myself I guess it depends wether the model includes an intercept or not. If it does, I would assume that the intercept estimate is dragged down to zero. If there is no intercept, zero values would not have an impact at all. But this is just my thought on it... $\endgroup$ – shenflow Jul 23 '20 at 10:27
  • $\begingroup$ If both the dependent and independent variables have a lot of zeros and these are valid data, then a zero value of the independent variable is a pretty good predictor of the dependent variable for those cases. However we would need to know more about the rest of your data to say much more. $\endgroup$ – Robert Long Jul 23 '20 at 11:59
  • $\begingroup$ Yes I understand that. But in the case of zeroes only, how does this affect the estimated coefficient? Because 0*1 is the same as 0*1000000. In other words, if the dependent variable aswell as the independent variables are zeroes only, the coefficients could take on any value as far as I understand basic algebra. There is no "optimal" value looking at the zeroes solely. Thus, I am left wondering how they effect the global estimate of the regression coefficients. $\endgroup$ – shenflow Jul 23 '20 at 12:28
  • $\begingroup$ We need more info about the data. $\endgroup$ – Robert Long Jul 23 '20 at 13:01
  • 1
    $\begingroup$ Consider a two-dimensional plane. In $y = \beta_0+\beta_1x$ setting, it is clear that adding more and more points $(0,0)$ will drag the line to pass through origin, making $\beta_0$ closer to zero. Where the line goes (slope, $\beta_1$) is determined by other non- $(0,0)$ points. I don't know if this is what you're looking for, though. $\endgroup$ – Nuclear03020704 Jul 28 '20 at 4:03
1
+50
$\begingroup$

Yes, it will definitely have an impact on your model. Here's a simple simulation to demonstrate. Feel free to fiddle with this code on your own.

It will change slope, intercept, and MSE. It could give you the illusion of a better fit because those (0,0) points will have very low residuals. If the center of your data is not (0, 0), then they'll be high-leverage too.

You need to consider the origin of your zeros. Are they true zeros? Or are they censored values? What to do depends on your question of interest.

library(ggplot2)
library(gridExtra)

# Specify model parameters
n_nonzero = 20
n_zero = 20

beta0 = 5
beta1 = 0.2

mean_x = 5


# Generate data without zeros
x = rnorm(n_nonzero, mean_x, 1)
y = beta0 + beta1 * x + rnorm(n_nonzero, 0, 1)

dat_no_zeros = data.frame(cbind(y, x))
dat_no_zeros = dat_no_zeros[sample(nrow(dat_no_zeros)),]
# Plot
p_no_zeros = ggplot(aes(x=x, y=y), data=dat_no_zeros) + 
  geom_point() + 
  geom_smooth(method='lm') +
  ggtitle('Without zeros')

# Add zeros to above data
x2 = c(x, rep(0, n_zero))
y2 = c(y, rep(0, n_zero))

dat_with_zeros = data.frame(cbind(y2, x2))
dat_with_zeros = dat_with_zeros[sample(nrow(dat_with_zeros)),]
# Plot
p_with_zeros = ggplot(aes(x=x2, y=y2), data=dat_with_zeros) + 
  geom_point() + 
  geom_smooth(method='lm') +
  ggtitle('With zeros')

grid.arrange(p_no_zeros, p_with_zeros, ncol=2)

enter image description here

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.