How do zeroes impact regression estimates?

Assume I am estimating a simple cross-sectional regression model. What happens if a large portion of the cross sections consists of zeros only? That is, both the dependent and the independent variables are zero.

Do these cross-sections even have an impact on the estimation at all? Or are they excluded from the estimation.

• Thinking about it myself I guess it depends wether the model includes an intercept or not. If it does, I would assume that the intercept estimate is dragged down to zero. If there is no intercept, zero values would not have an impact at all. But this is just my thought on it... Jul 23, 2020 at 10:27
• If both the dependent and independent variables have a lot of zeros and these are valid data, then a zero value of the independent variable is a pretty good predictor of the dependent variable for those cases. However we would need to know more about the rest of your data to say much more. Jul 23, 2020 at 11:59
• Yes I understand that. But in the case of zeroes only, how does this affect the estimated coefficient? Because 0*1 is the same as 0*1000000. In other words, if the dependent variable aswell as the independent variables are zeroes only, the coefficients could take on any value as far as I understand basic algebra. There is no "optimal" value looking at the zeroes solely. Thus, I am left wondering how they effect the global estimate of the regression coefficients. Jul 23, 2020 at 12:28
• Consider a two-dimensional plane. In $y = \beta_0+\beta_1x$ setting, it is clear that adding more and more points $(0,0)$ will drag the line to pass through origin, making $\beta_0$ closer to zero. Where the line goes (slope, $\beta_1$) is determined by other non- $(0,0)$ points. I don't know if this is what you're looking for, though. Jul 28, 2020 at 4:03

Yes, it will definitely have an impact on your model. Here's a simple simulation to demonstrate. Feel free to fiddle with this code on your own.

It will change slope, intercept, and MSE. It could give you the illusion of a better fit because those (0,0) points will have very low residuals. If the center of your data is not (0, 0), then they'll be high-leverage too.

You need to consider the origin of your zeros. Are they true zeros? Or are they censored values? What to do depends on your question of interest.

library(ggplot2)
library(gridExtra)

# Specify model parameters
n_nonzero = 20
n_zero = 20

beta0 = 5
beta1 = 0.2

mean_x = 5

# Generate data without zeros
x = rnorm(n_nonzero, mean_x, 1)
y = beta0 + beta1 * x + rnorm(n_nonzero, 0, 1)

dat_no_zeros = data.frame(cbind(y, x))
dat_no_zeros = dat_no_zeros[sample(nrow(dat_no_zeros)),]
# Plot
p_no_zeros = ggplot(aes(x=x, y=y), data=dat_no_zeros) +
geom_point() +
geom_smooth(method='lm') +
ggtitle('Without zeros')

# Add zeros to above data
x2 = c(x, rep(0, n_zero))
y2 = c(y, rep(0, n_zero))

dat_with_zeros = data.frame(cbind(y2, x2))
dat_with_zeros = dat_with_zeros[sample(nrow(dat_with_zeros)),]
# Plot
p_with_zeros = ggplot(aes(x=x2, y=y2), data=dat_with_zeros) +
geom_point() +
geom_smooth(method='lm') +
ggtitle('With zeros')

grid.arrange(p_no_zeros, p_with_zeros, ncol=2)