3
$\begingroup$

For a model, $y = \beta_0 + \beta_1 x + \epsilon$, does the variance for $\hat{\beta_0}$ and $\hat{\beta_1}$ always decrease when more datapoints are added?

I've decuded the variance expressions as: \begin{equation} Var[\hat{\beta_0}] = \sigma^2 (\frac{1}{n} + \frac{\bar{x}^2}{S_{xx}}), \end{equation} and \begin{equation} Var[\hat{\beta_1}] = \frac{\sigma^2}{S_{xx}}. \end{equation}

But I'm still yet to draw a conclusion, I can think of examples where $S_{xx}$ would remain constant if adding $x_i = 0$ to a set that already has $\bar{x} = 0$, so I don't think $Var[\hat{\beta_1}]$ has to decrease, however I'm not entirely sure. What can be said about $Var[\hat{\beta_0}]$? Obviously, it would decrease considering the same example as for $\hat{\beta_1}$ since $1/n$ naturally does, but can something be said in general?

Thank you.

$\endgroup$

1 Answer 1

0
$\begingroup$

Here is an example for when the estimated variance of the two coefficients is larger for a larger sample. (Your notation $\sigma^2$ might suggest that you are asking the question for the true error variance - here, of course, the additional data point also affects the error variance estimate.)

set.seed(9)
n <- 5
y <- rnorm(n)
y[5] <- 3
x <- rnorm(n)

plot(x[-n], y[-n], ylim=c(-1,4), pch=19, col="red")
points(x[n], y[n], col="blue", pch=19)
abline(lm(y~x), lwd=2, col="blue")
abline(lm(y[-n]~x[-n]), lwd=2, col="red")
summary(lm(y[-n]~x[-n]))
summary(lm(y~x))

Essentially, the blue additional point is an outlier in the y direction that will not affect the estimate for $\bar x$, but will have a large residual and hence affect the error variance estimate.

enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.