Convergence to the mean estimate with bootstrap

Question

Let's say I am running a regression over the full sample and I get my coefficient values for each variable.

y = intercept + alpha X1 + beta X2

coef:

y = 1.8354 + 0.23234 X1 + 0.4564 X2

I woudl like to the check the values I obtained using bootstraping (because I don't have any hold out sample ).

Running a bootstrap on the regression I was wondering if the mean values for the coefficients obtained (alpha and beta) would always converge to the value from the full sample regression. And thus the only "meaningful" insight would be the confidence intervals ?

Answer 1

Yes they do converge. When bootstrapping you draw m samples $B_i$ of size n with replacement (means that some observations can get into the sample more than one time). Thus when you draw enough samples the coefficients, i. e. the mean of the samples, will converge to that of the full sample. The standard errors are robust against heteroscedasticity and thus compared to standard OLS they could be greater (approximately like robust standard errors).

$\hat\theta =\bar B = \frac1m\sum_\limits{i=1}^m\hat B_{i}$

Hence the estimated standard error is obtained by

$\hat s_\hat\theta=\sqrt{\frac1{m-1}\sum\limits_{i=1}^m(\hat B_i-\bar B)^2}$.

Assuming a normal distribution the confidence intervals are obtained accordingly:

$[\hat\theta-\hat s_\hat\theta z_{1-\alpha/2}, \hat\theta+\hat s_\hat\theta z_{1-\alpha/2}]$.

For comparison in OLS the standard error is estimated by

$\hat s_{\hat\beta_j} =\sqrt{\sigma^2\frac{1}{\sum\nolimits_{i=1}^n(X_i-\bar X)^2}}$

where $\sigma^2$ is the variance of the measurement errors.