6
$\begingroup$

I have read quite a bit of bootstrapping, but the issue I want to address seem not to appear.

Consider a simple regression model:

$$ y_{i} = \beta_{0} + \beta_{1}x_{i} + e_{i}$$

I am aware that bootstrapping is quite useful for obtaining standard error of estimated coefficients $\hat{\beta_{0}}$ and $\hat{\beta_{1}}$, and for other statistics of the regression. But my interests lays in the predicted errors $\hat{e_{i}}$. Every bootstrap iteration generates a set of $\hat{e_{i}}$. Thus, for each unit of observation $i$, I have $M$ values of $\hat{e_{i}}$.

Can I use these $M$ values to obtain a standard error for $\hat{e_{i}}$ for each $i$? This would be very useful, for example, to identify units with high standard errors, which might be an indication of measurement errors or coding errors, etc. But there are other uses too.

Intuitively, this seems possible to me, but I want to confirm this makes sense theoretically.

$\endgroup$
  • 1
    $\begingroup$ I do this all the time by training on bootstrap samples and scoring on a hold out set. It's useful to make a plot where you sort the test set by predicted value (from a model trained on the full training set) and then display bootstrapped intervals. So yah, good idea, do it : ) $\endgroup$ – Matthew Drury Aug 21 '16 at 15:21
  • 2
    $\begingroup$ Haha. I don't want to sound rude but, besides the fact that you use it, is there a more formal justification? $\endgroup$ – luchonacho Aug 21 '16 at 15:23
  • 1
    $\begingroup$ No offense taken, fair enough. I think the justification is the same in bootstrapping any statistic. There is a thing you are trying to estimate $p (y_i)$, training a model on some set $T$ gives you a point estimate $\hat y_i$ of this value, and bootstrapping gives you an approximation of the distribution of $\hat y_i$ over the sampling distribution of various training sets. $\endgroup$ – Matthew Drury Aug 21 '16 at 15:25
  • $\begingroup$ Well, I've just done so, and it is beautiful! Any suggestion on the number of replications? $\endgroup$ – luchonacho Aug 22 '16 at 19:20
  • 1
    $\begingroup$ What conditions are known about your $e_i$? Are they, for instance, IID? Or do they follow a known covariance structure, e.g. autoregressive or compound symmetric? If we are completely agnostic about $e_i$ then we are out of options and cannot, in fact, pull ourselves up by our bootstraps. $\endgroup$ – AdamO Mar 7 '18 at 16:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.