7
$\begingroup$

I have read quite a bit of bootstrapping, but the issue I want to address seem not to appear.

Consider a simple regression model:

$$ y_{i} = \beta_{0} + \beta_{1}x_{i} + e_{i}$$

I am aware that bootstrapping is quite useful for obtaining standard error of estimated coefficients $\hat{\beta_{0}}$ and $\hat{\beta_{1}}$, and for other statistics of the regression. But my interests lays in the predicted errors $\hat{e_{i}}$. Every bootstrap iteration generates a set of $\hat{e_{i}}$. Thus, for each unit of observation $i$, I have $M$ values of $\hat{e_{i}}$.

Can I use these $M$ values to obtain a standard error for $\hat{e_{i}}$ for each $i$? This would be very useful, for example, to identify units with high standard errors, which might be an indication of measurement errors or coding errors, etc. But there are other uses too.

Intuitively, this seems possible to me, but I want to confirm this makes sense theoretically.

Improve this question
$\endgroup$
8
  • 1
    $\begingroup$ I do this all the time by training on bootstrap samples and scoring on a hold out set. It's useful to make a plot where you sort the test set by predicted value (from a model trained on the full training set) and then display bootstrapped intervals. So yah, good idea, do it : ) $\endgroup$
    – Matthew Drury
    Aug 21, 2016 at 15:21
  • 2
    $\begingroup$ Haha. I don't want to sound rude but, besides the fact that you use it, is there a more formal justification? $\endgroup$
    – luchonacho
    Aug 21, 2016 at 15:23
  • 1
    $\begingroup$ No offense taken, fair enough. I think the justification is the same in bootstrapping any statistic. There is a thing you are trying to estimate $p (y_i)$, training a model on some set $T$ gives you a point estimate $\hat y_i$ of this value, and bootstrapping gives you an approximation of the distribution of $\hat y_i$ over the sampling distribution of various training sets. $\endgroup$
    – Matthew Drury
    Aug 21, 2016 at 15:25
  • $\begingroup$ Well, I've just done so, and it is beautiful! Any suggestion on the number of replications? $\endgroup$
    – luchonacho
    Aug 22, 2016 at 19:20
  • 1
    $\begingroup$ What conditions are known about your $e_i$? Are they, for instance, IID? Or do they follow a known covariance structure, e.g. autoregressive or compound symmetric? If we are completely agnostic about $e_i$ then we are out of options and cannot, in fact, pull ourselves up by our bootstraps. $\endgroup$
    – AdamO
    Mar 7, 2018 at 16:36

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Browse other questions tagged or ask your own question.