1
$\begingroup$

I have a question about how to use bootstrap to generate the confidence intervals for my model predictions. I do not care about the confidence intervals of the single parameters of the model (indeed, I would like something I can apply even to a non-parametric model). Have a look at the post

Two ways of using bootstrap to estimate the confidence interval of coefficients in regression

which generated a some really good replies. Consider the methodology described at point 2 by the original poster. Suppose I have a more complex linear model than an OLS (lasso, elastic net etc...). Somehow, I have trained my model and I got

\begin{equation} \mathbf{y}=\cal{M}(\mathbf{X})+\mathbf{\epsilon} \end{equation}

where $\mathbf{y}=y_1, y_2, ...,y_N$ is the vector of the dependent variable and $\mathbf{X}$ and $\mathbf{\epsilon}$ are a matrix of independent variables and the residuals of my model $\cal{M}(\mathbf{X})$. As in that post, I can define $\mathbf{y^*}=\mathbf{\hat y}+\mathbf{\epsilon^*}$, where $\mathbf{\hat y}=\cal{M}(\mathbf{X})$ is the expectation value of $\mathbf{y}$ obtained from the model (which I fit only once) and $\mathbf{\epsilon^*}$ are the values of the residuals resampled with replacement.

If I am interested only in estimated the uncertainties in the model prediction prediction, is it OK to simply collect the values of $\mathbf{y^*}$ and for instance evaluate the quantiles of the distribution? This way I need to fit the model only once. Or should I rather fit the model from scratch on any new set $\{\mathbf{y^*}, \mathbf{X}\}$ and then calculate every time a new expectation value of the model, let us call it $\mathbf{\hat y^*}$?

Many thanks for any clarification.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.