Return to Question

I have recently been bootstrapping the confidence intervals of a neural network model estimated to data.

I execute the following psudo-code, which seems similar to previous bootstraps I have done:

1. Take all observations of data $x_i$, estimate $\hat{y} = \hat{\theta}(x_i)$
2. Resample $N$ observations from the set of $x_i$ with replacement. Estimate $\tilde{y} = \tilde{\theta}(x_i)$. Repeat until we have $M$ different versions of $\tilde{y}_m$
3. Calculate the standard deviation of everything in $\tilde{y}_M$ as $\sigma_{\tilde{y}}$ and then the confidence interval of $\hat{y}$ is $\hat{y} \pm z * \sigma_{\tilde{y}}$ where z is chosen to be an appropriate value from a standard normal distribution, usually 1.96 for 95% CI.

See here for more reading on bootstrap intervals.

At the same time, I see other approaches that use the quantiles of $\tilde{y}_M$ in order to construct intervals of some sort, such as here and here. They do this instead of the $\hat{y} \pm z * \sigma_{\tilde{y}}$ I have come to expect. The intuition for such an argument is very strong and seems valid - but is unfamiliar. What's going on with the use of quantiles? Is something else being calculated (prediction intervals, etc.) that are similar in flavor but are not the same?

I have recently been bootstrapping the confidence intervals of a neural network model estimated to data.

I execute the following psudo-code, which seems similar to previous bootstraps I have done:

1. Take all observations of data $x_i$, estimate $\hat{y} = \hat{\theta}(x_i)$
2. Resample $N$ observations from the set of $x_i$ with replacement. Estimate $\tilde{y} = \tilde{\theta}(x_i)$. Repeat until we have $M$ different versions of $\tilde{y}_m$
3. Calculate the standard deviation of everything in $\tilde{y}_M$ as $\sigma_{\tilde{y}}$ and then the confidence interval of $\hat{y}$ is $\hat{y} \pm z * \sigma_{\tilde{y}}$ where z is chosen to be an appropriate value from a standard normal distribution, usually 1.96 for 95% CI.

See here for more reading on bootstrap intervals.

At the same time, I see other approaches that use the quantiles of $\tilde{y}_M$ in order to construct intervals of some sort, such as here and here. They do this instead of the $\hat{y} \pm z * \sigma_{\tilde{y}}$ I have come to expect. The intuition for such an argument is very strong and seems valid - but the approach is unfamiliar. What's going on with the use of quantiles? Is something else being calculated (prediction intervals, etc.) that are similar in flavor but are not the same?

I have recently been bootstrapping the confidence intervals of an NN model estimated to data.

I execute the following psudo-code, which seems similar to previous bootstraps I have done:

1. Take all observations of data $x_i$, estimate $\hat{y} = \theta(x_i)$
2. Resample $N$ observations from the set of $x_i$ with replacement. Estimate $\tilde{y} = \tilde{\theta}(x_i)$. Repeat until we have $M$ different versions of $\tilde{y}_m$
3. Calculate the standard deviation of everything in $\tilde{y}_M$ as $\sigma_{\tilde{y}}$ and then the confidence interval of $\hat{y}$ is $\hat{y} \pm z * \sigma_{\tilde{y}}$ where z is chosen to be an appropriate value from a standard normal distribution, usually 1.96 for 95% CI.

See here for more reading on bootstrap intervals.

At the same time, I see other approaches that use the quantiles of $\tilde{y}_M$ in order to construct intervals of some sort, such as here and here. The intuition for such an argument is very strong and seems valid - but is unfamiliar. What's going on with the use of quantiles? Is something else being calculated (prediction intervals, etc.) that are similar in flavor but are not the same?

I have recently been bootstrapping the confidence intervals of a neural network model estimated to data.

I execute the following psudo-code, which seems similar to previous bootstraps I have done:

1. Take all observations of data $x_i$, estimate $\hat{y} = \hat{\theta}(x_i)$
2. Resample $N$ observations from the set of $x_i$ with replacement. Estimate $\tilde{y} = \tilde{\theta}(x_i)$. Repeat until we have $M$ different versions of $\tilde{y}_m$
3. Calculate the standard deviation of everything in $\tilde{y}_M$ as $\sigma_{\tilde{y}}$ and then the confidence interval of $\hat{y}$ is $\hat{y} \pm z * \sigma_{\tilde{y}}$ where z is chosen to be an appropriate value from a standard normal distribution, usually 1.96 for 95% CI.

See here for more reading on bootstrap intervals.

At the same time, I see other approaches that use the quantiles of $\tilde{y}_M$ in order to construct intervals of some sort, such as here and here. They do this instead of the $\hat{y} \pm z * \sigma_{\tilde{y}}$ I have come to expect. The intuition for such an argument is very strong and seems valid - but is unfamiliar. What's going on with the use of quantiles? Is something else being calculated (prediction intervals, etc.) that are similar in flavor but are not the same?

I have recently been bootstrapping the confidence intervals of an NN model estimated to data.

I execute the following psudo-code, which seems similar to previous bootstraps I have done:

1. Take all observations of data $x_i$, estimate $\hat{y} = \theta(x_i)$
2. Resample $N$ observations from the set of $x_i$ with replacement. Estimate $\tilde{y} = \tilde{\theta}(x_i)$. Repeat until we have $M$ different versions of $\tilde{y}_m$
3. Calculate the standard deviation of everything in $\tilde{y}_M$ as $\sigma_{\tilde{y}}$ and then the prediction interval of $\hat{y}$ is $\hat{y} \pm z * \sigma_{\tilde{y}}$ where z is chosen to be an appropriate value from a standard normal distribution, usually 1.96 for 95% CI.

See here for more reading on bootstrap intervals.

At the same time, I see other approaches that use the quantiles of $\tilde{y}_M$ in order to construct intervals of some sort, such as here and here. The intuition for such an argument is very strong and seems valid - but is unfamiliar. What's going on with the use of quantiles? Is something else being calculated (prediction intervals, etc.) that are similar in flavor but are not the same?

I have recently been bootstrapping the confidence intervals of an NN model estimated to data.

I execute the following psudo-code, which seems similar to previous bootstraps I have done:

1. Take all observations of data $x_i$, estimate $\hat{y} = \theta(x_i)$
2. Resample $N$ observations from the set of $x_i$ with replacement. Estimate $\tilde{y} = \tilde{\theta}(x_i)$. Repeat until we have $M$ different versions of $\tilde{y}_m$
3. Calculate the standard deviation of everything in $\tilde{y}_M$ as $\sigma_{\tilde{y}}$ and then the confidence interval of $\hat{y}$ is $\hat{y} \pm z * \sigma_{\tilde{y}}$ where z is chosen to be an appropriate value from a standard normal distribution, usually 1.96 for 95% CI.

See here for more reading on bootstrap intervals.

At the same time, I see other approaches that use the quantiles of $\tilde{y}_M$ in order to construct intervals of some sort, such as here and here. The intuition for such an argument is very strong and seems valid - but is unfamiliar. What's going on with the use of quantiles? Is something else being calculated (prediction intervals, etc.) that are similar in flavor but are not the same?