Skip to main content
deleted 1 character in body
Source Link

I disagree with the initial answer with respect to the claim that the mean of the bootstraps is in general a redundant quantity because it only is under certain conditions and these depend strongly on the model and parameters you are considering.

To give my answer some context, let's set up an example: If we estimate a linear model via ML over a multivariate normal distribution, the MLEs have certain favorable estimatesfeatures: They are asymptotically unbiased, consistent, efficient and - relevant for CI computation - normally distributed – if your data are really drawn from a multivariate normal distribution.

If multivariate normal distribution holds and your sample is large, MLEs and ML-CIs are the best thing you could ever report (with respect to the listed properties). Reporting bootstrap CIs can have mainly two reasons, either the distributional assumption is violated or it is just too complicated to derive the second derivative of the likelihood to get analytical standard errors.

Assuming that the distributional assumption is violated, the properties of the MLE are not guaranteed any more. Sometimes it is possible to derive the MLEs without the distributional assumption (for instance, the MLEs happen to be equivalent to the least square estimates in linear regression). Thus, you might know your MLEs are unbiased and consistent irrespective of the distribution. However, the distribution of MLEs across samples may then be unknown. Bootstrapping is an easy way out of this problem.

There are many approaches to get CIs from a bootstrap and some of them perform poorly if the distribution across bootstrap samples is not nearly symmetric or normal. The mean of the bootstrap samples is a straightforward indicator of not so well-behaved bootstrap distributions because, e.g., skewed bootstrap distributions generally result in a difference between the original estimate and the mean across bootstraps. You could also turn the argument around: If the two are close, it is likely that the simplest ways of computing bootstrap CIs suffice (and that bootstrapping is not generally inappropriate). I have once had a reviewer who wanted me to report the means across bootstraps for this very reason.

To be clear, I am not proposing that the mean of the bootstrap samples is the ideal indicator of potential problems or that it must always be reported. All I am trying to point out is that it is not - in general - a redundant quantity.

I attach some further readings: Bootstrap based bias correction

https://garstats.wordpress.com/2018/01/23/bias-correction/

When is the bootstrap estimate of bias valid?

I disagree with the initial answer with respect to the claim that the mean of the bootstraps is in general a redundant quantity because it only is under certain conditions and these depend strongly on the model and parameters you are considering.

To give my answer some context, let's set up an example: If we estimate a linear model via ML over a multivariate normal distribution, the MLEs have certain favorable estimates: They are asymptotically unbiased, consistent, efficient and - relevant for CI computation - normally distributed – if your data are really drawn from a multivariate normal distribution.

If multivariate normal distribution holds and your sample is large, MLEs and ML-CIs are the best thing you could ever report (with respect to the listed properties). Reporting bootstrap CIs can have mainly two reasons, either the distributional assumption is violated or it is just too complicated to derive the second derivative to get analytical standard errors.

Assuming that the distributional assumption is violated, the properties of the MLE are not guaranteed any more. Sometimes it is possible to derive the MLEs without the distributional assumption (for instance, the MLEs happen to be equivalent to the least square estimates in linear regression). Thus, you might know your MLEs are unbiased and consistent irrespective of the distribution. However, the distribution of MLEs across samples may then be unknown. Bootstrapping is an easy way out of this problem.

There are many approaches to get CIs from a bootstrap and some of them perform poorly if the distribution across bootstrap samples is not nearly symmetric or normal. The mean of the bootstrap samples is a straightforward indicator of not so well-behaved bootstrap distributions because, e.g., skewed bootstrap distributions generally result in a difference between the original estimate and the mean across bootstraps. You could also turn the argument around: If the two are close, it is likely that the simplest ways of computing bootstrap CIs suffice (and that bootstrapping is not generally inappropriate). I have once had a reviewer who wanted me to report the means across bootstraps for this very reason.

To be clear, I am not proposing that the mean of the bootstrap samples is the ideal indicator of potential problems or that it must always be reported. All I am trying to point out is that it is not - in general - a redundant quantity.

I attach some further readings: Bootstrap based bias correction

https://garstats.wordpress.com/2018/01/23/bias-correction/

When is the bootstrap estimate of bias valid?

I disagree with the initial answer with respect to the claim that the mean of the bootstraps is in general a redundant quantity because it only is under certain conditions and these depend strongly on the model and parameters you are considering.

To give my answer some context, let's set up an example: If we estimate a linear model via ML over a multivariate normal distribution, the MLEs have certain favorable features: They are asymptotically unbiased, consistent, efficient and - relevant for CI computation - normally distributed – if your data are really drawn from a multivariate normal distribution.

If multivariate normal distribution holds and your sample is large, MLEs and ML-CIs are the best thing you could ever report (with respect to the listed properties). Reporting bootstrap CIs can have mainly two reasons, either the distributional assumption is violated or it is just too complicated to derive the second derivative of the likelihood to get analytical standard errors.

Assuming that the distributional assumption is violated, the properties of the MLE are not guaranteed any more. Sometimes it is possible to derive the MLEs without the distributional assumption (for instance, the MLEs happen to be equivalent to the least square estimates in linear regression). Thus, you might know your MLEs are unbiased and consistent irrespective of the distribution. However, the distribution of MLEs across samples may then be unknown. Bootstrapping is an easy way out of this problem.

There are many approaches to get CIs from a bootstrap and some of them perform poorly if the distribution across bootstrap samples is not nearly symmetric or normal. The mean of the bootstrap samples is a straightforward indicator of not so well-behaved bootstrap distributions because, e.g., skewed bootstrap distributions generally result in a difference between the original estimate and the mean across bootstraps. You could also turn the argument around: If the two are close, it is likely that the simplest ways of computing bootstrap CIs suffice (and that bootstrapping is not generally inappropriate). I have once had a reviewer who wanted me to report the means across bootstraps for this very reason.

To be clear, I am not proposing that the mean of the bootstrap samples is the ideal indicator of potential problems or that it must always be reported. All I am trying to point out is that it is not - in general - a redundant quantity.

I attach some further readings: Bootstrap based bias correction

https://garstats.wordpress.com/2018/01/23/bias-correction/

When is the bootstrap estimate of bias valid?

added 23 characters in body
Source Link

I disagree with the initial answer inwith respect to the claim that the mean of the bootstraps is in general a redundant quantity because it only is under certain conditions and these depend strongly on the model and parameters you are considering.

To give my answer some context, let's set up an example: If we estimate a linear model via ML over a multivariate normal distribution, the MLEs have certain favorable estimates: They are asymptotically unbiased, consistent, efficient and - relevant for CI computation - normally distributed – if your data are really drawn from a multivariate normal distribution.

If multivariate normal distribution holds and your sample is large, MLEs and ML-CIs are the best thing you could ever report (with respect to the listed properties). Reporting bootstrap CIs can have mainly two reasons, either the distributional assumption is violated or it is just too complicated to derive the second derivative to get analytical standard errors.

Assuming that the distributional assumption is violated, the properties of the MLE are not guaranteed any more. Sometimes it is possible to derive the MLEs without the distributional assumption (for instance, the MLEs happen to be equivalent to the least square estimates in linear regression). Thus, you might know your MLEs are unbiased and consistent irrespective of the distribution. However, the distribution of MLEs across samples may then be unknown. Bootstrapping is an easy way out of this problem.

There are many approaches to get CIs from a bootstrap and some of them perform poorly if the distribution across bootstrap samples is not nearly symmetric or normal. The mean of the bootstrap samples is a straightforward indicator of not so well-behaved bootstrap distributions because, e.g., skewed bootstrap distributions generally result in a difference between the original estimate and the mean across bootstraps. You could also turn the argument around: If the two are close, it is likely that the simplest ways of computing bootstrap CIs suffice (and that bootstrapping is not generally inappropriate). I have once had a reviewer who wanted me to report the means across bootstraps for this very reason.

To be clear, I am not proposing that the mean of the bootstrap samples is the ideal indicator of potential problems or that it must always be reported. All I am trying to point out is that it is not - in general - a redundant quantity.

I attach some further readings: Bootstrap based bias correction

https://garstats.wordpress.com/2018/01/23/bias-correction/

When is the bootstrap estimate of bias valid?

I disagree with the initial answer in that the mean of the bootstraps is in general a redundant quantity because it only is under certain conditions and these depend strongly on the model and parameters you are considering.

To give my answer some context, let's set up an example: If we estimate a linear model via ML over a multivariate normal distribution, the MLEs have certain favorable estimates: They are asymptotically unbiased, consistent, efficient and - relevant for CI computation - normally distributed – if your data are really drawn from a multivariate normal distribution.

If multivariate normal distribution holds and your sample is large, MLEs and ML-CIs are the best thing you could ever report (with respect to the listed properties). Reporting bootstrap CIs can have mainly two reasons, either the distributional assumption is violated or it is just too complicated to derive the second derivative to get analytical standard errors.

Assuming that the distributional assumption is violated, the properties of the MLE are not guaranteed any more. Sometimes it is possible to derive the MLEs without the distributional assumption (for instance, the MLEs happen to be equivalent to the least square estimates in linear regression). Thus, you might know your MLEs are unbiased and consistent irrespective of the distribution. However, the distribution of MLEs across samples may then be unknown. Bootstrapping is an easy way out of this problem.

There are many approaches to get CIs from a bootstrap and some of them perform poorly if the distribution across bootstrap samples is not nearly symmetric or normal. The mean of the bootstrap samples is a straightforward indicator of not so well-behaved bootstrap distributions because, e.g., skewed bootstrap distributions generally result in a difference between the original estimate and the mean across bootstraps. You could also turn the argument around: If the two are close, it is likely that the simplest ways of computing bootstrap CIs suffice (and that bootstrapping is not generally inappropriate). I have once had a reviewer who wanted me to report the means across bootstraps for this very reason.

To be clear, I am not proposing that the mean of the bootstrap samples is the ideal indicator of potential problems or that it must always be reported. All I am trying to point out is that it is not - in general - a redundant quantity.

I attach some further readings: Bootstrap based bias correction

https://garstats.wordpress.com/2018/01/23/bias-correction/

When is the bootstrap estimate of bias valid?

I disagree with the initial answer with respect to the claim that the mean of the bootstraps is in general a redundant quantity because it only is under certain conditions and these depend strongly on the model and parameters you are considering.

To give my answer some context, let's set up an example: If we estimate a linear model via ML over a multivariate normal distribution, the MLEs have certain favorable estimates: They are asymptotically unbiased, consistent, efficient and - relevant for CI computation - normally distributed – if your data are really drawn from a multivariate normal distribution.

If multivariate normal distribution holds and your sample is large, MLEs and ML-CIs are the best thing you could ever report (with respect to the listed properties). Reporting bootstrap CIs can have mainly two reasons, either the distributional assumption is violated or it is just too complicated to derive the second derivative to get analytical standard errors.

Assuming that the distributional assumption is violated, the properties of the MLE are not guaranteed any more. Sometimes it is possible to derive the MLEs without the distributional assumption (for instance, the MLEs happen to be equivalent to the least square estimates in linear regression). Thus, you might know your MLEs are unbiased and consistent irrespective of the distribution. However, the distribution of MLEs across samples may then be unknown. Bootstrapping is an easy way out of this problem.

There are many approaches to get CIs from a bootstrap and some of them perform poorly if the distribution across bootstrap samples is not nearly symmetric or normal. The mean of the bootstrap samples is a straightforward indicator of not so well-behaved bootstrap distributions because, e.g., skewed bootstrap distributions generally result in a difference between the original estimate and the mean across bootstraps. You could also turn the argument around: If the two are close, it is likely that the simplest ways of computing bootstrap CIs suffice (and that bootstrapping is not generally inappropriate). I have once had a reviewer who wanted me to report the means across bootstraps for this very reason.

To be clear, I am not proposing that the mean of the bootstrap samples is the ideal indicator of potential problems or that it must always be reported. All I am trying to point out is that it is not - in general - a redundant quantity.

I attach some further readings: Bootstrap based bias correction

https://garstats.wordpress.com/2018/01/23/bias-correction/

When is the bootstrap estimate of bias valid?

Source Link

I disagree with the initial answer in that the mean of the bootstraps is in general a redundant quantity because it only is under certain conditions and these depend strongly on the model and parameters you are considering.

To give my answer some context, let's set up an example: If we estimate a linear model via ML over a multivariate normal distribution, the MLEs have certain favorable estimates: They are asymptotically unbiased, consistent, efficient and - relevant for CI computation - normally distributed – if your data are really drawn from a multivariate normal distribution.

If multivariate normal distribution holds and your sample is large, MLEs and ML-CIs are the best thing you could ever report (with respect to the listed properties). Reporting bootstrap CIs can have mainly two reasons, either the distributional assumption is violated or it is just too complicated to derive the second derivative to get analytical standard errors.

Assuming that the distributional assumption is violated, the properties of the MLE are not guaranteed any more. Sometimes it is possible to derive the MLEs without the distributional assumption (for instance, the MLEs happen to be equivalent to the least square estimates in linear regression). Thus, you might know your MLEs are unbiased and consistent irrespective of the distribution. However, the distribution of MLEs across samples may then be unknown. Bootstrapping is an easy way out of this problem.

There are many approaches to get CIs from a bootstrap and some of them perform poorly if the distribution across bootstrap samples is not nearly symmetric or normal. The mean of the bootstrap samples is a straightforward indicator of not so well-behaved bootstrap distributions because, e.g., skewed bootstrap distributions generally result in a difference between the original estimate and the mean across bootstraps. You could also turn the argument around: If the two are close, it is likely that the simplest ways of computing bootstrap CIs suffice (and that bootstrapping is not generally inappropriate). I have once had a reviewer who wanted me to report the means across bootstraps for this very reason.

To be clear, I am not proposing that the mean of the bootstrap samples is the ideal indicator of potential problems or that it must always be reported. All I am trying to point out is that it is not - in general - a redundant quantity.

I attach some further readings: Bootstrap based bias correction

https://garstats.wordpress.com/2018/01/23/bias-correction/

When is the bootstrap estimate of bias valid?