3
$\begingroup$

Suppose we obtain a point estimate using a maximum a posteriori estimator $\hat{\theta}_{MAP}$. Note that I'm aware Bayesian approaches generally do seek point estimates, but suppose this is an example where a point estimator is specifically needed, and we want to incorporate some prior information using a prior distribution. Then we need some way to quantify the accuracy of our estimator $\hat{\theta}_{MAP}$ - and suppose that an analytical approach to deriving it is not available.

My question is whether it is acceptable to use standard bootstrap or jackknife methods to estimate the variance of $\hat{\theta}_{MAP}$? For example, suppose we obtain thousands of bootstrap resamples, each with a boostrap estimate $\hat{\theta}_{MAP}^b$, and estimate the standard error of our estimator based off of these boostrap sample estimates.

The reason I am confused is that all mentions of stanndard bootstrap/jackknife that I have read consider it solely in the frequentist domain, and I can't really find any reference to normal bootstrap/jackknife when it comes to Bayesian point estimates. Again, I guess this is because Bayesian approaches generally do not seek point estimates over descriptive statistics of the posterior.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
3
+50
$\begingroup$

Yes, you can absolutely do this. One of the great things about resampling methods (including the bootstrap and jackknife) is that they really don't care how you came up with your point estimator. One thing to realize, though, is that by using these tools you're estimating the sampling variance of your estimator, which is not generally the same thing as the variance of the bayesian posterior distribution.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Does the sampling variance of a MAP estimator have any uses though? It can't be use to calculate a p-value, I would imagine. And as you say, it doesn't necessarily correspond to the posterior variance. $\endgroup$
    – Eoin
    Jan 28, 2021 at 19:17
  • $\begingroup$ Out of interest, I wrote some simulation code to see whether the Bootstrap corresponded to the posterior variance for a simple example. Not finished, but leaving it here for safekeeping: gist.github.com/EoinTravers/603f705a7acdc286acddce3f18b8deda $\endgroup$
    – Eoin
    Jan 28, 2021 at 19:40
  • $\begingroup$ "Does the sampling variance of a MAP estimator have any uses though?" Well, that's a different question... ;) Under certain conditions (approximately normal sampling distribution, sampling variance not changing too quickly in the neighborhood of the estimated parameters) I could imagine the sampling variance of the MAP estimate being used to construct an approximate (frequentist) CI for the MAP estimate, or an approximate p-value... But maybe it would be wise to ask why @adriankahk wants to estimate the sampling variance in the first place... $\endgroup$
    – Adam L. Taylor
    Jan 28, 2021 at 20:59
  • $\begingroup$ My desire is to assess how significant the estimation is from a specific value. i.e., I want to see if the deviation of $\hat{\theta}_{MAP}$ is statistically significant from some $\theta_0 \in \mathbb{R}$. I figure to do this I need some sort of standard error estimate. So while I am not interested in p-values, I am interested in making some preliminary inferences. $\endgroup$
    – user281754
    Jan 29, 2021 at 6:13
  • 1
    $\begingroup$ OK, so it sounds like for now you just want a quick-and-dirty quantification of how accurate your estimator is, and the bootstrap variance will tell you that. But I guess if it were me I would consider actually computing a proper bootstrap confidence interval (as e.g. described on p. 211 of stat.cmu.edu/~larry/=sml/Boot.pdf). I'm not sure what can, in general, be said about the relationship between the bootstrap variance and a bootstrap CI. But I think a CI is what you really want. $\endgroup$
    – Adam L. Taylor
    Feb 2, 2021 at 4:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.