6 votes

Is it OK to choose the MH proposal as the prior in a posterior simulation problem?

Besides the inefficiency of using the prior pointed out in other answers, there is one specific setting where one cannot use the prior distribution as proposal. This is when the prior distribution $\...
Xi'an's user avatar
  • 106k
6 votes

Why does MCMC estimate the variance in a logit-normal model incorrectly?

This is a specific issue with the logit-normal distribution: since the noise is on the level of the logits, noise observations are of the form $\epsilon = \mathrm{logit}(X) - \mathrm{logit}(U)$. I ...
Eike P.'s user avatar
  • 3,058
5 votes
Accepted

Terms and assumptions in trans-dimensional MCMC (RJ-MCMC) for Green 1995 paper

The key idea here is that we want a ratio of forward and backward proposal measures that is finite and non-zero. If $x$ and $x'$ have different numbers of dimensions, you will get infinite or zero ...
Thomas Lumley's user avatar
5 votes

What is the correct implementation of MCMC

You've specified a class of MCMC algorithms referred to as Metropolis-Hastings. With these algorithms, choosing the proposal distribution is the key question when specializing the algorithm. Under ...
Cliff AB's user avatar
  • 21.1k
4 votes
Accepted

MCMC seems very sensible to the evidence

Since $$A(\theta|\theta') = \min\left(\frac{\tilde{P}(\theta|D)}{\tilde{P}(\theta'|D)}, 1\right) = \min\left(\frac{{P}(D|\theta)P(\theta)}{P(D|\theta')P(\theta')}, 1\right)$$ the evidence $P(D)$ does ...
Xi'an's user avatar
  • 106k
4 votes
Accepted

How does Rao-Blackwellization of the Metropolis-Hastings algorithm work?

Let me try to summarise here the argument of our paper. (Note that I am mostly using different notations, cut&pasted from my set of slides.) The original Metropolis-Hastings estimate of $\mathbb E[...
Xi'an's user avatar
  • 106k
3 votes
Accepted

Writing MCMC Sampling Code by Hand

The code appears to be correct. Note however that the step ...
Xi'an's user avatar
  • 106k
3 votes

Estimating sigma in Bayesian inference

A completely flat inverse-Gamma, i.e. letting the shape and scale tend to zero, will often lead to the same problems in practice. Gelman's 2006 paper on prior distributions for variance parameters is ...
Doctor Milt's user avatar
  • 3,216
3 votes
Accepted

Sampling from an approximate distribution to estimate posterior mean

The issue with the question is that the expression$$\mathbb E[\theta_1|x]$$is not well-defined: either $(\theta_1,\theta_2)$ is considered a random vector with joint prior $\pi(\theta_1,\theta_2)$, ...
Xi'an's user avatar
  • 106k
3 votes
Accepted

What could lead to this misbehavior for the expected sample size (ESS)?

This is actually not an error - it is possible for the effective sample size to be larger than the actual sample size. This means that your MCMC samples provide more information about the parameter, ...
Anders Gorm's user avatar
2 votes

Is there a Quasi-Monte Carlo variant of the Metropolis-Hastings algorithm?

The obvious question is: How do we need to compute the acceptance probability 𝛼? Or the different question is "Why do we need to compute the acceptance probability 𝛼?" If your point is to ...
Sextus Empiricus's user avatar
2 votes

Advice on sensitivity analysis for priors in Bayesian statistics

A reasonable place to start in this particular case is to recognize that the model is unidentified: tau1 and tau2 cannot be ...
dipetkov's user avatar
  • 9,940
2 votes

Sampling from the posterior with a constraint on the posterior mean

Given that both $X$ and $Y$ are integer valued, e.g. with a finite number of values, a likely trick in deriving the marginals of $X$ and $Y$ stands in writing enough linear relations between $\mathbb ...
Xi'an's user avatar
  • 106k
2 votes

Is it possible to merge credible intervals from different Bayesian prediction models into a single estimate?

Here is one way to think about problems of this sort. This approach may or may not be directly applicable to your case, in part depending on whether the assumptions are reasonable for your case. ...
mef's user avatar
  • 3,226
2 votes

How does Rao-Blackwellization of the Metropolis-Hastings algorithm work?

In order for myself to understand this better, and hopefully for others as well, I have made the graphical representation below. bottom panel What the algorithm effectively does is creating a sample ...
Sextus Empiricus's user avatar
1 vote
Accepted

Show that the total variation distance of the Metropolis kernel to its proposal kernel is equal to the rejection probability

Conditional on $x$, the "coupling" interpretation is to start with the move attached to $Q(x,\cdot)$ and to propose to couple the output with $\kappa(x,\cdot)$, a coupling that is accepted ...
Xi'an's user avatar
  • 106k
1 vote
Accepted

How should you determine the probability returned by a flat uniform prior function

There is a more formal answer to your question, but I won't give it here. In broader terms, though: What I don't understand is why they would take the log of 1 and not the log of 1/5.5, which is the ...
Eoin's user avatar
  • 9,007
1 vote
Accepted

BVAR model: Draws and Burn-In?

In a Bayesian model, we are often mainly interested in the posterior distribution, as it describes our knowledge about the parameters of interest given our priors and after having seen the data. Now, ...
Christoph Hanck's user avatar
1 vote
Accepted

How can I sample a multivariate normal vector that satisfies a linear equality constraint?

Our approach will be to extend the matrix $A$ to a full rank $n\times n$ matrix $C = \begin{pmatrix}P \\ A \end{pmatrix}$, find the conditional distribution of $PX$ given $AX = b$, sample $Px$, then ...
David Thiessen's user avatar
1 vote
Accepted

Adjustment needed for multivariate Dvoretzky–Kiefer–Wolfowitz inequality on MCMC samples?

No, this isn't going to work in any straightforward way. The difference between IID and Markov Chain bounds is the whole problem. That is, going from bounds for a mean to bounds for an empirical CDF ...
Thomas Lumley's user avatar
1 vote

How to diagnose HMC results like r-hat for a Mixture Model?

This is only a partial answer, but in general you can go a long way by adding an order constraint to your model: enforcing $\theta_1 \lt \theta_2 \lt\dots\lt \theta_k$. This is trickier to do if $\...
Eoin's user avatar
  • 9,007
1 vote
Accepted

Jacobian and proposal ratio of Birth/death step in RJMCMC of Gaussian mixture model

For the birth step, we have to create $𝑤_{𝑗^∗}$ and $(μ_{𝑗^∗},σ_{𝑗^∗})$ pair and death step, pair of $𝑤_{𝑗^∗}$ and $(μ_{𝑗^∗},σ_{𝑗^∗})$ are deleted. Correct. However, the probability $A$ in (...
Xi'an's user avatar
  • 106k
1 vote

Assumptions and setting for bayesian mixture model (for RJMCMC)

When considering a random variable distributed from a mixture model $$Y\sim\sum_{j=1}^k w_j f(y|\theta_j)\tag{1}$$ this random variable can be expressed as the marginal of a pair $(Y,Z)$ of random ...
Xi'an's user avatar
  • 106k
1 vote

Why do we need to scale the variables in a Bayesian model?

In general, you do not need to scale the variables for a Bayesian model. This may be needed in some, specific, scenarios where you run into numerical issues with the sampling or optimization algorithm ...
Tim's user avatar
  • 138k
1 vote

How can I pool Bayesian parameter estimates after multiple imputation?

Combine the samples from all the individual fits to the imputed data and then use the pooled posterior samples as you would a single fit (e.g. to compute expectations, quantiles, ...). As the number ...
Martin Modrák's user avatar
1 vote

Running Metropolis-Hastings algorithm with changing proposal kernel; each time the kernel is changing starting the algorithm afresh. Does it work?

If I understand the proposed algorithm correctly, we can prove that this doesn't generally sample from the target distribution by way of counter example. And while this algorithm will be a bit ...
Cliff AB's user avatar
  • 21.1k
1 vote

Is it OK to choose the MH proposal as the prior in a posterior simulation problem?

You can choose any distribution you want as a proposal distribution, including, of course, the prior distribution. But there's an interesting thing: Suppose the likelihood function $L(\theta)=P(x|\...
C.K.'s user avatar
  • 131

Only top scored, non community-wiki answers of a minimum length are eligible