Questions tagged [metropolis-hastings]

A special type of Markov Chain Monte Carlo (MCMC) algorithm used to simulate from complex probability distributions. It is validated by Markov chain theory and offers a wide range of possible implementations.

Filter by
Sorted by
Tagged with
3
votes
0answers
774 views

Criteria in determining “step size” of Metropolis-hasting algorithms

I am training a complex Bayesian model using Gibbs sampling and Metropolis-Hasting algorithm. Most of the parameters are directly sampled by using conjugate priors except for 3 params which are ...
9
votes
1answer
1k views

MCMC in a frequentist setting

I have been trying to get a sense of the different problems in frequentist settings where MCMC is used. I am familiar that MCMC (or Monte Carlo) is used in fitting GLMMs and in maybe Monte Carlo EM ...
4
votes
1answer
126 views

What is the common criterion to decide the performance of prior selection in MCMC

For a model with likelihood $p(Y|\theta)$, in which $Y$ is the data and $\theta$ is the parameters. Based on Bayes Rule, we have the posterior $p(\theta|Y) \propto p(Y|\theta) p(\theta)$ My ...
3
votes
2answers
171 views

How to solve MALA when the target density is known up to a constant?

If you look at the wikipedia explanation of Metropolis adjusted Langevin Algorithm, the acceptance ratio is given by The second equation involves taking the gradient of the log of $\pi(x)$. However, ...
5
votes
1answer
1k views

Convergence of the Independent Metropolis-Hastings algorithm

I am interested in the convergence properties of the Metropolis-within-Gibbs sampler with Independent or Random walk. In this paper, I have read that in the case of an Independent walk, the proposal ...
1
vote
0answers
71 views

RW Metropolis and ARMS fail

I've been trying to estimate a series of simulated Gamma-distributed random variables and its structural parameters with MCMC for a stochastic volatility model. However, both the random walk ...
7
votes
2answers
885 views

Use of Metropolis & Rejection & Inverse Transform sampling methods

I know that the Inverse Transform method is not always a good option to sample from distributions because it is a analytical method dependent on the shape of the distribution function. For example, ...
2
votes
1answer
389 views

Metropolis–Hastings algorithm variance isn't converging in R?

I'm trying to simulate a sample from a t distribution with 4 degrees of freedom. The candidate density I'm using is a normal(0,1) distribution. Although the mean does converge to 0, the variance keeps ...
20
votes
1answer
668 views

Can adaptive MCMC be trusted?

I am reading about adaptive MCMC (see e.g., Chapter 4 of the Handbook of Markov Chain Monte Carlo, ed. Brooks et al., 2011; and also Andrieu & Thoms, 2008). The main result of Roberts and ...
2
votes
1answer
995 views

MCMC how to choose standard deviation of sampling density

I'm using Metropolis-Hastings MCMC to find the set of parameters $\theta$ of a model $M$ that best fits my experimental data $x$ (with noise), where it is possible to directly calculate $x$ (without ...
7
votes
1answer
139 views

Approximate Metropolis algorithm - does it make sense?

Some time ago Xi'an asked What is the equivalent for cdfs of MCMC for pdfs? The naive answer would be to use "approximate" Metropolis algorithm in form Given $X^{(t)} = x^{(t)}$ 1. generate $Y \...
1
vote
2answers
1k views

Is MCMC really better than raw MC to Sample a region?

I have implemented the Markov Chain Monte Carlo (MCMC) with the Metropolis-Hastings sample selection criteria. Basically, as I understand it and as I have implemented it is: ...
1
vote
1answer
529 views

Use of Metropolis-Hastings in Bayesian Inference

I am now studying the Metropolis-Hastings algorithm and I want to apply it in order to made a Bayesian Inference of a function $y=f(x)$ to a dataset $D=\{x_i,y_i\}$. Five parameters of the function ...
4
votes
2answers
2k views

Multi parameter Metropolis-Hastings

I need to formulate a multi parameter Metropolis-Hastings algorithm. My question is related to how to define the condition to accept or not the candidate value. In my problem (it is a curve fitting)...
1
vote
1answer
2k views

METROPOLIS-HASTINGS with likelihood

I am trying to set up a Metropolis-Hastings algorithm in Matlab in order to estimate the parameters ${\theta}$ (it is a vector of 5 elements) to fit a curve to a set of data $D={X_i,Y_i,\delta_i}$. $X$...
2
votes
1answer
430 views

Metropolis-Hastings Convergence in R

I'm trying to implement the algorithm Metropolis Hastings for the next FDP. \begin{equation} f_{X}(x)=(5+\exp(5/2))*\sqrt{2\pi}\left[\exp\left\{-\frac{(x-2)^{2}-2x}{2}\right\}+5\exp\left\{-\frac{x+2}{...
3
votes
1answer
490 views

What is the difference and relationship between posterior distribution function and likelihood function in MCMC?

I am learning MCMC in class, and I encounter one question about the relationship between posterior probability and likelihood function. In our lecture, the professor asked us to take samples from ...
3
votes
2answers
125 views

Metropolis-Hasting: compute acceptance based on asymmetric continous independent chain proposal

The title is a mouthful, but here is what it amounts to: Under a proposal distribution using an independent chain, the probability of jumping to point $x$ is independent of the current position $y$ ...
9
votes
1answer
6k views

Proposal distribution - Metropolis Hastings MCMC

In Metropolis-Hastings Markov chain Monte Carlo, the proposal distribution can be anything including the Gaussian (according to the Wikipedia). Q: What's the motivation for using anything other than ...
2
votes
1answer
134 views

Symmetric PDFs in Metropolis-Hastings

My textbook says that a symmetric PDF satisfies $$f(x|y)=f(y|x).$$ Can anyone explain this? Is it equivalent to $f(x+a)=f(x-a)$?
2
votes
1answer
494 views

MCMC Metropolis-Hastings initial values [closed]

my posterior values that I obtained via Metroplis-Hasting are always around my initial values. For instance if I chose $\theta_0 =(1,2)$ my posterior values, after either taking mean or median, are ...
4
votes
1answer
621 views

dmvnorm produce 0 likelihood

I am implementing an MCMC algorithm in R using the "mvtnorm" package. The data is about 150 dimensions so the likelihood produced by dmvnorm is usually zero (or -inf if "log=TRUE" is set), which make ...
2
votes
1answer
308 views

Multiple-Try Metropolis question

I read Multiple-Try Metropolis from Wikipedia and I do not understand some points. Suppose the current state is $\mathbf{x}$. The MTM algorithm is as follows: Draw ''k'' independent ...
1
vote
1answer
154 views

Are there analytically derivable posteriors that save from doing MCMC other than conjugate priors? [duplicate]

Posteriors for conjugate priors can be analytically derived and save us from doing MCMC. Conjugate priors simply have a posterior in the same family as the prior distribution. Are there other ...
6
votes
2answers
463 views

Bayes Factor approximation

A brute force method to approximate the Bayes Factor (the ratio of the denominators (normalizing constants) in the Bayes formula) is to do the following for the two models of interest: repeat ...
3
votes
0answers
127 views

Singular proposal in MCMC

Suppose we want to obtain samples of the density $f(\mathbf{x})$ where $\mathbf{x}$ is a $d$-dimensional vector, i.e. $\mathbf{x} = (x_1, x_2, \dots, x_d)$. To that end, we choose the Metropolis-...
5
votes
1answer
786 views

Fitting power function to data

I am trying to implement an MH algorithm to fit a power function to my data. The power function has the following form: $\hat{y} = a * x^b$ The data are assumed to be normally distributed ...
3
votes
3answers
736 views

A question about the choice and interpretation of the jumping distribution in Metropolis-Hastings algorithm

In order to implement the MH algorithm you need a proposal density or jumping distribution $q(⋅|⋅)$, from which it is easy to sample. If you want to sample from a distribution $f(⋅)$, the MH algorithm ...
1
vote
2answers
1k views

For Metropolis-Hastings algorithm, should target density and proposal distribution have the same distribution?

I watched some youtube videos about the Metropolis-Hastings algorithm. They used a Gaussian as a proposal function to estimate an unknown Gaussian, or used a Gamma function as the proposal function to ...
2
votes
1answer
259 views

Metropolis-Hastings simulation of independent geometric random variables

Consider the following Metropolis-Hastings scheme to sample independent geometric random variables $X = (X_1, \dots, X_N)$, where each $X_j$ has pmf $\mathbb{P}(X_j = x) = p(1-p)^x$ for $x \geq 0$. At ...
6
votes
1answer
2k views

Mathematical foundation of using MCMC in global optimization

MCMC is commonly used to compute the integral in the form of $$\text{Problem A.}~~\int F(x)\pi(x) $$ where $\pi$ is hidden. In the literature, it is explained why MCMC can handle problem A by ...
2
votes
0answers
67 views

Can someone explain Metropolis Hastings algorithm with simplest possible example? [duplicate]

I am just beginning to learn about Metropolis Hastings algorithm and MCMC techniques. I have a basic understanding of Markov chains and stationary distributions and need for the Metropolis Hasting but ...
42
votes
1answer
17k views

What is the difference between Metropolis-Hastings, Gibbs, Importance, and Rejection sampling?

I have been trying to learn MCMC methods and have come across Metropolis-Hastings, Gibbs, Importance, and Rejection sampling. While some of these differences are obvious, i.e., how Gibbs is a special ...
2
votes
2answers
154 views

“True” answer for MCMC model

Theoretically, given a model with $N$ parameters and $\forall x \in \mathbb{R}.\; p(x)>0$ in the prior of all parameters. If i'm interested only in the end result and not in time-to-convergence, ...
6
votes
1answer
1k views

Gibbs sampling from a complex full conditional

I have a sampling question relating to Gibbs sampling of a complicated full conditional. Supposed I have a complicated full conditional that I want a single sample from $p(\theta_i$|$\theta_{-i}$, $...
1
vote
0answers
278 views

MCMC with dependent variables

I want to run Metropolis-Hastings on a problem which involves two parameters that are not independent. I.e. I want to estimate both of these parameters. At the moment I'm trying to understand if this ...
2
votes
0answers
98 views

Conditional Density for Sigma (Bayesian Lasso)

I found that in Bayesian Lasso commonly $\beta \sim N(0,\sigma^2*diag(\tau))$ and $\sigma,\tau \sim \pi(\sigma,\tau)$ is used. Whereas $\pi(\cdot)$ is a product of Laplace distributions. Is it ...
1
vote
0answers
108 views

Narrow distribution and Hasting-Metropolis

I would like to sample from a density $\tilde{\pi}=C(\pi)\pi$ whose support is $[0,1]$. The normalization constant $C(\pi)$ of the function $\pi$ is unknown and $\pi$ is very narrow. To see how narrow ...
0
votes
0answers
267 views

Is it possible for Metropolis sampling to converge to the wrong value?

I have simulated data under three parameters of interest, say a, b, c. The prior I put on c was a Gamma, so it only takes positive values. The full conditionals of a and b are known distributions, but ...
0
votes
1answer
231 views

Metropolis sampling (symmetric proposal distribution)

Can Metropolis sampling be used in conjunction with Gibbs sampling? So for example, if I have three parameters of interest, but only two of them have full conditionals that are known distributions, ...
1
vote
0answers
96 views

Gibbs within Collapsed Gibbs?

I have a model with variables $X_{1}, X_{2}, X_{3}, X_{4}$. I would like to sample it within a larger MCMC chain using: $(X_{1}, X_{2}) \sim P(X_{1}, X_{2})$ $(X_{3}, X_{4}) \sim P(X_{3}, X_{4} \mid ...
0
votes
1answer
20 views

Odd Acceptance Ratio

Recently, I read some paper and sometimes they draw a sample $s\sim N(a,b)\times exp(d)$. But they defined the prior as $N(A,b) \times exp(D)$ with unknown $A$ and $D$. Therefore in the acceptance ...
3
votes
1answer
246 views

I Want to see Bi-Modal Posterior in Bayesian Linear Regression!

I'm playing around with a Metropolis-Hastings MCMC algorithm as described in this post. I made an example data set with points taken from two lines shown below. Both lines have a y-intercept of 0 ...
3
votes
1answer
666 views

Bayesian inference for power laws

Let $G$ be a graph with $N$ nodes. Let $p(d_i)$ be the probability of node $i$ to have $d$ connections. If this follows a power-law: $$ p(d_i) = \frac{d_i^\alpha}{\sum_{j=1}^{N} d_j^\alpha} $$ $\...
6
votes
1answer
508 views

Do sampling methods (MCMC/SMC) work for combination of continuous and discrete random variables?

Consider a distribution $P = \frac{1}{2}P_1 + \frac{1}{2}P_2$ where $P_1, P_2$ are probability measures on a measurable space $(\mathbb R, \mathcal B)$ such that $P_1: A \to \int_A \mathcal N(x; 0, 1)...
2
votes
1answer
1k views

What is the acceptance ratio? (Metropolis-Hastings)

I have a really basic question. But I am a little bit confused about that. How do I calculate the acceptance ratio within a Metropolis-Hasting step? I have something like $min\left\{1,\frac{p(new)}{p(...
2
votes
2answers
847 views

Likelihood overflow in Metropolis-Hastings acceptance probability

Consider a Bayesian framework where we have priors for some parameters and a likelihood based on the data. Consider the likelihood (and its parametric format) to be very sensitive to the choice of the ...
0
votes
0answers
86 views

Sampling the scale parameter of a Laplace distribution

I need to adjust the scale parameter $\lambda$ of a Laplace prior ($p(x|\lambda)=(1/2\lambda)* exp(-|x|/\lambda)$) within metropolis hastings. That means I have a couple of draws for x and now I have ...
2
votes
1answer
513 views

Sampling on a logarithmic scale

I have to draw samples (variance parameter) based on a Gaussian kernel but on a logarithmic scale. I have no clue how to implement that as a part of the Metropolis-Hastings algorithm. In particular, ...
4
votes
1answer
1k views

Gibbs sampling with mixed prior using a Metropolis-Hastings step

My questions are about a sampling procedure for fitting a Bayesian hierarchical model where one of the priors is a mixture distribution of discrete and continuous parts. The model is not my own but I ...

1
4 5
6
7 8