Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [metropolis-hastings]

A special type of Markov Chain Monte Carlo (MCMC) algorithm used to simulate from complex probability distributions. It is validated by Markov chain theory and offers a wide range of possible implementations.

0
votes
1answer
12 views

Direct/indirect sampling of conditionals in Gibbs sampling

I have some problems understanding the definition of Gibbs sampling. Let us take into consideration a bivariate distribution \begin{equation} \pi(x_1,x_2): S \subset \mathcal{R^2} \rightarrow \...
0
votes
1answer
28 views

How to Show that a Distribution is a Stationary Distribution for Metropolis-Hastings? [closed]

For an Ising Model with a (2L+ 1) by (2L+ 1) square grid of magnetic particles, show that $$\pi(\xi)=\frac{1}{Z_\beta}e^{\beta\sum_{x,y=x}{\xi_x\xi_y}}$$ Is indeed a stationary distribution for the ...
1
vote
1answer
40 views

MCMC - target distribution, proposal distribution and likelihood function?

i have just started out in MCMC and I am not sure if I fully understand the concepts of MCMC with respect to the above terms. Let me try to explain that in my own words and include some thoughts / ...
2
votes
0answers
35 views

How to implement a M-H step in a Gibbs sampling

I am having trouble implementing a Metropolis Hastings step in a Gibbs sampling problem. The following code was taken from https://www.stat.colostate.edu/computationalstatistics/ Details: It is a ...
0
votes
0answers
15 views

Which gradient to compute in a hierarchical model for M-H MCMC?

We have the following model: $$y_t=Mx_t+\epsilon_t$$ with $M$ being a matrix such that $M\sim F_{\lambda}$(assume it's a conjugate prior). The $\lambda$ does not appear in $M$, only in its ...
3
votes
1answer
48 views

Estimate the asymptotic efficiency of a Markov chain sampling by the method of batching

In the paper Efficient Metropolis Jumping Rules, the author is writing that he used "the method of batching" for the estimation of $\operatorname{eff}_{\overline\theta_i}$ in Table 1 (on page 605). ...
3
votes
1answer
192 views

How does the Metropolis Algorithm “get off the ground”?

I'm thoroughly confused by the Metropolis Algorithm as defined in Casella and Berger's Statistical Inference. Namely, here's the definition (p.254): Let $Y \sim f_Y(y)$ and $V \sim f_V(v)$, where $...
6
votes
1answer
81 views

How can we verify the intuition that in the RW-Metropolis-Hastings algorithm with Gaussian proposal too small and too large variances are bad choices

Let $d\in\mathbb N$ and consider the Random Walk Metropolis-Hastings algorithm with a Gaussian proposal kernel $Q$ such that $Q(x,\;\cdot\;)=\mathcal N_d(x,\sigma^2_dI_d)$ for all $x\in\mathbb R^d$. ...
0
votes
1answer
41 views

Relation between Uniform distribution, Metropolis Algorithm, and Symmetric Proposal Distribution

I am having some confusion over the Metropolis algorithm. Let $g(x|y)$ be our proposal distribution for the algorithm. For the Metropolis, $g$ must be symmetric (from Wikipedia). In the discrete case, ...
2
votes
0answers
15 views

Using some objective priors (for unbounded space) in a Metropolis-Hastings MCMC

I'm doing some simulations using a M-H MCMC, and I was thinking of using some objective priors for some parameters. These parameters must be in $\mathbb{R}^+$. I was thinking of using $\pi(\theta)\...
5
votes
1answer
170 views

Likelihood modification in Metropolis Hastings ratio for transformed parameter

I want to use MH to get samples from $p(\theta \mid y) \approx p(y \mid \theta) p(\theta)$. Let's assume $\theta$ is heavily constrained and I transform $\theta$ to $f(\theta)$ so I can sample from ...
4
votes
3answers
228 views

Conditional distribution of $\exp(-|x|-|y|-a \cdot |x-y|)$

I am trying to use Gibbs sampling or Metropolis-Hastings to draw samples from the joint distribution$$f(x,y)\propto\exp(-|x|-|y|-a \cdot |x-y|)$$ For this I need the conditional distributions of $x$ ...
2
votes
1answer
37 views

slice sampling correctness

Theoretically, the slice sampling has equilibrium distribution as the target distribution. If we can sample exactly as follows, $y' = U(0, p^*(x))$ $x' = U\{x: p^*(x) > y' \}$ However, in the ...
0
votes
0answers
49 views

Metropolis-Hastings algorithm for autocorrelated data

I have auto-correlated data and I wish to apply the Metropolis-Hastings algorithm on it. The data was obtained by simulating the time evolution of a system, and computing the values of some magnitude ...
1
vote
0answers
25 views

Convergence in total distribution distance in the Random Walk Metropolis-Hastings algorithm

I'm searching for a proof of the convergence in total distribution distance of the transition probabilities of a Markov chain generated by the Random Walk Metropolis-Hastings algorithm to its ...
1
vote
0answers
20 views

Assumptions on the target density in the RWM optimal scaling paper by Roberts, Gelman and Gilks

In the famous paper Weak Convergence and Optimal Scaling of Random Walk Metropolis Algorithms by Roberts, Gelman and Gilks, at the bottom of page 116, the supremum of the third derivative of $\ln f$ ...
0
votes
0answers
20 views

Hamiltonian MCMC information gathering [duplicate]

I started gathering information about Hamiltonian MCMC and I would like to ask if someone knows some good papers or books.If it possible notes that give a detailed explanation of Hamiltonian MCMC. ...
-1
votes
1answer
31 views

2 versions of Metropolis-Hastings : are they equivalent?

I have seen 2 different versions of Metropolis algorithm. First one : Second one : I don't understand the differences between the 2 versions, especially in the second one where I have to use the ...
0
votes
1answer
101 views

Metropolis Hastings - Acceptance ratio, proposal and lkelihood

From a previous post : First to explain the MH algorithm, it's used to approximate numerically a target distribution, in this case $p(\theta|D)$. At each stage of the algorithm: A value ...
4
votes
1answer
74 views

How are deltas chosen for the proposal distribution in multivariate metropolis hastings sampling?

Say I want to use Metropolis Hastings algorithm to get posterior draws of multivariate parameters. In the one variable case, you could manipulate delta until you found something that worked (gave 40% ...
1
vote
1answer
45 views

Metropolis algorithm to Bernoulli likelihood and beta prior (Kruschke 7.3.1)

This question pertains to a specific line written in the book Doing Bayesian Data Analysis by John K. Kruschke. In section 7.3.1, he applies Metropolis algorithm to a case with: $prior = beta(\...
1
vote
0answers
12 views

Some priors for these parameters with a preferential range, and a proposal?

I have the following parameters, with a specified range, but with also a preferential range of values. I'm looking for some priors for them. Later on, I will have to use a Metropolis-Hastings type of ...
1
vote
0answers
70 views

Some pointers on constructing a proposal density in a Metropolis-Hastings Algorithm for Gaussian Processes

I have a likelihood similar to a Normal density. I'm thinking of using matrix-valued covariance function a $\sum^Q_{q=1}A_q \otimes k(X,X)$, where $k$ is a Matérn class covariance function. The other ...
0
votes
2answers
83 views

Rule of thumb for number of iterations Metropolis Hastings

Is there a rule of thumb for the number of iterations needed for the Metropolis Hastings algorithm? I would appreciate good references
2
votes
0answers
116 views

Metropolis sampling for Bayesian networks

Gibbs sampling is a profound and popular technique for creating samples of Bayesian networks (BNs). Metropolis sampling is another popular technique, though - in my opinion - a less accessible method. ...
0
votes
1answer
47 views

Monte Carlo Metropolis: Standard Error and Acceptance

In a time series data generated by Monte Carlo Metropolis algorithm, when is the standard error (correlation between two data points is assumed to be negligible) is higher - when the change in the ...
0
votes
1answer
125 views

Is the truncated normal distribution symmetric?

I am running a Metropolis-Hastings MCMC to find the distribution of a parameter that takes real, positive values. I was considering using the truncated normal distribution, and was wondering if I have ...
0
votes
0answers
82 views

Autocorrelation, Autocovariance and Large lag Standard Error

I have time series generated data from Monte Carl-Metropolis Simulation. I have estimated correlation coefficients using: $r_k = \frac{c_k}{c_0}$ where $c_0$ is the varaiance and $c_k = \frac{1}{N}\...
0
votes
0answers
30 views

Standard Error in Auto correlated Data

I have time-series data generated via Metropolis algorithm - Monte Carlo simulations. Since these data must have some correlation between them, the formula of the standard error for IIDs variable must ...
0
votes
1answer
138 views

Metropolis-Hastings in a Bayesian Hierarchical model

I am trying to estimate a Bayesian Hierarchical model using the random-walk Metropolis-Hastings algorithm. While in a non-Hierarchical model, the algorithm is staight-forward, I am not sure I am ...
1
vote
1answer
70 views

MCMC samples for constructing a histogram

I am interested in generating samples from a density $\pi(\theta)$ to construct a histogram for $\pi(\theta)$ and to use these samples to generate samples of $f(\theta)$ for some function $f$. I may ...
1
vote
2answers
86 views

What to do once states are rejected in MCMC?

I need to generate samples from a pdf given by $\frac{f_Z(z)\cdot 1_{Z \in B}}{P(Z \in B)}$ where $Z \in \mathbb{R}^d$ is a normal random vector with independent components. $Z \in B$ is a set that is ...
1
vote
0answers
20 views

Choosing a custom proposal distribution in Metropolis-Hastings Monte Carlo

I have many states and have calculated a good custom proposal distribution for my Monte Carlo simulation. The system reaches a good solution faster than if it were to just use a randomly selected ...
1
vote
1answer
36 views

How can the support of proposal distribution impact convergence of RH-MH algorithm?

In the book Introducing Monte Carlo Methods by Casella and Robert, there's a sentence with which I'm having some trouble to understand. «If the domain explored in $q$ [proposal] is too small, ...
1
vote
1answer
54 views

Understanding the Delayed Rejection Metropolis algorithm (Mira, 2001a)

I'm having trouble understanding the algorithm as briefly described here, and I can't find the original paper by Mira since it seems to be from some obscure print journal (Metron Volume 59). The ...
1
vote
0answers
52 views

Understanding Adaptive Metropolis MCMC by Haario et al. 2001 [closed]

I'm using the Delayed Rejection Adaptive Metropolis (DRAM) algorithm (Haario et al., 2006) for some Bayesian inference and trying to get an intuition for it so I can be sure to use it properly. So far ...
2
votes
1answer
29 views

Metropolis Hastings algorithm without enough data [closed]

In a metropolis hastings algorithm if i have not data or enough data, this will give me the prior means? I am asking this because I have made an algorithm and when i use just a few data this is not ...
1
vote
1answer
110 views

Implementation of Metropolis-Hastings with conditional posterior

I'm trying to understand how to estimate the parameter vector $\mathbf{\theta} = (\theta_1,\theta_2, \theta_3)$ of a model using the MH algorithm. I am given a joint posterior density: $p(\mathbf{\...
2
votes
0answers
24 views

GARCH(2, 3) model with Metropolis-Hastings algorithm

Let's say I have a $GARCH(2, 3)$ model with $$\nu_i = \sigma_i\epsilon_i$$ where $\epsilon_i \sim N(0, 1)$ and $$\sigma_i^2 = a_0 + \sum\limits_{k = 1}^{2} a_k\sigma_{i - k}^2 + \sum\limits_{l = 1}^{3}...
1
vote
1answer
180 views

Formula for Autocorrelation

I have time-series data generated via Metropolis algorithm - Monte Carlo simulations. I need to know correlation between data points generated given by $r_k = c_k/c_0$ where $c_0$ is the variance of ...
3
votes
2answers
281 views

Why take the minimum in the acceptance ratio in the Metropolis-Hastings algorithm?

The Metropolis-Hastings ratio is defined as $$ \alpha(x'|x) = \min\left(1, \frac{P(x')g(x|x')}{P(x)g(x'|x)}\right) $$ and the state $x'$ is accepted if $u \leq \alpha(x'|x)$, where $u$ is ...
0
votes
0answers
75 views

The Hessian of multinomial Probit model

I wanted to implement multinomial probit in Bayesian with random-walk Metropolis Hasting. To achieve the best numerical efficiency when drawing $\beta$, I need to use the hessian matrix of $\beta$. ...
0
votes
0answers
21 views

Markov Chain Monte Carlo [duplicate]

what are the differences between M-H algorithm and M-H-within-Gibbs algorithm. If possible, upload for me the two algorithms please.
15
votes
1answer
1k views

Stan $\hat{R}$ versus Gelman-Rubin $\hat{R}$ definition

I was going through the Stan documentation which can be downloaded from here. I was particularly interested in their implementation of the Gelman-Rubin diagnostic. The original paper Gelman & ...
2
votes
1answer
87 views

Can the Acceptance rate for Metropolis-Hastings be greater than 1?

Can the acceptance rate in MH algo be greater than 1? When that case occurs the proposal will off coruse be accepted with probability 1. But is it "ok" to allow a acceptance rate greater than 1?
1
vote
1answer
129 views

Metropolis-Hastings acceptance ratio for truncated proposal

I have a proposal distribution for one parameter theta_guess theta_guess = guessleft(theta_accept(1,r-1), 0.01,0) which is a ...
3
votes
2answers
256 views

Compute the likelihood in Metropolis–Hastings: How does it relate to a posterior in Bayesian Analysis?

Basic question about MCMC Metropolis–Hastings algorithm. I am trying to understand the Metropolis–Hastings algorithm and it's connection to Bayesian Analysis. Suppose I want to construct an MCMC MH ...
1
vote
0answers
50 views

Metropolis Hastings Kernel - Relation Indicator function and Dirac Mass

Why does the indicator function is equivalent to the integral over the Dirac mass? In my lecture notes the proof for the Kernel of the Metropolis Hastings is given as follows: $$P(X^t \in \mathcal{X}...
1
vote
1answer
82 views

Convergence issue in simple 1D Metropolis algorithm

I want to write a Metropolis sampler to sample independent rvs $x$ from the mixture model $X \sim \frac{1}{2}\big[\mathscr{N}(\mu_1, \sigma_1) + \mathscr{N}(\mu_2, \sigma_2)\big]$. My algorithm is ...
8
votes
1answer
83 views

sampling cost of $O(d)$ versus $O(2^d)$

I came across the following simulation problem: given a set $\{\omega_1,\ldots,\omega_d\}$ of known real numbers, a distribution on $\{-1,1\}^d$ is defined by $$\mathbb{P}(X=(x_1,\ldots,x_d))\propto (...