Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [metropolis-hastings]

A special type of Markov Chain Monte Carlo (MCMC) algorithm used to simulate from complex probability distributions. It is validated by Markov chain theory and offers a wide range of possible implementations.

0
votes
0answers
14 views

Hamiltonian MCMC information gathering [duplicate]

I started gathering information about Hamiltonian MCMC and I would like to ask if someone knows some good papers or books.If it possible notes that give a detailed explanation of Hamiltonian MCMC. ...
-1
votes
1answer
24 views

2 versions of Metropolis-Hastings : are they equivalent?

I have seen 2 different versions of Metropolis algorithm. First one : Second one : I don't understand the differences between the 2 versions, especially in the second one where I have to use the ...
0
votes
1answer
46 views

Metropolis Hastings - Acceptance ratio, proposal and lkelihood

From a previous post : First to explain the MH algorithm, it's used to approximate numerically a target distribution, in this case $p(\theta|D)$. At each stage of the algorithm: A value ...
0
votes
0answers
22 views

Multi parameters - Metropolis Hastings Algorithm - Concrete example [closed]

have a little project for which I have to estimate parameters on a PSF (Point Spread Function = response of the system to a dirac, i.e a star in my case). Thanks to the Metropolis-Hastings algorithm, ...
4
votes
1answer
46 views

How are deltas chosen for the proposal distribution in multivariate metropolis hastings sampling?

Say I want to use Metropolis Hastings algorithm to get posterior draws of multivariate parameters. In the one variable case, you could manipulate delta until you found something that worked (gave 40% ...
1
vote
1answer
40 views

Metropolis algorithm to Bernoulli likelihood and beta prior (Kruschke 7.3.1)

This question pertains to a specific line written in the book Doing Bayesian Data Analysis by John K. Kruschke. In section 7.3.1, he applies Metropolis algorithm to a case with: $prior = beta(\...
1
vote
0answers
12 views

Some priors for these parameters with a preferential range, and a proposal?

I have the following parameters, with a specified range, but with also a preferential range of values. I'm looking for some priors for them. Later on, I will have to use a Metropolis-Hastings type of ...
1
vote
0answers
56 views

Some pointers on constructing a proposal density in a Metropolis-Hastings Algorithm for Gaussian Processes

I have a likelihood similar to a Normal density. I'm thinking of using matrix-valued covariance function a $\sum^Q_{q=1}A_q \otimes k(X,X)$, where $k$ is a Matérn class covariance function. The other ...
0
votes
2answers
54 views

Rule of thumb for number of iterations Metropolis Hastings

Is there a rule of thumb for the number of iterations needed for the Metropolis Hastings algorithm? I would appreciate good references
1
vote
0answers
69 views
+100

Metropolis sampling for Bayesian networks

Gibbs sampling is a profound and popular technique for creating samples of Bayesian networks (BNs). Metropolis sampling is another popular technique, though - in my opinion - a less accessible method. ...
0
votes
1answer
26 views

Monte Carlo Metropolis: Standard Error and Acceptance

In a time series data generated by Monte Carlo Metropolis algorithm, when is the standard error (correlation between two data points is assumed to be negligible) is higher - when the change in the ...
0
votes
1answer
55 views

Is the truncated normal distribution symmetric?

I am running a Metropolis-Hastings MCMC to find the distribution of a parameter that takes real, positive values. I was considering using the truncated normal distribution, and was wondering if I have ...
0
votes
0answers
39 views

Autocorrelation, Autocovariance and Large lag Standard Error

I have time series generated data from Monte Carl-Metropolis Simulation. I have estimated correlation coefficients using: $r_k = \frac{c_k}{c_0}$ where $c_0$ is the varaiance and $c_k = \frac{1}{N}\...
0
votes
0answers
20 views

Standard Error in Auto correlated Data

I have time-series data generated via Metropolis algorithm - Monte Carlo simulations. Since these data must have some correlation between them, the formula of the standard error for IIDs variable must ...
0
votes
1answer
55 views

Metropolis-Hastings in a Bayesian Hierarchical model

I am trying to estimate a Bayesian Hierarchical model using the random-walk Metropolis-Hastings algorithm. While in a non-Hierarchical model, the algorithm is staight-forward, I am not sure I am ...
1
vote
1answer
43 views

MCMC samples for constructing a histogram

I am interested in generating samples from a density $\pi(\theta)$ to construct a histogram for $\pi(\theta)$ and to use these samples to generate samples of $f(\theta)$ for some function $f$. I may ...
1
vote
2answers
72 views

What to do once states are rejected in MCMC?

I need to generate samples from a pdf given by $\frac{f_Z(z)\cdot 1_{Z \in B}}{P(Z \in B)}$ where $Z \in \mathbb{R}^d$ is a normal random vector with independent components. $Z \in B$ is a set that is ...
1
vote
0answers
17 views

Choosing a custom proposal distribution in Metropolis-Hastings Monte Carlo

I have many states and have calculated a good custom proposal distribution for my Monte Carlo simulation. The system reaches a good solution faster than if it were to just use a randomly selected ...
1
vote
1answer
29 views

How can the support of proposal distribution impact convergence of RH-MH algorithm?

In the book Introducing Monte Carlo Methods by Casella and Robert, there's a sentence with which I'm having some trouble to understand. «If the domain explored in $q$ [proposal] is too small, ...
1
vote
1answer
49 views

Understanding the Delayed Rejection Metropolis algorithm (Mira, 2001a)

I'm having trouble understanding the algorithm as briefly described here, and I can't find the original paper by Mira since it seems to be from some obscure print journal (Metron Volume 59). The ...
1
vote
0answers
37 views

Understanding Adaptive Metropolis MCMC by Haario et al. 2001 [closed]

I'm using the Delayed Rejection Adaptive Metropolis (DRAM) algorithm (Haario et al., 2006) for some Bayesian inference and trying to get an intuition for it so I can be sure to use it properly. So far ...
2
votes
1answer
29 views

Metropolis Hastings algorithm without enough data [closed]

In a metropolis hastings algorithm if i have not data or enough data, this will give me the prior means? I am asking this because I have made an algorithm and when i use just a few data this is not ...
1
vote
1answer
61 views

Implementation of Metropolis-Hastings with conditional posterior

I'm trying to understand how to estimate the parameter vector $\mathbf{\theta} = (\theta_1,\theta_2, \theta_3)$ of a model using the MH algorithm. I am given a joint posterior density: $p(\mathbf{\...
2
votes
0answers
21 views

GARCH(2, 3) model with Metropolis-Hastings algorithm

Let's say I have a $GARCH(2, 3)$ model with $$\nu_i = \sigma_i\epsilon_i$$ where $\epsilon_i \sim N(0, 1)$ and $$\sigma_i^2 = a_0 + \sum\limits_{k = 1}^{2} a_k\sigma_{i - k}^2 + \sum\limits_{l = 1}^{3}...
1
vote
1answer
38 views

Formula for Autocorrelation

I have time-series data generated via Metropolis algorithm - Monte Carlo simulations. I need to know correlation between data points generated given by $r_k = c_k/c_0$ where $c_0$ is the variance of ...
3
votes
2answers
262 views

Why take the minimum in the acceptance ratio in the Metropolis-Hastings algorithm?

The Metropolis-Hastings ratio is defined as $$ \alpha(x'|x) = \min\left(1, \frac{P(x')g(x|x')}{P(x)g(x'|x)}\right) $$ and the state $x'$ is accepted if $u \leq \alpha(x'|x)$, where $u$ is ...
0
votes
0answers
36 views

The Hessian of multinomial Probit model

I wanted to implement multinomial probit in Bayesian with random-walk Metropolis Hasting. To achieve the best numerical efficiency when drawing $\beta$, I need to use the hessian matrix of $\beta$. ...
0
votes
0answers
21 views

Markov Chain Monte Carlo [duplicate]

what are the differences between M-H algorithm and M-H-within-Gibbs algorithm. If possible, upload for me the two algorithms please.
14
votes
1answer
700 views

Stan $\hat{R}$ versus Gelman-Rubin $\hat{R}$ definition

I was going through the Stan documentation which can be downloaded from here. I was particularly interested in their implementation of the Gelman-Rubin diagnostic. The original paper Gelman & ...
2
votes
1answer
67 views

Can the Acceptance rate for Metropolis-Hastings be greater than 1?

Can the acceptance rate in MH algo be greater than 1? When that case occurs the proposal will off coruse be accepted with probability 1. But is it "ok" to allow a acceptance rate greater than 1?
1
vote
1answer
70 views

Metropolis-Hastings acceptance ratio for truncated proposal

I have a proposal distribution for one parameter theta_guess theta_guess = guessleft(theta_accept(1,r-1), 0.01,0) which is a ...
3
votes
2answers
214 views

Compute the likelihood in Metropolis–Hastings: How does it relate to a posterior in Bayesian Analysis?

Basic question about MCMC Metropolis–Hastings algorithm. I am trying to understand the Metropolis–Hastings algorithm and it's connection to Bayesian Analysis. Suppose I want to construct an MCMC MH ...
1
vote
0answers
31 views

Metropolis Hastings Kernel - Relation Indicator function and Dirac Mass

Why does the indicator function is equivalent to the integral over the Dirac mass? In my lecture notes the proof for the Kernel of the Metropolis Hastings is given as follows: $$P(X^t \in \mathcal{X}...
1
vote
1answer
63 views

Convergence issue in simple 1D Metropolis algorithm

I want to write a Metropolis sampler to sample independent rvs $x$ from the mixture model $X \sim \frac{1}{2}\big[\mathscr{N}(\mu_1, \sigma_1) + \mathscr{N}(\mu_2, \sigma_2)\big]$. My algorithm is ...
7
votes
1answer
71 views

sampling cost of $O(d)$ versus $O(2^d)$

I came across the following simulation problem: given a set $\{\omega_1,\ldots,\omega_d\}$ of known real numbers, a distribution on $\{-1,1\}^d$ is defined by $$\mathbb{P}(X=(x_1,\ldots,x_d))\propto (...
0
votes
0answers
42 views

How does Metropolis acceptance rate vary with the number of dimensions?

Intuitively, if I want to update two parameters in one step, I have to come up with a proposal that are good for both parameters. Assuming that the parameters are independent, is it correct to ...
0
votes
1answer
206 views

Show How to Generate a Poisson Random Variable with Parameter $\lambda$ using Metropolis-Hastings

Additionally: Use a simple symmetric random walk as the proposal distribution. Source: "Introduction to Stochastic Processes with R" - Robert P. Dobrow, Chapter 5 Exercises: Question 5.6 I know this ...
3
votes
2answers
82 views

Is it possible to merge acceptance probability with proposal distribution in Metropolis Hastings algorithm?

For an ergodic Markov chain, it doesn't necessarily have to be $Detailed\ Balanced $ when it converges to stationary distribution, which means that: $\pi(\theta)\ P(\theta^{\prime}|\theta) \neq \pi(\...
1
vote
1answer
176 views

MCMC: How to choose an efficient proposal distribution with continuous and discrete variables

I am using MCMC with the Metropolos-Hasting algorithm to generate solutions of a non linear regression problem. Likelihood My likelihood is a gaussian distribution centered in 0 of the residuals ...
5
votes
1answer
132 views

sampling from an unnormalised distribution

If one has to sample (with replacement) from a population $(x_1,x_2,\ldots)$ with weights $(\omega_1,\omega_2,\ldots)$, possibly infinite (although this is asking too much without further details), a ...
3
votes
1answer
187 views

Strategies to re-write the slow part of Metropolis Hastings in Rcpp [closed]

I want to speed up my R implementation of a Metropolis Hasting procedure by replacing the slow parts with functions written in Rcpp. There are already some examples online using Rcpp to speed up ...
3
votes
1answer
744 views

Metropolis-Within-Gibbs sampling with only marginal distribution known for a subset of variables

Typically in Gibbs sampling we want to sample from a joint distribution $p(X_1, X_2, ..., X_N)$, but because the joint is hard to sample from directly, we instead achieve this by iteratively sampling ...
1
vote
1answer
106 views

Question on Detailed Balance and Bayes' Rule

I think I am confused due to the lax notation typically used when dealing with probabilities and not having a formal probability background. Bayes' Rule tells me that $$Pr(X_t=a|X_{t+1}=b)Pr(X_{t+1}=...
2
votes
1answer
277 views

Why does detailed balance not provide a stopping criterion in MCMC?

Like I undestand MCMC sampling, the fulfillment of the detailed balance equation guarantees that our MC has reached its stationary distribution (given we ensure ergodicity). Detailed Balance is: $\...
3
votes
1answer
90 views

Metropolis-Hastings for heteroskedastic regression

Consider a heteroskedastic model of the form, $y_i|x_i \sim \mathcal{N}\left(x_i, \text{exp}\{\boldsymbol\beta^\top\boldsymbol{x}\}\right)$ where $\boldsymbol{\beta}=\left[\beta_0,\beta_1\right]$ and $...
4
votes
1answer
99 views

What is the name of this random variable?

Let $X \sim \text{Normal}(\mu, \sigma^2)$. Define $Y = \frac{e^X -1}{e^X+1}$. The inverse transformation is $X = \text{logit}\left(\frac{1+Y}{2}\right) = \log\left(\frac{1+Y}{1-Y} \right)$. By the ...
0
votes
1answer
60 views

Bayesian Modeling Understanding Metropolis Sampling

I'm working through a book called Bayesian Analysis in Python. The book focuses heavily on the package PyMC3 but is a little vague on the theory behind it. Say I'm looking at a model like this My ...
5
votes
1answer
191 views

Understanding the Typical Set for Markov chain Monte Carlo sampling

I started reading "A Conceptual Introduction to Hamiltonian Monte Carlo" today, and I've gotten stuck on understanding Betancourt's explanation of what a "typical set" is. If $q_1, q_2, \ldots, q_n$ ...
0
votes
1answer
134 views

Metropolis-Hastings Algorithm for Numerical Integration [duplicate]

I'm attempting to implement a Metropolis-Hastings Algorithm to evaluate integrals of the following form: $$I =\frac{1}{\sqrt\pi}\int_{-\infty}^{\infty} {f(x)\exp(-x^2)} \text{d}x$$ Now we can ...
1
vote
1answer
52 views

How would the size of my dataset influence MCMC output?

I'm runing MCMC using Metropolis-Hasting algorithm to fit an equation with 6 parameters on a dataset composed of 30 instances. How will the fact that my dataset is so small impact the posterio ...