Questions tagged [metropolis-hastings]

A special type of Markov Chain Monte Carlo (MCMC) algorithm used to simulate from complex probability distributions. It is validated by Markov chain theory and offers a wide range of possible implementations.

Filter by
Sorted by
Tagged with
0
votes
0answers
16 views

Proposal for correlation matrix with LKJ prior

I am writing a Gibbs sampler from scratch. As recommended in various places (http://www3.stat.sinica.edu.tw/statistica/oldpdf/A10n416.pdf, and in another question Covariance matrix proposal ...
2
votes
1answer
39 views

Metropolis Hastings proposal for one parameter restricted to less than the other

Suppose I have parameters $\theta_0$ and $\theta_1$ with prior $$ p(\theta_0,\theta_1)=p(\theta_0|\theta_0<\theta_1)p(\theta_1),$$ that is, $\theta_0$ is less than $\theta_1$. The distributions ...
1
vote
0answers
30 views

Metropolis-Hastings algorithm for a possibly negative probability distribution

Let $I$ be a finite nonempty set $\zeta$ denote the counting measure on $(I,2^I)$ $(E,\mathcal E,\lambda)$ be measure space $p_i:E\to[0,\infty)$ be $\mathcal E$-measurable with $$\int p_i\:{\rm d}\...
0
votes
0answers
22 views

Adjusting for probabilities of different updates in Metropolis Hastings

I have a problem with specifying the update probability in MH. Assume we have the following setup in Metropolis-Hastings. We want to target a (2N-1)-dimensional posterior of parameters $(\alpha_2, \...
0
votes
2answers
32 views

Is Autocorrelation of Posterior Samples always a problem in MCMC

I am experimenting with MCMC methods and have implemented a basic Metropolis-Hastings algorithm. One potential issue with this is that MH posterior samples are autocorrelated. I could verify that ...
1
vote
0answers
33 views

Using the pseudo marginal approach for estimating unknown number of Markov chains in this model

I have $K>0$ iid unknown Markov chains $\{X_n^k : n \in \mathbb{N}\}, k=1, \dots, K$ on a discrete state space $S_X = \{1,2,3\}$, each chain runs and gives rise to observations of the form $\{Y_n^k ...
0
votes
0answers
31 views

Multiple Importance Sampling and Metropolis-Hastings on extended state space

Let $(E,\mathcal E,\lambda),(E',\mathcal E',\lambda')$ be measure spaces $k\in\mathbb N$ $p,q_1,\ldots,q_k:E\to(0,\infty)$ be probability densities on $(E,\mathcal E,\lambda)$ $w_1,\ldots,w_k:E\to[0,...
1
vote
0answers
22 views

Is there a reason why we should run the Metorpolis-Hastings algorithm with a target density approximating the density we're actually after?

Let $(E,\mathcal E,\lambda)$ be a measure space, $p:E\to[0,\infty)$ be $\mathcal E$-measurable with $$c:=\int p\:{\rm d}\lambda$$ and $$\mu:=\underbrace{\frac1cp}_{=:\:\tilde p}\lambda$$ denote the ...
1
vote
1answer
20 views

Metropolis Sampling sample order

I am new to Metropolis sampling, here is a question that confuses me. Assume that there are two sets of variables $a$ and $b$ we want to sample. Let $X$ denote the observations and $p(X|a,b)$ denote ...
-1
votes
1answer
32 views

MCMC with slowly varying Log-Likelihood

I am using MCMC (Metropolis-Hastings) to simulate values of $\theta$: I have a Log-likelihood (using 10 inputs $x_i$) $$L=-\frac{n}{2}\ln(2\pi)-\frac{1}{2}\sum_{i=1}^n(x_i-\theta)^2$$ The variation ...
0
votes
0answers
33 views

Metropolis-within-Gibbs for parametric inference of a regressive model

I have a regressive model of this form \begin{equation} y=f(\theta)+\varepsilon \end{equation} to describe observations $y$, with noise $\varepsilon$ and a parametric function $f$ with parameters $\...
3
votes
0answers
147 views

Pareto optimality in Metropolis sampling

In the Metropolis sampling algorithm, we have some function $f(x)$ proportional to a probability distribution $P(x)$. To generate a random walk with stationary distribution $P(x)$, we generate a ...
1
vote
1answer
27 views

Reconciling Langevin MC methods as one-step HMC versus as diffusion or brownian motion

I have a basic understanding of Hamiltonian monte carlo and why it works. I've read that Langevin MC is basically a special case of HMC when you only step the dynamics forward a single timestep before ...
0
votes
1answer
46 views

In the context of a MCMC: how to create/interpret a trace for a matrix?

In a Metropolis-Hastings algorithm, I'm drawing a matrix from a proposal. The accepted matrices should constitute draws from a posterior. If it were just a parameter, I would know how to interpret ...
2
votes
0answers
51 views

Normal-Gamma: Metropolis-Hastings on log-scale, but no Jacobian?

I am reading the paper by Griffin and Brown (2010) where at one step in their MCMC procedure they need to sample from the following conditional posterior: $$ p(\lambda|\gamma, \Psi)\propto \pi(\...
1
vote
0answers
36 views

MCMC Metropolis-Hastings sampler - Estimation of multiple parameters

First time that I ask a question on this platform! Here I go... I have a dataset with two random variables X1 and X2 and an output Y which comes from a discrete Weibull distribution. I've been trying ...
0
votes
0answers
36 views

MCMC Metropolis-Hasting with binomial distribution

For some time I have struggled with understanding MCMC Metropolis-Hasting application. Especially the terms in the acceptance ratio. I understand this is the correct form \begin{align} \alpha=min\...
2
votes
1answer
63 views

Metropolis-Hastings - interpreting the transition kernel: alpha*proposal

I thought I had great intuition and mathematical understanding of the Metropolis-Hastings algorithm, until closer inspection... as I started compiling my notes, I realized I do not understand the ...
0
votes
0answers
16 views

How to evaluate the draw from the proposal of a Metropolis-Hastings?

In the Metropolis-Hastings step of a MCMC, given a $\theta_n$, I'm drawing $\theta_{n+1} \sim F(\mu(\theta_n), \Sigma)$ where the $\mu $ is a location vector and $\Sigma$ is a scale matrix. When ...
0
votes
1answer
18 views

Direct/indirect sampling of conditionals in Gibbs sampling

I have some problems understanding the definition of Gibbs sampling. Let us take into consideration a bivariate distribution \begin{equation} \pi(x_1,x_2): S \subset \mathcal{R^2} \rightarrow \...
0
votes
1answer
30 views

How to Show that a Distribution is a Stationary Distribution for Metropolis-Hastings? [closed]

For an Ising Model with a (2L+ 1) by (2L+ 1) square grid of magnetic particles, show that $$\pi(\xi)=\frac{1}{Z_\beta}e^{\beta\sum_{x,y=x}{\xi_x\xi_y}}$$ Is indeed a stationary distribution for the ...
1
vote
1answer
90 views

MCMC - target distribution, proposal distribution and likelihood function?

i have just started out in MCMC and I am not sure if I fully understand the concepts of MCMC with respect to the above terms. Let me try to explain that in my own words and include some thoughts / ...
2
votes
0answers
44 views

How to implement a M-H step in a Gibbs sampling

I am having trouble implementing a Metropolis Hastings step in a Gibbs sampling problem. The following code was taken from https://www.stat.colostate.edu/computationalstatistics/ Details: It is a ...
0
votes
0answers
18 views

Which gradient to compute in a hierarchical model for M-H MCMC?

We have the following model: $$y_t=Mx_t+\epsilon_t$$ with $M$ being a matrix such that $M\sim F_{\lambda}$(assume it's a conjugate prior). The $\lambda$ does not appear in $M$, only in its ...
3
votes
1answer
62 views

Estimate the asymptotic efficiency of a Markov chain sampling by the method of batching

In the paper Efficient Metropolis Jumping Rules, the author is writing that he used "the method of batching" for the estimation of $\operatorname{eff}_{\overline\theta_i}$ in Table 1 (on page 605). ...
3
votes
1answer
204 views

How does the Metropolis Algorithm “get off the ground”?

I'm thoroughly confused by the Metropolis Algorithm as defined in Casella and Berger's Statistical Inference. Namely, here's the definition (p.254): Let $Y \sim f_Y(y)$ and $V \sim f_V(v)$, where $...
6
votes
1answer
95 views

How can we verify the intuition that in the RW-Metropolis-Hastings algorithm with Gaussian proposal too small and too large variances are bad choices

Let $d\in\mathbb N$ and consider the Random Walk Metropolis-Hastings algorithm with a Gaussian proposal kernel $Q$ such that $Q(x,\;\cdot\;)=\mathcal N_d(x,\sigma^2_dI_d)$ for all $x\in\mathbb R^d$. ...
0
votes
1answer
66 views

Relation between Uniform distribution, Metropolis Algorithm, and Symmetric Proposal Distribution

I am having some confusion over the Metropolis algorithm. Let $g(x|y)$ be our proposal distribution for the algorithm. For the Metropolis, $g$ must be symmetric (from Wikipedia). In the discrete case, ...
2
votes
0answers
18 views

Using some objective priors (for unbounded space) in a Metropolis-Hastings MCMC

I'm doing some simulations using a M-H MCMC, and I was thinking of using some objective priors for some parameters. These parameters must be in $\mathbb{R}^+$. I was thinking of using $\pi(\theta)\...
5
votes
1answer
189 views

Likelihood modification in Metropolis Hastings ratio for transformed parameter

I want to use MH to get samples from $p(\theta \mid y) \approx p(y \mid \theta) p(\theta)$. Let's assume $\theta$ is heavily constrained and I transform $\theta$ to $f(\theta)$ so I can sample from ...
5
votes
3answers
243 views

Conditional distribution of $\exp(-|x|-|y|-a \cdot |x-y|)$

I am trying to use Gibbs sampling or Metropolis-Hastings to draw samples from the joint distribution$$f(x,y)\propto\exp(-|x|-|y|-a \cdot |x-y|)$$ For this I need the conditional distributions of $x$ ...
2
votes
1answer
41 views

slice sampling correctness

Theoretically, the slice sampling has equilibrium distribution as the target distribution. If we can sample exactly as follows, $y' = U(0, p^*(x))$ $x' = U\{x: p^*(x) > y' \}$ However, in the ...
0
votes
0answers
83 views

Metropolis-Hastings algorithm for autocorrelated data

I have auto-correlated data and I wish to apply the Metropolis-Hastings algorithm on it. The data was obtained by simulating the time evolution of a system, and computing the values of some magnitude ...
1
vote
0answers
28 views

Convergence in total distribution distance in the Random Walk Metropolis-Hastings algorithm

I'm searching for a proof of the convergence in total distribution distance of the transition probabilities of a Markov chain generated by the Random Walk Metropolis-Hastings algorithm to its ...
1
vote
0answers
21 views

Assumptions on the target density in the RWM optimal scaling paper by Roberts, Gelman and Gilks

In the famous paper Weak Convergence and Optimal Scaling of Random Walk Metropolis Algorithms by Roberts, Gelman and Gilks, at the bottom of page 116, the supremum of the third derivative of $\ln f$ ...
0
votes
0answers
20 views

Hamiltonian MCMC information gathering [duplicate]

I started gathering information about Hamiltonian MCMC and I would like to ask if someone knows some good papers or books.If it possible notes that give a detailed explanation of Hamiltonian MCMC. ...
-1
votes
1answer
35 views

2 versions of Metropolis-Hastings : are they equivalent?

I have seen 2 different versions of Metropolis algorithm. First one : Second one : I don't understand the differences between the 2 versions, especially in the second one where I have to use the ...
0
votes
1answer
172 views

Metropolis Hastings - Acceptance ratio, proposal and lkelihood

From a previous post : First to explain the MH algorithm, it's used to approximate numerically a target distribution, in this case $p(\theta|D)$. At each stage of the algorithm: A value ...
4
votes
1answer
107 views

How are deltas chosen for the proposal distribution in multivariate metropolis hastings sampling?

Say I want to use Metropolis Hastings algorithm to get posterior draws of multivariate parameters. In the one variable case, you could manipulate delta until you found something that worked (gave 40% ...
1
vote
1answer
54 views

Metropolis algorithm to Bernoulli likelihood and beta prior (Kruschke 7.3.1)

This question pertains to a specific line written in the book Doing Bayesian Data Analysis by John K. Kruschke. In section 7.3.1, he applies Metropolis algorithm to a case with: $prior = beta(\...
1
vote
0answers
12 views

Some priors for these parameters with a preferential range, and a proposal?

I have the following parameters, with a specified range, but with also a preferential range of values. I'm looking for some priors for them. Later on, I will have to use a Metropolis-Hastings type of ...
1
vote
0answers
73 views

Some pointers on constructing a proposal density in a Metropolis-Hastings Algorithm for Gaussian Processes

I have a likelihood similar to a Normal density. I'm thinking of using matrix-valued covariance function a $\sum^Q_{q=1}A_q \otimes k(X,X)$, where $k$ is a Matérn class covariance function. The other ...
0
votes
2answers
107 views

Rule of thumb for number of iterations Metropolis Hastings

Is there a rule of thumb for the number of iterations needed for the Metropolis Hastings algorithm? I would appreciate good references
3
votes
0answers
149 views

Metropolis sampling for Bayesian networks

Gibbs sampling is a profound and popular technique for creating samples of Bayesian networks (BNs). Metropolis sampling is another popular technique, though - in my opinion - a less accessible method. ...
0
votes
1answer
57 views

Monte Carlo Metropolis: Standard Error and Acceptance

In a time series data generated by Monte Carlo Metropolis algorithm, when is the standard error (correlation between two data points is assumed to be negligible) is higher - when the change in the ...
0
votes
1answer
180 views

Is the truncated normal distribution symmetric?

I am running a Metropolis-Hastings MCMC to find the distribution of a parameter that takes real, positive values. I was considering using the truncated normal distribution, and was wondering if I have ...
0
votes
0answers
106 views

Autocorrelation, Autocovariance and Large lag Standard Error

I have time series generated data from Monte Carl-Metropolis Simulation. I have estimated correlation coefficients using: $r_k = \frac{c_k}{c_0}$ where $c_0$ is the varaiance and $c_k = \frac{1}{N}\...
0
votes
0answers
83 views

Standard Error in Auto correlated Data

I have time-series data generated via Metropolis algorithm - Monte Carlo simulations. Since these data must have some correlation between them, the formula of the standard error for IIDs variable must ...
0
votes
1answer
211 views

Metropolis-Hastings in a Bayesian Hierarchical model

I am trying to estimate a Bayesian Hierarchical model using the random-walk Metropolis-Hastings algorithm. While in a non-Hierarchical model, the algorithm is staight-forward, I am not sure I am ...
1
vote
1answer
94 views

MCMC samples for constructing a histogram

I am interested in generating samples from a density $\pi(\theta)$ to construct a histogram for $\pi(\theta)$ and to use these samples to generate samples of $f(\theta)$ for some function $f$. I may ...