Questions tagged [metropolis-hastings]

A special type of Markov Chain Monte Carlo (MCMC) algorithm used to simulate from complex probability distributions. It is validated by Markov chain theory and offers a wide range of possible implementations.

Filter by
Sorted by
Tagged with
0
votes
0answers
21 views

Bayesian multivariate regression with common coefficients

In a hierarchical model I'm working on, I have $K$ different $N\times P$ predictor matrices, each denoted $X_k$ and $K$ length $N$ outcome vectors each denoted $y_k$. Essentially, I have a ...
0
votes
1answer
25 views

Dynamically adjusting parameters of Markov chain

I am using a Metropolis algorithm to generate samples from a complicated (high-dimensional) probability distribution. As is common, the proposed updates depend on some "step size" parameter $...
1
vote
1answer
55 views

How to understand the scaling in Metropolis Hastings MCMC

We know the Metropolis Hastings (MH) in MCMC: target distribution: $\pi(x)$ proposal distribution: $p(y|x)$ acceptance: $\alpha(x,y) = \min \Big(1, \dfrac{\pi(y)p(y|x)}{\pi(x)p(x|y)}\Big)$ Here are ...
0
votes
0answers
19 views

Measure to capture within-chain fluctuations in MCMC?

I am using two kinds of updates for a particular parameter in MCMC estimation of my model. First update gives the following trace plot: Second update gives the following trace plot: Note that the ...
3
votes
0answers
15 views

Metropolis-Hastings Reversibility: What is the measure used in the definition of reversibility?

Let $\pi$ be a target probability distribution on a measurable space $(E, \mathcal{E})$. MCMC obtains dependent samples from $\pi$ by using a Markov Chain with transition kernel $\mathrm{K}:E\times \...
1
vote
1answer
66 views

What is the interpretation of this modified Metropolis algorithm?

Modified Metropolis-Hastings Consider a model with parameters $\theta = (\alpha, \gamma)$ and consider a modified Metropolis-Hastings algorithm which can be summarized (with brevity) as follows. ...
3
votes
1answer
65 views

Does the MCMC algorithm use the likelihood function to move to a new proposal value?

I'm trying to understand how MCMC and related algorithms work for Bayesian inference. In this paper the authors use an example of a normal distribution in explaining MCMC. They state, "If the ...
0
votes
0answers
47 views

Metropolis Algorithm for a high dimensional Bimodal distribution

I am using metropolis mcmc for an $n=8$ dimensional system on an (n-1)-sphere. I was considering the 2d case, as it can be visualized. For this case, the probability density,up to a normalization, is \...
1
vote
1answer
54 views

When is metropolis Hastings algorithm useful

I need to do a data analysis project and am considering the Metropolis Hastings algorithm to estimate the parameters of a logistic regression model. I would draw from the complete data log likelihood ...
1
vote
1answer
40 views

Is it possible to improve Markov Chain Monte Carlo performance by decomposing a Binomial Likelihood?

Suppose that we have sampled $y_{1},y_{2},...,y_{n}$ from a Binomial distribution $Bin(N,p)$. Also, let's assume that $p$ is known and our goal is to infer the unknown parameter $N$, with the use of ...
1
vote
0answers
34 views

MCMC algorithm for Hierarchical Bayes model with variable number of mixture components

I am trying to develop an MCMC algorithm for clustering $n$ data-points $y_{1},y_{2},\dots,y_{n}$ using a Gaussian mixture model, but with a prior defined on the number of components K. The ...
2
votes
0answers
20 views

Drift and Minorization for Metropolis Hastings algorithm

Can some one point me to articles or literature with an example of the drift and minorization condition proof. At the moment i have come across the Gibbs Sampler and the Random Walk Metropolis ...
0
votes
1answer
42 views

Question about MCMC independent proposals

I've a question re MCMC proposals, I was hoping you could help me with that. I need to implement for work an Independent Metropolis-Hastings algorithm to sample from a 10-dimensional posterior. I am ...
1
vote
0answers
30 views

Best Proposal Distribution for asymmetric target in Metropolis Hastings (Exponential Target)

Suppose I know my target distribution is asymmetric, for instance, suppose my target distribution is an exponential $$ p(x) = e^{-x} \qquad\qquad x\in [0, +\infty) $$ Is there any theory regarding ...
3
votes
1answer
82 views

Rao–Blackwellization of Metropolis–Hastings

I am trying to achieve a Rao–Blackwellization of Metropolis–Hastings algorithm. In the paper by Robert et al. 2018, the following is given. \begin{align} ℑ=&\frac{1}{T}\sum_{t=1}^Th(\theta^{(t)})=\...
0
votes
0answers
30 views

High Acceptance rate with Uniform Proposal Distribution

I'm using a metropolis algorithm to model a physical system. There are N sets of 8 real parameters, each set of 8 real numbers is normalized. The target is a Boltzmann distribution with the energy for ...
0
votes
1answer
54 views

Stratification of Random Walk Jumps in Metropolis-Hastings

I aim to achieve variance reduction in Random Walk Metropolis Hastings algorithm by introducing stratification to the random walk jumps. What I have tried is to make use of Latin Hypercube Sampling in ...
1
vote
0answers
15 views

Gibbs sampling step for variables that have a complex offline prior in an MCMC hybrid

I have a question about how to use an offline function as a prior when performing a Gibbs/hybrid analysis. Let's say I have data $y$ and some parameters which I'll simplify to $\theta_1, \theta_2$. ...
0
votes
0answers
48 views

How to sample from a joint probability distribution of two variables? [duplicate]

I want to randomly sample from a joint probability distribution characterized by two variables x and y. So, basically, I have the information of the joint PDF which has the dimension 100x50. As per my ...
4
votes
1answer
57 views

Mean acceptance rate for Metropolis-Hastings algorithm

My question relates to the result stated on page 4 of: http://stat.columbia.edu/~gelman/research/published/baystat5.pdf which claims that the mean acceptance probability when performing the Metropolis-...
-1
votes
1answer
139 views

Metropolis Hastings with Gamma Proposal Density

I am trying to use Metropolis Hastings to sample from a shifted gamma distribution. Since it is shifted, it has a domain of $(n, \infty)$. I tried using a Gaussian proposal density and ran into the ...
1
vote
1answer
56 views

Random Walk Metropolis: acceptance probability with truncated normal proposal

I want to draw from my target density $p(\theta)$ using Random Walk Metropolis. $\theta$ has domain $[2, +\infty)$, and I am using as proposal a truncated normal, namely: $$q(\theta_t') \sim N(\theta_{...
2
votes
1answer
115 views

Monte Carlo Methods: [closed]

Can someone explain to me the following statement from “Introducing Monte Carlo methods with R!” By Robert Christian. “If the exploration mechanism has enough energy to reach as far as the boundaries ...
0
votes
1answer
49 views

Computing the acceptance rate (empirically) of samples from the Metropolis algorithm, where samples are "thinned"

I have a number of queries about computing the acceptance rate of samples generated from the Metropolis (symmetric random walk) algorithm empirically, that is, in the presence of burning-in and ...
0
votes
0answers
35 views

Thinning and burn-in in Metropolis-Hastings algorithm

I have written a Metropolis-Hastings algorithm manually in Julia language for a customized distribution, and i want now to know how can i perform the thinning and the burn-in to increase the ...
0
votes
1answer
31 views

How do we define the kernel to calculate the acceptance ratio for Metropolis-Hastings Markov Chain Monte Carlo?

I am having a lot of difficulty understanding how to apply the algorithm to a real scenario. The part that confuses me is that we are looking for a target distribution (the real distribution of our ...
0
votes
1answer
69 views

Metropolis-Hastings with non-centered Proposal

I am trying to draw samples from the Laplace distribution $\pi^* = \text{exp}(-|\theta|)$, using Metropolis Hastings algorithm with a noncentered proposal, meaning that regular Metropolis wont work.. ...
1
vote
0answers
117 views

what is the optimal step size for metropolis-hastings algorithm to have independent state

In the PRML chapter 11, The Metropolis-Hasting algorithm, For a sampler with Gaussian distribution as proposal distribution. The original distribution is correlated multivariate Gaussian distribution, ...
0
votes
0answers
20 views

How to sample posterior distribution for models with random effects?

I have a time series model contains some fixed parameters ($\beta_{1}$, k, m, etc. ) and also a random effect (i.e. $\beta_{t}$ follows a random walk, with starting value $\beta_{1}$ and variance ...
1
vote
0answers
29 views

Metropolis-Hastings algorithm for a completely specified distribution

Consider a random variable $X\sim f(x)$, such that $$ f(x)=\frac{1}{c}\times K(x)\propto K(x), $$ where c: normalizing constant, K(x): the kernel of the distribution (ie the part which involves $x$). $...
1
vote
1answer
85 views

Application of Metropolis Hastings

I am trying to implement the Metropolis Hastings algorithm for Bayesian analysis. In this case, the parameter of interest is the scale parameter for a Weibull distribution. The context is for ...
0
votes
3answers
97 views

Real-life example in which Markov chain Monte Carlo is desirable? [duplicate]

A typical introduction to the Metropolis--Hastings algorithm, and hence to Markov chain Monte Carlo techniques in general, starts with the following assumptions on some probability distribution $P(x)$ ...
1
vote
0answers
27 views

Metropolis Hastings for BART: Calculation of Tree Prior and Transition Kernel

I am trying to understand the details of BART (Bayesian Additive Regression Trees). In particular, I would like to know how the Metropolis Hastings acceptance probability is calculated for BART. My ...
2
votes
0answers
32 views

Gibbs updating algorithm (Gibbs steps) for computationally expensive likelihood

I am looking for a good way to update steps in a Gibbs sampler where the likelihood function is computationally expensive. Here is what I tried so far: By default JAGS uses a slice sampler. However, ...
1
vote
0answers
29 views

Bias in unbiased pseudo-marginal estimation?

In the Pseudo-marginal Metropolis-Hastings algorithm exact sampling of a posterior distribution is performed when using an unbiased estimate of the marginal likelihood. However, I am having problems ...
0
votes
0answers
10 views

Updating only a subset during Metropolis updates: Is this method ergodic

I am currently implementing a multivariate random walk Metropolis sampler (Metropolis within Gibbs). One problem I have is that computing the likelihood is computationally expensive. Thus, I am ...
0
votes
0answers
59 views

Metropolis Hastings Stuck in Local maxima

I've been running the metropolis hastings algorithm to infer some parameters. After running multiple chains, there are typically two places the chains get stuck in, one of which has a higher ...
0
votes
1answer
158 views

Metropolis Hastings algorithm for joint posterior of probability of heads for 2 coins

I am trying to implement a simple metropolis hastings algorithm to simulate the joint posterior of the probability of flipping heads for 2 coins, $\theta_1,\theta_2$. I am following the problem ...
-1
votes
1answer
40 views

Discrete Proposal distribution MCMC

If you want to perform MCMC (Metropolis-Hastings) to infer discrete values, what are some proposal distributions you can use for this. I can't think of a way to extend the notion of a gaussian ...
1
vote
1answer
53 views

Computing the hastings ratio, g(x|x')/g(x'|x) for asymmetric proposal distributions in MH algorithm?

I understand the Metropolis algorithm. Where I get confused is the MH algorithm where asymmetric proposal distributions may be used. I understand that P(x) and P(x') represent the likelihood/...
2
votes
1answer
313 views

Metropolis -> Metropolis-Hastings for asymmetric proposal distributions?

The below python code implements the Metropolis algorithm and samples from a single variable gaussian distribution. The initial value is sampled uniformly within 5 standard deviations of the mean. ...
0
votes
0answers
108 views

Jacobian term for Metropolis Hastings algorithm?

Suppose an acceptance ratio of the MH sampler without parameter transformation is like this: \begin{gather} r=\frac{f( \theta ',\gamma '|y,m) q( \theta ',\gamma '|\theta ,\gamma )}{f( \theta ,\gamma |...
2
votes
1answer
59 views

Approximation in hierarchical model

Consider a simple Bayesian hierarchical model: $y | \theta \sim P(y | \theta)$ $\theta | \phi \sim P(\theta | \phi)$ $\phi \sim P(\phi)$ I'm interested in drawing from the posterior distribution of $\...
1
vote
0answers
75 views

How should I implement an adaptive Metropolis algorithm for a Gibbs sampler with two Metropolis steps?

Consider the Gibbs sampler Sample $\theta' \sim p(\theta|\tau, D)$ Sample $\tau' \sim p(\tau|\theta', D)$ Both conditional distributions are sampled with a Metropolis step. The joint distribution is ...
1
vote
1answer
290 views

Metropolis-Hastings exercise with Cauchy and normal distributions [self-study]

I have the following exercise to solve and would appreciate some help. Consider a linear regression model $y = X\beta + \varepsilon$, where $y = (y_1,...,y_T)'$, $X = (x_1,...,x_T)$, $x_t$ is a single ...
4
votes
0answers
38 views

Escape unsuccessful accept-reject step in MCMC

I have an MCMC procedure that samples latent variables $h_1, \dots, h_T$. It is based on Shephard and Pitt (1997), https://doi.org/10.1093/biomet/84.3.653. Let $f$ be the true conditional posterior ...
3
votes
1answer
195 views

Metropolis-Hastings: target distribution with two modes; deterministic transformation

I'm trying to construct a Metropolis-Hastings algorithm to sample a target distribution $p(x)$ with two different and isolated modes. The example I'm working with is \begin{equation} p(x) = \frac{\...
5
votes
2answers
158 views

Optimization as sampling for stochastic functions

Given an input space $X$ and a function $f: X\rightarrow \mathbb R$, we want to find $x^*=argmin_{x\in X} f(x)$. One way is to cast this problem as a sampling, where we define a distribution $p(x)\...
2
votes
1answer
71 views

Quickest Way For Me To Learn About Metropolis Hastings

First of all, thanks for reading. I have a month to learn about Metropolis-Hastings with mathematical rigour, and i don't have other responsibilities. I am using second edition of "Monte Carlo ...
0
votes
1answer
39 views

Metropolis-Hastings undersampling near kink in distribution function

I'm trying to use Metropolis-Hastings to sample from a distribution that's very close to $$\exp(-|x|/\ell)$$ and I'm finding that the method is undersampling near the origin, where there's a kink in ...

1
2 3 4 5
9