Questions tagged [metropolis-hastings]

A special type of Markov Chain Monte Carlo (MCMC) algorithm used to simulate from complex probability distributions. It is validated by Markov chain theory and offers a wide range of possible implementations.

Filter by
Sorted by
Tagged with
0 votes
0 answers
8 views

Binned resampling of correlated data with bootstrap method

The goal Compute the Binder cumulant defined as the estimator $$\text{B.C.}=\frac{\langle x^4\rangle}{\langle x^2\rangle^2} $$ and its statistical error on a sample of normally distributed data points ...
2 votes
1 answer
90 views

Why do we sample from the uniform distribution in Metropolis-Hastings for acceptance?

For each iteration of the MH, sample $x'=q(x|x')$, then the acceptance probability is computed:$$A=\min(1,a)$$ where $$ \alpha=\frac{p(x')q(x|x')}{p(x)q(x'|x)} $$ Now, I've seen that the algorithm ...
  • 487
2 votes
1 answer
40 views

Slice sampling in Particle Gibbs with Ancestral Sampling

Bear with me as I am not from statistical background. My question is about the implementation of PGAS algorithm as given in Lindsten et. al 2014 concerning sampling in state-space models. The two ...
  • 121
0 votes
0 answers
14 views

Metropolis-Hastings or other MCMC method with an unknown asymmetric proposal distribution?

When working with the Metropolis-Hastings algorithm, we can work with an asymmetric proposal density $g(x^\prime | x)$ provided we know the distribution in order to calculate the ratio $\frac{g(x|x^\...
2 votes
0 answers
39 views

parallelizing log-sum-exp

I have some approximate likelihoods: $L_1, \ldots, L_n$. Each is quite expensive to calculate. They're approximate because they use random numbers. Each of them is being calculated on the same data ...
  • 18.9k
1 vote
1 answer
23 views

Computing the Hastings ratio for multinomial distribution as a proposal distribution in Metropolis-Hastings accept-reject step

I have a question concerning calculating the Hastings ratio in a specific case (multinomial proposal distribution). I consider a discrete vector $M$ with integer values that sum up to some number $N$. ...
1 vote
0 answers
17 views

Parametrizing and Sampling Multivariate Garch Parameters Metropolis-Hastings MCMC

My question is how to sample multivariate GARCH parameters from a proposal distribution (multivariate normal) for a Metropolis-Hastings algorithm. Considering the different dimensions of the parameter ...
0 votes
0 answers
14 views

M-H algorithm for scale parameter ($\sigma>0$)

Suppose I have a continuous distribution with a location parameter $\mu\,[\mu\in \mathbb{R}]$ and scale parameter $\sigma(\sigma>0)$. I obtain full conditional posterior of $\mu$ and $\sigma$ and ...
  • 93
0 votes
1 answer
62 views

Metropolis - Hastings algorithm on a set of countable sequences

I want to simulate $\sigma$ from a measure $\pi(\sigma)$ through the Metropolis-Hastings algorithm, where $\sigma$ is a sequence of 0's and 1's on $S = \{0, 1\}^n$, the set of all sequences of 0's ...
  • 119
0 votes
0 answers
18 views

Checking the results of the Metropolis - Hastings generated sample

I've generated a sample for the distribuition $f(x) \propto cos(x), x \in [-\frac{\pi}{2}, \frac{\pi}{2}]$ through the Metropolis - Hastings algorithm. For that I've used as candidate distribuition $q ...
  • 119
2 votes
1 answer
33 views

Can the proposal distribution for Metropolis-Hastings within Gibbs be conditioned on other variabless?

I am drawing samples from my posterior, $P(x,y|z)$, using Gibbs sampling. When I sample $x$, I use a Metropolis-Hastings step. My question is whether I am allowed to use a proposal distribution for $x'...
1 vote
1 answer
59 views

Binomial Bayesian Regression with Metropolis Hastings in R

I'm trying to implement the Metropolis Hastings algorithm in this problem but I'm having problems with the convergence. $$Y_i|\beta_0,\beta_1 \sim \text{Binomial}(m_i,\theta_i)$$ where $logit(\theta) =...
  • 57
0 votes
0 answers
52 views

Sampling for multimodal posterior using Metropolis-Hastings

I was wondering how the well-known Metropolis-Hastings algorithm changes when sampling from a multi-modal posterior. In particular I was wondering what would change in the acceptance ratio in order to ...
  • 101
0 votes
0 answers
17 views

Simulating a Strauss Point Process with Birth-Death Algorithm

Suppose that I have the following two unconditional Strauss Process $$f_{1}(x_{1}, x_{2},..., x_{n};a,\delta) \propto \prod_{i}^{n}\phi(x_{i}) \prod_{1\leq i \leq j \leq n}a^{\left | x_{i}-x_{j} \...
  • 1,833
2 votes
1 answer
173 views

Distribution of conditional posterior for Gibbs sampling

The following is a description of how the authors (Yongning Wang & Ruey S. Tsay) of this (2019) paper Clustering Multiple Time Series with Structural Breaks want to perform Gibbs sampling to ...
-2 votes
1 answer
39 views

Can you give an example of Metropolis and Metropolis-Hastings algorithm?

I have studied many books and tried to understand both the Metropolis and Metropolis-Hastings algorithm. Everywhere it is written in the context of the Ising model or Lenard-Jones Energy. I am having ...
  • 1,530
1 vote
1 answer
30 views

Do the following normalizing constants cancel out in Reversible Jump ratio?

We know that a Strauss Point Process has density $$p(x_{1}, x_{2},..., x_{K})\propto \prod_{i=1}^{K}\phi(x_{i};\theta)\prod_{1\leq i\leq j \leq K}a^{1(\left | x_{i}-x_{j} \right |\leq \delta)}$$ where ...
  • 1,833
2 votes
0 answers
207 views

Hamiltonian Monte Carlo vs. "Metropolis-Hastings with a Hamiltonian step"

In Hamiltonian Monte Carlo the proposal is accepted with probability: $$ \alpha\left(\mathbf{x}_n(0),\mathbf{x}_n(L\Delta t)\right) = \min\left(1, \frac{\exp\left[-H\left(\mathbf{x}_n(L\Delta t),\...
  • 1,772
0 votes
1 answer
85 views

Calculating Metropolis-Hastings algorithm correction term

I am teaching myself about different samplers and currently going through a course. They write up an example of the Metropolis algorithm (MA), which I am pretty sure I understand. They next propose ...
1 vote
1 answer
55 views

Is the Jacobian term needed if the prior is on the transformation parameter?

Suppose I have a strictly positive parameter $\sigma$ and I need to estimate it using the random walk Metropolis-Hasting algorithm. I know that I can do a parameter transform, i.e., $\beta=log(\sigma)$...
  • 423
0 votes
0 answers
32 views

Interpret and assess the output of MCMC in high dimension?

In one of my recent presentations I used MCMC to generate 50,000 samples from a 13-dimensional random variable. And the audience of this presentation is largely made up of layperson that has no ...
0 votes
1 answer
40 views

How can I find the acceptance probability for a joint Metropolis-Hastings proposal?

Suppose that I want to generate a proposal $(x^*,y^*,z^*)$ with the following: $$z^*\sim p(z|\alpha,\beta)$$ $$x^*\sim p(x|z^*,\boldsymbol{\gamma}_x)$$ $$y^*\sim p(y|z^*,\boldsymbol{\gamma}_y),$$ ...
  • 5
0 votes
0 answers
47 views

Is this Categorical Distribution a valid proposal for Metropolis Hastings?

Is it possible to use a categorical distribution as proposal distribution for Metropolis-Hastings? For example, suppose that at the current iteration we are at the state $x_{i}=1$, from which there ...
  • 1,833
0 votes
1 answer
248 views

Beta-Binomial Gibbs Sampler

I am self-studying Bayesian statistics from the book Computational Bayesian Statistics by Turkman et al, but I am stuck on Problem 6.3 from the book: Suppose we want to consider a Binomial (unknown $\...
's user avatar
0 votes
1 answer
104 views

Metropolis-Hastings algorithm for logarithmic probability density

Similar question to posted here: Metropolis-Hastings using log of the density however my question is around sampling a random number from a uniform distribution. I am following the steps outlined in ...
0 votes
0 answers
38 views

multivariate potential scale reduction factor less than one

I am attempting to implement the multivariate potential scale reduction factor (PSRF) mentioned in this answer and originally described by Brooks and Gelman (1998). When I use a basic Metropolis ...
0 votes
1 answer
38 views

Is the Markov property important in the Metropolis algorithm?

I’m taking a class in Bayesian statistics, and we’re learning about the Metropolis algorithm. Suppose for simplicity that we just have one parameter we’re trying to estimate: $\theta$. According to ...
0 votes
1 answer
71 views

Proposed transition matrix for MCMC in two-state Markov Chain

Suppose we would like to model the weather (either sunny $S$ or cloudy $C$) using a two-state Markov Chain, given a set of data collected from 10000 days: $$CCCSSSSSSCCCSSSSSSCCCC...$$ We can use the ...
  • 101
0 votes
0 answers
16 views

With knowing the exact form of the prior density, how to improve the convergence/acceptance rate of an independent Metropolis Hasting algorithm?

I have a set of parameters $[\eta_1,\eta_2,...,\eta_k]$ (k can be a bit large, sometimes k=40) to estimate via the Bayesian MCMC method. I know that each component of $\eta_{k}$ is independently ...
  • 423
0 votes
0 answers
66 views

Help to Proof of Gibbs sampler acceptance rate

i was wondering if someone could explain how the acceptance rate in the Gibbs Sampling works. In the literature it says that in Gibbs case, the acceptance is always 1 because part of the "...
0 votes
0 answers
124 views

Bayesian analysis example with convergence under Gibbs but not Metropolis-Hastings

Having a conceptual understanding of algorithms such as Metropolis-Hastings, Gibbs and Hamiltonian Monte Carlo can provide ideas of remediation to apply when models do not converge. This question ...
12 votes
2 answers
683 views

In Bayesian models, can you use Uniform(-inf, inf) as a prior?

In Bayesian models, can you use Uniform(-inf, inf) as a prior? I ask because in an class, we looked at MH MCMC sampler, and showed that to sample from a distribution, we need not explicitly solve for ...
  • 1,728
0 votes
0 answers
49 views

Optimal way to calculate the sample variance of MCMC data?

I assume that the calculation of the average and variance of a sample generated by a Metropolis-Hastings MCMC does not depend on the existence of correlation. Recently, I was told that in the presence ...
  • 101
0 votes
0 answers
41 views

Metropolis Hastings on hierarchical bayes update question:

[I have this somewhat complicated hierarchical bayesian model]1 Here the $y$ on $\theta$ are Poisson, $\theta$ are deterministically generated from the $att, def$ (and $home$). Then the last ...
  • 1
0 votes
0 answers
32 views

Bayesian multivariate regression with common coefficients

In a hierarchical model I'm working on, I have $K$ different $N\times P$ predictor matrices, each denoted $X_k$ and $K$ length $N$ outcome vectors each denoted $y_k$. Essentially, I have a ...
0 votes
1 answer
45 views

Dynamically adjusting parameters of Markov chain

I am using a Metropolis algorithm to generate samples from a complicated (high-dimensional) probability distribution. As is common, the proposed updates depend on some "step size" parameter $...
  • 218
2 votes
1 answer
130 views

How to understand the scaling in Metropolis Hastings MCMC

We know the Metropolis Hastings (MH) in MCMC: target distribution: $\pi(x)$ proposal distribution: $p(y|x)$ acceptance: $\alpha(x,y) = \min \Big(1, \dfrac{\pi(y)p(y|x)}{\pi(x)p(x|y)}\Big)$ Here are ...
  • 1,117
0 votes
0 answers
21 views

Measure to capture within-chain fluctuations in MCMC?

I am using two kinds of updates for a particular parameter in MCMC estimation of my model. First update gives the following trace plot: Second update gives the following trace plot: Note that the ...
3 votes
0 answers
73 views

Metropolis-Hastings Reversibility: What is the measure used in the definition of reversibility?

Let $\pi$ be a target probability distribution on a measurable space $(E, \mathcal{E})$. MCMC obtains dependent samples from $\pi$ by using a Markov Chain with transition kernel $\mathrm{K}:E\times \...
  • 1,560
1 vote
1 answer
92 views

What is the interpretation of this modified Metropolis algorithm?

Modified Metropolis-Hastings Consider a model with parameters $\theta = (\alpha, \gamma)$ and consider a modified Metropolis-Hastings algorithm which can be summarized (with brevity) as follows. ...
  • 6,388
3 votes
1 answer
286 views

Does the MCMC algorithm use the likelihood function to move to a new proposal value?

I'm trying to understand how MCMC and related algorithms work for Bayesian inference. In this paper the authors use an example of a normal distribution in explaining MCMC. They state, "If the ...
  • 93
0 votes
0 answers
88 views

Metropolis Algorithm for a high dimensional Bimodal distribution

I am using metropolis mcmc for an $n=8$ dimensional system on an (n-1)-sphere. I was considering the 2d case, as it can be visualized. For this case, the probability density,up to a normalization, is \...
2 votes
1 answer
214 views

When is metropolis Hastings algorithm useful

I need to do a data analysis project and am considering the Metropolis Hastings algorithm to estimate the parameters of a logistic regression model. I would draw from the complete data log likelihood ...
  • 533
1 vote
1 answer
44 views

Is it possible to improve Markov Chain Monte Carlo performance by decomposing a Binomial Likelihood?

Suppose that we have sampled $y_{1},y_{2},...,y_{n}$ from a Binomial distribution $Bin(N,p)$. Also, let's assume that $p$ is known and our goal is to infer the unknown parameter $N$, with the use of ...
  • 1,833
1 vote
0 answers
57 views

MCMC algorithm for Hierarchical Bayes model with variable number of mixture components

I am trying to develop an MCMC algorithm for clustering $n$ data-points $y_{1},y_{2},\dots,y_{n}$ using a Gaussian mixture model, but with a prior defined on the number of components K. The ...
2 votes
0 answers
40 views

Drift and Minorization for Metropolis Hastings algorithm

Can some one point me to articles or literature with an example of the drift and minorization condition proof. At the moment i have come across the Gibbs Sampler and the Random Walk Metropolis ...
0 votes
1 answer
113 views

Question about MCMC independent proposals

I've a question re MCMC proposals, I was hoping you could help me with that. I need to implement for work an Independent Metropolis-Hastings algorithm to sample from a 10-dimensional posterior. I am ...
  • 3
2 votes
0 answers
120 views

Best Proposal Distribution for asymmetric target in Metropolis Hastings (Exponential Target)

Suppose I know my target distribution is asymmetric, for instance, suppose my target distribution is an exponential $$ p(x) = e^{-x} \qquad\qquad x\in [0, +\infty) $$ Is there any theory regarding ...
3 votes
1 answer
125 views

Rao–Blackwellization of Metropolis–Hastings

I am trying to achieve a Rao–Blackwellization of Metropolis–Hastings algorithm. In the paper by Robert et al. 2018, the following is given. \begin{align} ℑ=&\frac{1}{T}\sum_{t=1}^Th(\theta^{(t)})=\...
  • 143
0 votes
0 answers
84 views

High Acceptance rate with Uniform Proposal Distribution

I'm using a metropolis algorithm to model a physical system. There are N sets of 8 real parameters, each set of 8 real numbers is normalized. The target is a Boltzmann distribution with the energy for ...

1
2 3 4 5
9