Questions tagged [metropolis-hastings]

A special type of Markov Chain Monte Carlo (MCMC) algorithm used to simulate from complex probability distributions. It is validated by Markov chain theory and offers a wide range of possible implementations.

Filter by
Sorted by
Tagged with
2
votes
0answers
92 views

Metropolis-Hastings to sample from dependent random variables

Imagine the goal is sampling from $p(X,Y)$ and X and Y are dependent real-valued random variables, i.e. $p(X|Y)\neq p(X)$. Now the question is how can we apply Metropolis-Hastings algorithm on the ...
1
vote
1answer
782 views

Bayesian model averaging for variable selection in R

I am trying to use Bayesian model averaging for variable selection with a large number of variables. In R, the BMS package allows to apply the method, with the option of using MCMC sampler (Metropolis ...
1
vote
1answer
294 views

Random walk on simplex as part of Metropolis-Hastings

I would like to perform a random walk on a J-dimensional simplex. However, since this is part of a metropolis-hastings algorithm application, my understanding is that the steps need to be drawn from a ...
5
votes
2answers
15k views

How to sample using MCMC from a posterior distribution in general?

Assume one has the posterior distribution of a parameter, $p(\theta|y)$ and what I mean by having it is that for each point of $\theta$, one can use Monte Carlo method+MCMC to calculate the $p(\theta|...
5
votes
1answer
9k views

Implementing a Metropolis Hastings Algorithm in R

Consider a univariate normal model with mean $µ$ and variance $τ$ . Suppose we use a Beta(2,2) prior for $µ$ (somehow we know µ is between zero and one) and a $log-normal(1,10)$ prior for $τ$ (recall ...
4
votes
1answer
250 views

Perceived circularity in the Metropolis-Hastings algorithm - Where is my error in reasoning?

If I understood it correctly, the Metropolis-Hastings algorithm allows one to sample from a distribution without an analytical representation, which comes in handy, for instance in the Bayesian ...
4
votes
0answers
5k views

Metropolis-Hastings within Gibbs sampling

Suppose we have the following classical normal linear regression model: $$y_i = \beta_1 x_{1i} + \beta_2x_{2i} + \beta_3x_{3i} + e_i$$ where $e_{i} \sim iid.N(0, \sigma^2)$ for all $i = 1, 2, \cdots,...
3
votes
2answers
1k views

Determine precision of average estimated with MCMC

I am using a Markov chain Monte Carlo method (Metropolis-Hastings) to estimate the mean of a distribution. What practical methods can be used to efficiently determine the precision of this estimate, ...
9
votes
2answers
2k views

Confusion related to Gibbs sampling

I came across this article where it says that in Gibbs sampling every sample is accepted. I am a bit confused. How come if every sample it accepted it converges to a stationary distribution. In ...
3
votes
1answer
940 views

Prior selection for Gaussian Processes (GP)

I am trying to select a prior for the covariance parameters of my Gaussian Process (GP) and have been running into numerical problems with my MCMC code. My model is the following: $$Y = D\beta + GP(...
15
votes
2answers
7k views

Confused with MCMC Metropolis-Hastings variations: Random-Walk, Non-Random-Walk, Independent, Metropolis

Over the past few weeks I have been trying to understand MCMC and the Metropolis-Hastings algorithm(s). Every time I think I understand it I realise that I am wrong. Most of the code examples I find ...
17
votes
1answer
14k views

Understanding Metropolis-Hastings with asymmetric proposal distribution

I have been trying to understand the Metropolis-Hastings algorithm in order to write a code for estimating the parameters of a model (i.e. $f(x)=a*x$). According to bibliography the Metropolis-...
1
vote
1answer
2k views

MCMC autocorrelation convergence diagnostic

I use MCMC (Metropolis-Hastings) to sample posterior distributions of three parameters using a nonlinear least-squares objective function to calculate the likelihood of a parameter sets. I want ...
13
votes
1answer
9k views

Understanding MCMC and the Metropolis-Hastings algorithm

Over the past few days I have been trying to understand how Markov Chain Monte Carlo (MCMC) works. In particular I have been trying to understand and implement the Metropolis-Hastings algorithm. So ...
1
vote
0answers
295 views

Gibbs sampling product of normals as conditional

I am deriving a gibbs sampler for a joint distribution, where the conditionals of various parameters are product of two non-standard normal distributions. Usually, I have seen that in Gibbs sampling ...
0
votes
1answer
4k views

Metropolis-Hastings algorithm, using a proposal distribution other than a Gaussian in Matlab

I am currently working on my final year project for my mathematics degree which is based on giving an overview of the Metropolis-Hastings algorithm and some numerical examples. So far I have got some ...
7
votes
2answers
142 views

Metropolis Sampling and invalid states

I have a short question about Monte Carlo integration with Metropolis sampling. I have a continuous state space, but only certain parts of this state space are valid. It is possible that the ...
2
votes
0answers
190 views

Can I adapt a MCMC proposal using a parallel chain?

I am running two MCMC chains (say chain A and chain B) in parallel, using the Metropolis-Hastings algorithm with acceptance probability: $P(accept\ x_t) = \min\{1, f(x_t)/f(x_{t-1})\}$. I would like ...
2
votes
1answer
783 views

Estimated error variance $\sigma^2$ for MCMC estimation in a high-dimensional space

Let $f$ be a function such that: $$f~:~(x,~\theta)\in\mathbb{R}^{3}\times\mathbb{R}^{12} \rightarrow f(x,~\theta)\in\mathbb{R}^3$$ My observations $y$ are noisy values taken by the function $f(\cdot ...
7
votes
1answer
1k views

When is blocked Metropolis sampling more efficient?

Consider the problem of sampling from $p(\mathbf{x}, \mathbf{y})$ using the Metropolis or Metropolis-Hastings (MH) algorithm. I can either propose samples for $p(\mathbf{x}, \mathbf{y})$ directly, or ...
21
votes
1answer
1k views

What are some well known improvements over textbook MCMC algorithms that people use for bayesian inference?

When I'm coding a Monte Carlo simulation for some problem, and the model is simple enough, I use a very basic textbook Gibbs sampling. When it's not possible to use Gibbs sampling, I code the textbook ...
2
votes
0answers
180 views

Estimation of a state-space model using Bayesian analysis with the Metropolis-Hastings algorithm

I have the following state-space model: $$\begin{aligned} y_t&=c+Ax_t+q_t, &q_t \sim \mathcal N(0,Q), \\ x_t&=\mu+Bx_{t-1} + v_t, &v_t \sim \mathcal N(0,R), \end{aligned} $$ where the ...
4
votes
2answers
846 views

Questions about acceptance rule of Metropolis algorithm

I have a question concerning about one step in the Metropolis algorithm. The algorithm proceeds as following, Generate a proposed new sample value from the jumping distribution $Q(x'|x_t)$ Calculate ...
2
votes
2answers
1k views

Proposal for transition matrix for Metropolis-Hastings phylogenetic inference

I am using the Metropolis-Hastings algorithm for phylogenetic inference. To do so I would like to draw the substitution matrix Q from the generalized time-reversible model. To do so I need proposal ...
2
votes
1answer
2k views

Should the proposal distribution in simulated annealing depend on the temperature?

Suppose we are using Simulated Annealing (SA) to minimize a cost function $L:\mathbb{R} \to \mathbb{R}$. Here is my algorithm: (1). Randomly choose a $x_0 \in \mathbb{R}$. Set $x=x_0$ and $T=T_0$. ...
9
votes
2answers
13k views

Acceptance ratio in Metropolis–Hastings algorithm

In the Metropolis–Hastings algorithm for sampling a target distribution, let: $\pi_{i}$ be the target density at state $i$, $\pi_j$ be the target density at the proposed state $j$, $h_{ij}$ be the ...
5
votes
2answers
4k views

How to reduce autocorrelation in Metropolis algorithm?

I've been using a Metropolis/Gibbs sampler combination to generate a joint density for some parameters(it is a hierarchical model, with $y_i\sim Poisson(\lambda_i)$, $\lambda_i\sim Gamma(\alpha,\beta)...
6
votes
1answer
262 views

Variance stabilization “rule” for MCMC jumps…anyone?

I have an implementation of an MCMC algorithm (Metropolis-Hastings and Adaptive Metropolis-Hastings) that I want to modify to suit my needs (it's pyMC, if anyone is interested on the details). My ...
2
votes
1answer
1k views

How can I calculate a probability from a likelihood, e.g. in the Metropolis-Hastings algorithm?

This is a follow-up to my previous question, how can I compute a posterior density estimate from a prior and a likelihood I am having difficulty understanding how it is possible to calculate the ...
2
votes
1answer
521 views

MCMC when the density involves integration over a simplex

I have the following setup. Parameters $W$ with density $\pi(w)$. Observed data $X_1,...,X_n$ iid. Density of $X_i|W=w$ is $f(x_i|w) = \int_{\Delta(x_i)} f(\mathbf c|w) \,d\mathbf c$. The simplex $\...
15
votes
4answers
3k views

Can I change the proposal distribution in random-walk MH MCMC without affecting Markovianity?

Random walk Metropolis-Hasitings with symmetric proposal $q(x|y)= g(|y-x|)$ has the property that the acceptance probability $$P(accept\ y) = \min\{1, f(y)/f(x)\}$$ does not depend on proposal $...
9
votes
2answers
1k views

Sampling from bivariate distribution with known density using MCMC

I tried to simulate from a bivariate density $p(x,y)$ using Metropolis algorithms in R and had no luck. The density can be expressed as $p(y|x)p(x)$, where $p(x)$ is Singh-Maddala distribution $p(x)...
4
votes
3answers
3k views

Minimization of a function by Metropolis-Hastings algorithms

When minimizing a function by general Metropolis-Hastings algorithms, the function is viewed as an unnormalized density of some distribution. (1) As density functions are required to be nonnegative, ...
20
votes
4answers
2k views

Metropolis-Hastings algorithms used in practice

I was reading Christian Robert's Blog today and quite liked the new Metropolis-Hastings algorithm he was discussing. It seemed simple and easy to implement. Whenever I code up MCMC, I tend to stick ...

1
4 5 6 7
8