Refers to a special case of Markov Chain Monte Carlo algorithm used to sample from probability distributions.

learn more… | top users | synonyms

1
vote
1answer
14 views

Use of Metropolis-Hastings in Bayesian Inference

I am now studying the Metropolis-Hastings algorithm and I want to apply it in order to made a Bayesian Inference of a function $y=f(x)$ to a dataset $D=\{x_i,y_i\}$. Five parameters of the function ...
0
votes
1answer
23 views

Multi parameter Metropolis-Hastings

I need to formulate a multi parameter Metropolis-Hastings algorithm. My question is related to how to define the condition to accept or not the candidate value. In my problem (it is a curve fitting)...
0
votes
1answer
29 views

METROPOLIS-HASTINGS with likelihood

I am trying to set up a Metropolis-Hastings algorithm in Matlab in order to estimate the parameters ${\theta}$ (it is a vector of 5 elements) to fit a curve to a set of data $D={X_i,Y_i,\delta_i}$. $X$...
1
vote
1answer
44 views

Metropolis-Hastings Convergence in R

I'm trying to implement the algorithm Metropolis Hastings for the next FDP. \begin{equation} f_{X}(x)=(5+\exp(5/2))*\sqrt{2\pi}\left[\exp\left\{-\frac{(x-2)^{2}-2x}{2}\right\}+5\exp\left\{-\frac{x+2}{...
2
votes
2answers
40 views

What is the difference and relationship between posterior distribution function and likelihood function in MCMC?

I am learning MCMC in class, and I encounter one question about the relationship between posterior probability and likelihood function. In our lecture, the professor asked us to take samples from ...
4
votes
2answers
34 views

Metropolis-Hasting: compute acceptance based on asymmetric continous independent chain proposal

The title is a mouthful, but here is what it amounts to: Under a proposal distribution using an independent chain, the probability of jumping to point $x$ is independent of the current position $y$ ...
3
votes
1answer
103 views

Proposal distribution - Metropolis Hastings MCMC

In Metropolis-Hastings Markov chain Monte Carlo, the proposal distribution can be anything including the Gaussian (according to the Wikipedia). Q: What's the motivation for using anything other than ...
3
votes
1answer
22 views

Symmetric PDFs in Metropolis-Hastings

My textbook says that a symmetric PDF satisfies $$f(x|y)=f(y|x).$$ Can anyone explain this? Is it equivalent to $f(x+a)=f(x-a)$?
2
votes
1answer
61 views

MCMC Metropolis-Hastings initial values [closed]

my posterior values that I obtained via Metroplis-Hasting are always around my initial values. For instance if I chose $\theta_0 =(1,2)$ my posterior values, after either taking mean or median, are ...
1
vote
1answer
50 views

dmvnorm produce 0 likelihood

I am implementing an MCMC algorithm in R using the "mvtnorm" package. The data is about 150 dimensions so the likelihood produced by dmvnorm is usually zero (or -inf if "log=TRUE" is set), which make ...
2
votes
1answer
41 views

Multiple-Try Metropolis question

I read Multiple-Try Metropolis from Wikipedia and I do not understand some points. Suppose the current state is $\mathbf{x}$. The MTM algorithm is as follows: Draw ''k'' independent ...
1
vote
1answer
93 views

Are there analytically derivable posteriors that save from doing MCMC other than conjugate priors? [duplicate]

Posteriors for conjugate priors can be analytically derived and save us from doing MCMC. Conjugate priors simply have a posterior in the same family as the prior distribution. Are there other ...
2
votes
0answers
54 views

Singular proposal in MCMC

Suppose we want to obtain samples of the density $f(\mathbf{x})$ where $\mathbf{x}$ is a $d$-dimensional vector, i.e. $\mathbf{x} = (x_1, x_2, \dots, x_d)$. To that end, we choose the Metropolis-...
3
votes
1answer
102 views

Fitting power function to data

I am trying to implement an MH algorithm to fit a power function to my data. The power function has the following form: $\hat{y} = a * x^b$ The data are assumed to be normally distributed ...
3
votes
3answers
118 views

A question about the choice and interpretation of the jumping distribution in Metropolis-Hastings algorithm

In order to implement the MH algorithm you need a proposal density or jumping distribution $q(⋅|⋅)$, from which it is easy to sample. If you want to sample from a distribution $f(⋅)$, the MH algorithm ...
0
votes
2answers
94 views

For Metropolis-Hastings algorithm, should target density and proposal distribution have the same distribution?

I watched some youtube videos about the Metropolis-Hastings algorithm. They used a Gaussian as a proposal function to estimate an unknown Gaussian, or used a Gamma function as the proposal function to ...
0
votes
0answers
22 views

MCMC regression model - fixed or stochastic variance

I'm trying to fit a regression model to my data. I have apriori stochastics $a,b,d,\omega$ and a formula: $$\mu_t = a + b*t + d*t*cos(\frac{2 \pi}{\omega$})$$ I want to fit this to my data using $Y_t \...
1
vote
1answer
58 views

Metropolis-Hastings simulation of independent geometric random variables

Consider the following Metropolis-Hastings scheme to sample independent geometric random variables $X = (X_1, \dots, X_N)$, where each $X_j$ has pmf $\mathbb{P}(X_j = x) = p(1-p)^x$ for $x \geq 0$. At ...
3
votes
1answer
86 views

Mathematical foundation of using MCMC in global optimization

MCMC is commonly used to compute the integral in the form of $$\text{Problem A.}~~\int F(x)\pi(x) $$ where $\pi$ is hidden. In the literature, it is explained why MCMC can handle problem A by ...
2
votes
0answers
28 views

Can someone explain Metropolis Hastings algorithm with simplest possible example? [duplicate]

I am just beginning to learn about Metropolis Hastings algorithm and MCMC techniques. I have a basic understanding of Markov chains and stationary distributions and need for the Metropolis Hasting but ...
2
votes
2answers
133 views

“True” answer for MCMC model

Theoretically, given a model with $N$ parameters and $\forall x \in \mathbb{R}.\; p(x)>0$ in the prior of all parameters. If i'm interested only in the end result and not in time-to-convergence, ...
2
votes
1answer
156 views

Gibbs sampling from a complex full conditional

I have a sampling question relating to Gibbs sampling of a complicated full conditional. Supposed I have a complicated full conditional that I want a single sample from $p(\theta_i$|$\theta_{-i}$, $...
1
vote
0answers
43 views

MCMC with dependent variables

I want to run Metropolis-Hastings on a problem which involves two parameters that are not independent. I.e. I want to estimate both of these parameters. At the moment I'm trying to understand if this ...
0
votes
0answers
15 views

Metropolis-Hastings - Best sample size [duplicate]

I need to implement an algorithm to find the best sample size for the given problem: Let $X_1, ... X_n$ iid such that $X_i|\theta \sim Poisson(\theta)$. Let $\theta \sim f(\theta)$ such that $$f(\...
2
votes
0answers
37 views

Conditional Density for Sigma (Bayesian Lasso)

I found that in Bayesian Lasso commonly $\beta \sim N(0,\sigma^2*diag(\tau))$ and $\sigma,\tau \sim \pi(\sigma,\tau)$ is used. Whereas $\pi(\cdot)$ is a product of Laplace distributions. Is it ...
1
vote
0answers
19 views

Narrow distribution and Hasting-Metropolis

I would like to sample from a density $\tilde{\pi}=C(\pi)\pi$ whose support is $[0,1]$. The normalization constant $C(\pi)$ of the function $\pi$ is unknown and $\pi$ is very narrow. To see how narrow ...
0
votes
0answers
34 views

Is it possible for Metropolis sampling to converge to the wrong value?

I have simulated data under three parameters of interest, say a, b, c. The prior I put on c was a Gamma, so it only takes positive values. The full conditionals of a and b are known distributions, but ...
0
votes
1answer
50 views

Metropolis sampling (symmetric proposal distribution)

Can Metropolis sampling be used in conjunction with Gibbs sampling? So for example, if I have three parameters of interest, but only two of them have full conditionals that are known distributions, ...
1
vote
0answers
18 views

Gibbs within Collapsed Gibbs?

I have a model with variables $X_{1}, X_{2}, X_{3}, X_{4}$. I would like to sample it within a larger MCMC chain using: $(X_{1}, X_{2}) \sim P(X_{1}, X_{2})$ $(X_{3}, X_{4}) \sim P(X_{3}, X_{4} \mid ...
0
votes
1answer
16 views

Odd Acceptance Ratio

Recently, I read some paper and sometimes they draw a sample $s\sim N(a,b)\times exp(d)$. But they defined the prior as $N(A,b) \times exp(D)$ with unknown $A$ and $D$. Therefore in the acceptance ...
3
votes
1answer
62 views

I Want to see Bi-Modal Posterior in Bayesian Linear Regression!

I'm playing around with a Metropolis-Hastings MCMC algorithm as described in this post. I made an example data set with points taken from two lines shown below. Both lines have a y-intercept of 0 ...
3
votes
1answer
114 views

Bayesian inference for power laws

Let $G$ be a graph with $N$ nodes. Let $p(d_i)$ be the probability of node $i$ to have $d$ connections. If this follows a power-law: $$ p(d_i) = \frac{d_i^\alpha}{\sum_{j=1}^{N} d_j^\alpha} $$ $\...
4
votes
1answer
156 views

Do sampling methods (MCMC/SMC) work for combination of continuous and discrete random variables?

Consider a distribution $P = \frac{1}{2}P_1 + \frac{1}{2}P_2$ where $P_1, P_2$ are probability measures on a measurable space $(\mathbb R, \mathcal B)$ such that $P_1: A \to \int_A \mathcal N(x; 0, 1)...
1
vote
1answer
96 views

What is the acceptance ratio? (Metropolis-Hastings)

I have a really basic question. But I am a little bit confused about that. How do I calculate the acceptance ratio within a Metropolis-Hasting step? I have something like $min\left\{1,\frac{p(new)}{p(...
1
vote
2answers
92 views

Likelihood overflow in Metropolis-Hastings acceptance probability

Consider a Bayesian framework where we have priors for some parameters and a likelihood based on the data. Consider the likelihood (and its parametric format) to be very sensitive to the choice of the ...
0
votes
0answers
85 views

Create optimal fantasy sports lineup given individual player performance distributions

So say I have to create a lineup of 9 players and I have a database full of hundreds of players. Each one has a possible point distribution and a salary value: ...
0
votes
0answers
22 views

Sampling the scale parameter of a Laplace distribution

I need to adjust the scale parameter $\lambda$ of a Laplace prior ($p(x|\lambda)=(1/2\lambda)* exp(-|x|/\lambda)$) within metropolis hastings. That means I have a couple of draws for x and now I have ...
0
votes
1answer
24 views

Sampling on a logarithmic scale

I have to draw samples (variance parameter) based on a Gaussian kernel but on a logarithmic scale. I have no clue how to implement that as a part of the Metropolis-Hastings algorithm. In particular, ...
2
votes
1answer
218 views

Gibbs sampling with mixed prior using a Metropolis-Hastings step

My questions are about a sampling procedure for fitting a Bayesian hierarchical model where one of the priors is a mixture distribution of discrete and continuous parts. The model is not my own but I ...
0
votes
0answers
20 views

Gaussian Kernel on a logarithmic scale

Have to implement an metroplis-hastings algorithm. Here the move of the parameter in every step is given by an 'Gaussian kernel'. More precisely I have to add an small value to the parameter p: p^new=...
1
vote
2answers
118 views

Is that OK to have the same Prior and Proposal Distribution in MH?

Is this ok to choose the same proposal distribution as the prior in Metropolis algorithm? Perhaps it's a simple question and to me, it's totally fine but as I always see people choose different ...
1
vote
0answers
96 views

Efficiency in Metropolis Vs Gibbs sampling

I have read that Gibbs sampling is more efficient than Metropolis algorithm. Why? Is this due only to the fact the in Gibbs sampling the acceptance rate is $1$, so that the chain needs fewer ...
0
votes
2answers
96 views

Metropolis-Hastings fails when the loglikelihood is monotonically increasing with a parameter

I'm trying to estimate the parameters of a Pareto distribution (actually the paretian tail of a generic distribution) via Metropolis-Hastings. The problem is that the loglikelihood, $$ l(\alpha, ...
3
votes
1answer
50 views

Can Metropolis be considered as evolutionary algorithm?

If we compare simple 1+1 evolutionary algorithm (e.g. Droste, Jansen, and Wegener, 2002) 1+1 evolutionary algorithm Set $p_m := 1/n$. Choose randomly an initial bit string $x \in \{0,1\}^...
2
votes
0answers
62 views

Can I use Adaptive MCMC in any setting?

In time series econometrics and finance, most Bayesian authors approximate their models with a Gibbs Sampler, this is especial true for state space models, SV and so forth. The dimensionality of ...
4
votes
2answers
634 views

Metropolis Hastings Algorithm - Prior vs Proposal vs Numerator of Bayes Theorem

I've been using this technique in 'black-box' form for a little while as a physics student. I have been struggling to understand what's happening under the hood for some time and I think I almost ...
3
votes
1answer
130 views

Metropolis-Hastings with two dimensional target distribution

I'm confused in the following situation: I want to sample by writing code (Java) from the following distribution that is characterized by the mean vectors and covariance matrices: $$ p\left ( \...
0
votes
0answers
15 views

Loss Function to Calibrate Kernel - Metropolis Hastings

I was wondering if there is anything I can read about calibrating the variance for the proposal distribution in the random walk metropolis hastings. I was thinking of using an optimizer and creating ...
2
votes
1answer
107 views

Metropolis-Hastings acceptance rate confusion

I ran a Bayesian model that have about 2700 parameters. Among these parameters, Adaptive Metropolis algorithm was implemented to estimate ~790 parameters in the I-group and Metropolis algorithm was ...
15
votes
1answer
375 views

Metropolis-Hastings integration - why isn't my strategy working?

Assume I have a function $g(x)$ that I want to integrate $$ \int_{-\infty}^\infty g(x) dx.$$ Of course assuming $g(x)$ goes to zero at the endpoints, no blowups, nice function. One way that I've been ...