Refers to a special case of Markov Chain Monte Carlo algorithm used to sample from probability distributions.

learn more… | top users | synonyms

0
votes
1answer
36 views

For Metropolis-Hastings algorithm, should target density and proposal distribution have the same distribution?

I watched some youtube videos about the Metropolis-Hastings algorithm. They used a Gaussian as a proposal function to estimate an unknown Gaussian, or used a Gamma function as the proposal function to ...
0
votes
0answers
15 views

MCMC regression model - fixed or stochastic variance

I'm trying to fit a regression model to my data. I have apriori stochastics $a,b,d,\omega$ and a formula: $$\mu_t = a + b*t + d*t*cos(\frac{2 \pi}{\omega$})$$ I want to fit this to my data using $Y_t ...
1
vote
1answer
52 views

Metropolis-Hastings simulation of independent geometric random variables

Consider the following Metropolis-Hastings scheme to sample independent geometric random variables $X = (X_1, \dots, X_N)$, where each $X_j$ has pmf $\mathbb{P}(X_j = x) = p(1-p)^x$ for $x \geq 0$. At ...
3
votes
1answer
58 views

Mathematical foundation of using MCMC in global optimization

MCMC is commonly used to compute the integral in the form of $$\text{Problem A.}~~\int F(x)\pi(x) $$ where $\pi$ is hidden. In the literature, it is explained why MCMC can handle problem A by ...
2
votes
0answers
25 views

Can someone explain Metropolis Hastings algorithm with simplest possible example? [duplicate]

I am just beginning to learn about Metropolis Hastings algorithm and MCMC techniques. I have a basic understanding of Markov chains and stationary distributions and need for the Metropolis Hasting but ...
2
votes
2answers
123 views

“True” answer for MCMC model

Theoretically, given a model with $N$ parameters and $\forall x \in \mathbb{R}.\; p(x)>0$ in the prior of all parameters. If i'm interested only in the end result and not in time-to-convergence, ...
1
vote
1answer
70 views

Gibbs sampling from a complex full conditional

I have a sampling question relating to Gibbs sampling of a complicated full conditional. Supposed I have a complicated full conditional that I want a single sample from $p(\theta_i$|$\theta_{-i}$, ...
1
vote
0answers
26 views

MCMC with dependent variables

I want to run Metropolis-Hastings on a problem which involves two parameters that are not independent. I.e. I want to estimate both of these parameters. At the moment I'm trying to understand if this ...
0
votes
0answers
14 views

Metropolis-Hastings - Best sample size [duplicate]

I need to implement an algorithm to find the best sample size for the given problem: Let $X_1, ... X_n$ iid such that $X_i|\theta \sim Poisson(\theta)$. Let $\theta \sim f(\theta)$ such that ...
2
votes
0answers
31 views

Conditional Density for Sigma (Bayesian Lasso)

I found that in Bayesian Lasso commonly $\beta \sim N(0,\sigma^2*diag(\tau))$ and $\sigma,\tau \sim \pi(\sigma,\tau)$ is used. Whereas $\pi(\cdot)$ is a product of Laplace distributions. Is it ...
1
vote
0answers
14 views

Narrow distribution and Hasting-Metropolis

I would like to sample from a density $\tilde{\pi}=C(\pi)\pi$ whose support is $[0,1]$. The normalization constant $C(\pi)$ of the function $\pi$ is unknown and $\pi$ is very narrow. To see how narrow ...
0
votes
0answers
25 views

Is it possible for Metropolis sampling to converge to the wrong value?

I have simulated data under three parameters of interest, say a, b, c. The prior I put on c was a Gamma, so it only takes positive values. The full conditionals of a and b are known distributions, but ...
0
votes
1answer
35 views

Metropolis sampling (symmetric proposal distribution)

Can Metropolis sampling be used in conjunction with Gibbs sampling? So for example, if I have three parameters of interest, but only two of them have full conditionals that are known distributions, ...
0
votes
0answers
7 views

Gibbs within Collapsed Gibbs?

I have a model with variables $X_{1}, X_{2}, X_{3}, X_{4}$. I would like to sample it within a larger MCMC chain using: $(X_{1}, X_{2}) \sim P(X_{1}, X_{2})$ $(X_{3}, X_{4}) \sim P(X_{3}, X_{4} \mid ...
0
votes
1answer
12 views

Odd Acceptance Ratio

Recently, I read some paper and sometimes they draw a sample $s\sim N(a,b)\times exp(d)$. But they defined the prior as $N(A,b) \times exp(D)$ with unknown $A$ and $D$. Therefore in the acceptance ...
3
votes
1answer
58 views

I Want to see Bi-Modal Posterior in Bayesian Linear Regression!

I'm playing around with a Metropolis-Hastings MCMC algorithm as described in this post. I made an example data set with points taken from two lines shown below. Both lines have a y-intercept of 0 ...
3
votes
1answer
58 views

Bayesian inference for power laws

Let $G$ be a graph with $N$ nodes. Let $p(d_i)$ be the probability of node $i$ to have $d$ connections. If this follows a power-law: $$ p(d_i) = \frac{d_i^\alpha}{\sum_{j=1}^{N} d_j^\alpha} $$ ...
4
votes
1answer
132 views

Do sampling methods (MCMC/SMC) work for combination of continuous and discrete random variables?

Consider a distribution $P = \frac{1}{2}P_1 + \frac{1}{2}P_2$ where $P_1, P_2$ are probability measures on a measurable space $(\mathbb R, \mathcal B)$ such that $P_1: A \to \int_A \mathcal N(x; 0, ...
1
vote
1answer
61 views

What is the acceptance ratio? (Metropolis-Hastings)

I have a really basic question. But I am a little bit confused about that. How do I calculate the acceptance ratio within a Metropolis-Hasting step? I have something like ...
1
vote
2answers
54 views

Likelihood overflow in Metropolis-Hastings acceptance probability

Consider a Bayesian framework where we have priors for some parameters and a likelihood based on the data. Consider the likelihood (and its parametric format) to be very sensitive to the choice of the ...
0
votes
0answers
77 views

Create optimal fantasy sports lineup given individual player performance distributions

So say I have to create a lineup of 9 players and I have a database full of hundreds of players. Each one has a possible point distribution and a salary value: ...
0
votes
0answers
21 views

Sampling the scale parameter of a Laplace distribution

I need to adjust the scale parameter $\lambda$ of a Laplace prior ($p(x|\lambda)=(1/2\lambda)* exp(-|x|/\lambda)$) within metropolis hastings. That means I have a couple of draws for x and now I have ...
0
votes
1answer
23 views

Sampling on a logarithmic scale

I have to draw samples (variance parameter) based on a Gaussian kernel but on a logarithmic scale. I have no clue how to implement that as a part of the Metropolis-Hastings algorithm. In particular, ...
2
votes
1answer
102 views

Gibbs sampling with mixed prior using a Metropolis-Hastings step

My questions are about a sampling procedure for ļ¬tting a Bayesian hierarchical model where one of the priors is a mixture distribution of discrete and continuous parts. The model is not my own but I ...
0
votes
0answers
19 views

Gaussian Kernel on a logarithmic scale

Have to implement an metroplis-hastings algorithm. Here the move of the parameter in every step is given by an 'Gaussian kernel'. More precisely I have to add an small value to the parameter p: ...
1
vote
2answers
64 views

Is that OK to have the same Prior and Proposal Distribution in MH?

Is this ok to choose the same proposal distribution as the prior in Metropolis algorithm? Perhaps it's a simple question and to me, it's totally fine but as I always see people choose different ...
1
vote
0answers
53 views

Efficiency in Metropolis Vs Gibbs sampling

I have read that Gibbs sampling is more efficient than Metropolis algorithm. Why? Is this due only to the fact the in Gibbs sampling the acceptance rate is $1$, so that the chain needs fewer ...
0
votes
2answers
68 views

Metropolis-Hastings fails when the loglikelihood is monotonically increasing with a parameter

I'm trying to estimate the parameters of a Pareto distribution (actually the paretian tail of a generic distribution) via Metropolis-Hastings. The problem is that the loglikelihood, $$ l(\alpha, ...
3
votes
1answer
43 views

Can Metropolis be considered as evolutionary algorithm?

If we compare simple 1+1 evolutionary algorithm (e.g. Droste, Jansen, and Wegener, 2002) 1+1 evolutionary algorithm Set $p_m := 1/n$. Choose randomly an initial bit string $x \in ...
2
votes
0answers
55 views

Can I use Adaptive MCMC in any setting?

In time series econometrics and finance, most Bayesian authors approximate their models with a Gibbs Sampler, this is especial true for state space models, SV and so forth. The dimensionality of ...
3
votes
2answers
218 views

Metropolis Hastings Algorithm - Prior vs Proposal vs Numerator of Bayes Theorem

I've been using this technique in 'black-box' form for a little while as a physics student. I have been struggling to understand what's happening under the hood for some time and I think I almost ...
3
votes
1answer
74 views

Metropolis-Hastings with two dimensional target distribution

I'm confused in the following situation: I want to sample by writing code (Java) from the following distribution that is characterized by the mean vectors and covariance matrices: $$ p\left ( ...
0
votes
0answers
13 views

Loss Function to Calibrate Kernel - Metropolis Hastings

I was wondering if there is anything I can read about calibrating the variance for the proposal distribution in the random walk metropolis hastings. I was thinking of using an optimizer and creating ...
2
votes
1answer
85 views

Metropolis-Hastings acceptance rate confusion

I ran a Bayesian model that have about 2700 parameters. Among these parameters, Adaptive Metropolis algorithm was implemented to estimate ~790 parameters in the I-group and Metropolis algorithm was ...
15
votes
1answer
257 views

Metropolis-Hastings integration - why isn't my strategy working?

Assume I have a function $g(x)$ that I want to integrate $$ \int_{-\infty}^\infty g(x) dx.$$ Of course assuming $g(x)$ goes to zero at the endpoints, no blowups, nice function. One way that I've been ...
4
votes
1answer
70 views

Metropolis : Set first sample value instead of randomly generate an arbitary value

According to Metropolis-Hasting algorithm, the first sample is an arbitrary value generated randomly at the Initialization step. ( http://en.wikipedia.org/wiki/Metropolis%E2%80%93Hastings_algorithm ) ...
2
votes
1answer
138 views

In Bayesian analysis, how to sample from full conditional given uniform prior and normal data likelihood?

[EDIT] This question comes from the example of OpenBUGS manual: Stagnant: a changepoint problem and an illustration of how NOT to do MCMC! I also asked another question regarding this example. ...
2
votes
1answer
55 views

Bayesian Mixture Model Gibbs Sampler for two linear relationships

I am attempting to use a Gibbs Sampler to model a mixture of two groups, where the group membership is defined by a linear relationship conditional on x. Both groups have the same slope and intercept, ...
2
votes
1answer
384 views

Metropolis-Hastings MCMC for Bayesian Regression in R

I am looking for a teaching example of a multivariate (not bivariate) implementation of Metropolis-Hastings for MCMC in R. I know several packages implement the algorithm more generally, but the code ...
2
votes
0answers
66 views

Is this how a Bayesian bootstrap works?

I am a bit new to the whole nonparametric and Bayesian idea, so tell me if this is correct: to estimate, say, the mean of a dataset's population we do the following: We define a function $f(x)$ that ...
3
votes
0answers
42 views

Simplification of my Hasting-Metropolis ratio

I am considering the following model : $$ y_{i,j} = f(x_{i,j},\beta,b_{i},d) + \varepsilon_{i,j} $$ where : $1 \leq i \leq s$ $1 \leq j \leq n_{s}$ $\beta$ is a (vector of) fixed effects $b_{i} ...
2
votes
1answer
82 views

Gibbs sampling for reducible chain

I am new to Gibbs sampling and I ran into a problem with irreducibility. For the Gibbs sampler to work the Markov chain has to be irreducible. But that assumption is not satisfied in my probability ...
1
vote
1answer
114 views

How to incorporate parameter constraints in Metropolis Hastings

I am working on parameter estimation of GARCH model with Metropolis Hastings. But the results I have got doesn't look reasonable, actually it is quite different from what I have got from Gibbs ...
1
vote
1answer
113 views

Independence-Metropolis-Hastings Algorithm

IMHA is an importance-sampling version of MCMC, where the proposal is drawn from a fixed distribution g. Usually, g is chosen to be an approximation to f. The acceptance probability becomes ...
0
votes
0answers
37 views

Explanations about the parallel tempering

I am reading this paper on parallel tempering but there are a few things I do not really understand. If I'm not mistaken, parallel tempering is a MCMC method which is quite convenient to sample from ...
3
votes
2answers
261 views

MCMC chain getting stuck

I am trying to use a Metropolis-within-Gibbs type algorithm to sample $\theta$ and $x$ from the following model. Starting with Bayes theorem I can write: $$ P(\theta, x | y) = \frac{P(y | x, \theta) ...
4
votes
1answer
70 views

Why does Slice sampler use the log of the density?

The Slice sampler1 takes as its argument the log of the density to be sampled from. Why is it doing this? A commenter on this question pointed out that it makes no sense to "sample" from the log of ...
0
votes
1answer
160 views

Metropolis-Hastings using log of the density

Does Metropolis-Hastings work with the log of the proposal and the density to be sampled from? That is, say we want to sample from a density $\pi(x)$, using a proposal $q(x|x^{old})$, will the ...
2
votes
2answers
624 views

Metropolis-Hastings Algorithm within Gibbs Sampling

I have this $f$ function below. $$ f(x_1,x_2)\propto \left(\dfrac{x_1}{x_2}\right)\left(\dfrac{\alpha}{x_2}\right)^{x_1-1}exp\left\{-\left(\dfrac{\alpha}{x_2}\right)^{x_1} \right\}I_{R^+}(x) $$ where ...
1
vote
2answers
319 views

Transformation of variables (Metropolis Hastings)

Say I have a bunch of data from a Poisson distribution and I want to find out my posterior i.e. I'm data fitting: $p(\lambda | X) \sim p(X|\lambda)p(\lambda)$ where $p(X|\lambda) = ...