Refers to a special case of Markov Chain Monte Carlo algorithm used to sample from probability distributions.

learn more… | top users | synonyms

1
vote
0answers
32 views

Metropolis algorithm to solve a problem [migrated]

I need to implement the metropolis algorithm to solve the example titled Cheating among students here. In summary the aim is to estimate the frequency of students cheating in an exam. The experiment ...
2
votes
2answers
39 views

What is the difference and relationship between posterior distribution function and likelihood function in MCMC?

I am learning MCMC in class, and I encounter one question about the relationship between posterior probability and likelihood function. In our lecture, the professor asked us to take samples from ...
4
votes
2answers
34 views

Metropolis-Hasting: compute acceptance based on asymmetric continous independent chain proposal

The title is a mouthful, but here is what it amounts to: Under a proposal distribution using an independent chain, the probability of jumping to point $x$ is independent of the current position $y$ ...
3
votes
1answer
97 views

Proposal distribution - Metropolis Hastings MCMC

In Metropolis-Hastings Markov chain Monte Carlo, the proposal distribution can be anything including the Gaussian (according to the Wikipedia). Q: What's the motivation for using anything other than ...
3
votes
1answer
22 views

Symmetric PDFs in Metropolis-Hastings

My textbook says that a symmetric PDF satisfies $$f(x|y)=f(y|x).$$ Can anyone explain this? Is it equivalent to $f(x+a)=f(x-a)$?
2
votes
1answer
56 views

MCMC Metropolis-Hastings initial values [closed]

my posterior values that I obtained via Metroplis-Hasting are always around my initial values. For instance if I chose $\theta_0 =(1,2)$ my posterior values, after either taking mean or median, are ...
1
vote
1answer
46 views

dmvnorm produce 0 likelihood

I am implementing an MCMC algorithm in R using the "mvtnorm" package. The data is about 150 dimensions so the likelihood produced by dmvnorm is usually zero (or -inf if "log=TRUE" is set), which make ...
2
votes
1answer
40 views

Multiple-Try Metropolis question

I read Multiple-Try Metropolis from Wikipedia and I do not understand some points. Suppose the current state is $\mathbf{x}$. The MTM algorithm is as follows: Draw ''k'' independent ...
1
vote
1answer
90 views

Are there analytically derivable posteriors that save from doing MCMC other than conjugate priors? [duplicate]

Posteriors for conjugate priors can be analytically derived and save us from doing MCMC. Conjugate priors simply have a posterior in the same family as the prior distribution. Are there other ...
2
votes
0answers
54 views

Singular proposal in MCMC

Suppose we want to obtain samples of the density $f(\mathbf{x})$ where $\mathbf{x}$ is a $d$-dimensional vector, i.e. $\mathbf{x} = (x_1, x_2, \dots, x_d)$. To that end, we choose the Metropolis-...
3
votes
1answer
100 views

Fitting power function to data

I am trying to implement an MH algorithm to fit a power function to my data. The power function has the following form: $\hat{y} = a * x^b$ The data are assumed to be normally distributed ...
3
votes
3answers
114 views

A question about the choice and interpretation of the jumping distribution in Metropolis-Hastings algorithm

In order to implement the MH algorithm you need a proposal density or jumping distribution $q(⋅|⋅)$, from which it is easy to sample. If you want to sample from a distribution $f(⋅)$, the MH algorithm ...
0
votes
2answers
86 views

For Metropolis-Hastings algorithm, should target density and proposal distribution have the same distribution?

I watched some youtube videos about the Metropolis-Hastings algorithm. They used a Gaussian as a proposal function to estimate an unknown Gaussian, or used a Gamma function as the proposal function to ...
0
votes
0answers
22 views

MCMC regression model - fixed or stochastic variance

I'm trying to fit a regression model to my data. I have apriori stochastics $a,b,d,\omega$ and a formula: $$\mu_t = a + b*t + d*t*cos(\frac{2 \pi}{\omega$})$$ I want to fit this to my data using $Y_t \...
1
vote
1answer
58 views

Metropolis-Hastings simulation of independent geometric random variables

Consider the following Metropolis-Hastings scheme to sample independent geometric random variables $X = (X_1, \dots, X_N)$, where each $X_j$ has pmf $\mathbb{P}(X_j = x) = p(1-p)^x$ for $x \geq 0$. At ...
3
votes
1answer
76 views

Mathematical foundation of using MCMC in global optimization

MCMC is commonly used to compute the integral in the form of $$\text{Problem A.}~~\int F(x)\pi(x) $$ where $\pi$ is hidden. In the literature, it is explained why MCMC can handle problem A by ...
2
votes
0answers
28 views

Can someone explain Metropolis Hastings algorithm with simplest possible example? [duplicate]

I am just beginning to learn about Metropolis Hastings algorithm and MCMC techniques. I have a basic understanding of Markov chains and stationary distributions and need for the Metropolis Hasting but ...
2
votes
2answers
132 views

“True” answer for MCMC model

Theoretically, given a model with $N$ parameters and $\forall x \in \mathbb{R}.\; p(x)>0$ in the prior of all parameters. If i'm interested only in the end result and not in time-to-convergence, ...
2
votes
1answer
150 views

Gibbs sampling from a complex full conditional

I have a sampling question relating to Gibbs sampling of a complicated full conditional. Supposed I have a complicated full conditional that I want a single sample from $p(\theta_i$|$\theta_{-i}$, $...
1
vote
0answers
40 views

MCMC with dependent variables

I want to run Metropolis-Hastings on a problem which involves two parameters that are not independent. I.e. I want to estimate both of these parameters. At the moment I'm trying to understand if this ...
0
votes
0answers
15 views

Metropolis-Hastings - Best sample size [duplicate]

I need to implement an algorithm to find the best sample size for the given problem: Let $X_1, ... X_n$ iid such that $X_i|\theta \sim Poisson(\theta)$. Let $\theta \sim f(\theta)$ such that $$f(\...
2
votes
0answers
36 views

Conditional Density for Sigma (Bayesian Lasso)

I found that in Bayesian Lasso commonly $\beta \sim N(0,\sigma^2*diag(\tau))$ and $\sigma,\tau \sim \pi(\sigma,\tau)$ is used. Whereas $\pi(\cdot)$ is a product of Laplace distributions. Is it ...
1
vote
0answers
18 views

Narrow distribution and Hasting-Metropolis

I would like to sample from a density $\tilde{\pi}=C(\pi)\pi$ whose support is $[0,1]$. The normalization constant $C(\pi)$ of the function $\pi$ is unknown and $\pi$ is very narrow. To see how narrow ...
0
votes
0answers
33 views

Is it possible for Metropolis sampling to converge to the wrong value?

I have simulated data under three parameters of interest, say a, b, c. The prior I put on c was a Gamma, so it only takes positive values. The full conditionals of a and b are known distributions, but ...
0
votes
1answer
49 views

Metropolis sampling (symmetric proposal distribution)

Can Metropolis sampling be used in conjunction with Gibbs sampling? So for example, if I have three parameters of interest, but only two of them have full conditionals that are known distributions, ...
1
vote
0answers
16 views

Gibbs within Collapsed Gibbs?

I have a model with variables $X_{1}, X_{2}, X_{3}, X_{4}$. I would like to sample it within a larger MCMC chain using: $(X_{1}, X_{2}) \sim P(X_{1}, X_{2})$ $(X_{3}, X_{4}) \sim P(X_{3}, X_{4} \mid ...
0
votes
1answer
15 views

Odd Acceptance Ratio

Recently, I read some paper and sometimes they draw a sample $s\sim N(a,b)\times exp(d)$. But they defined the prior as $N(A,b) \times exp(D)$ with unknown $A$ and $D$. Therefore in the acceptance ...
3
votes
1answer
60 views

I Want to see Bi-Modal Posterior in Bayesian Linear Regression!

I'm playing around with a Metropolis-Hastings MCMC algorithm as described in this post. I made an example data set with points taken from two lines shown below. Both lines have a y-intercept of 0 ...
3
votes
1answer
100 views

Bayesian inference for power laws

Let $G$ be a graph with $N$ nodes. Let $p(d_i)$ be the probability of node $i$ to have $d$ connections. If this follows a power-law: $$ p(d_i) = \frac{d_i^\alpha}{\sum_{j=1}^{N} d_j^\alpha} $$ $\...
4
votes
1answer
151 views

Do sampling methods (MCMC/SMC) work for combination of continuous and discrete random variables?

Consider a distribution $P = \frac{1}{2}P_1 + \frac{1}{2}P_2$ where $P_1, P_2$ are probability measures on a measurable space $(\mathbb R, \mathcal B)$ such that $P_1: A \to \int_A \mathcal N(x; 0, 1)...
1
vote
1answer
91 views

What is the acceptance ratio? (Metropolis-Hastings)

I have a really basic question. But I am a little bit confused about that. How do I calculate the acceptance ratio within a Metropolis-Hasting step? I have something like $min\left\{1,\frac{p(new)}{p(...
1
vote
2answers
87 views

Likelihood overflow in Metropolis-Hastings acceptance probability

Consider a Bayesian framework where we have priors for some parameters and a likelihood based on the data. Consider the likelihood (and its parametric format) to be very sensitive to the choice of the ...
0
votes
0answers
84 views

Create optimal fantasy sports lineup given individual player performance distributions

So say I have to create a lineup of 9 players and I have a database full of hundreds of players. Each one has a possible point distribution and a salary value: ...
0
votes
0answers
22 views

Sampling the scale parameter of a Laplace distribution

I need to adjust the scale parameter $\lambda$ of a Laplace prior ($p(x|\lambda)=(1/2\lambda)* exp(-|x|/\lambda)$) within metropolis hastings. That means I have a couple of draws for x and now I have ...
0
votes
1answer
24 views

Sampling on a logarithmic scale

I have to draw samples (variance parameter) based on a Gaussian kernel but on a logarithmic scale. I have no clue how to implement that as a part of the Metropolis-Hastings algorithm. In particular, ...
2
votes
1answer
208 views

Gibbs sampling with mixed prior using a Metropolis-Hastings step

My questions are about a sampling procedure for fitting a Bayesian hierarchical model where one of the priors is a mixture distribution of discrete and continuous parts. The model is not my own but I ...
0
votes
0answers
20 views

Gaussian Kernel on a logarithmic scale

Have to implement an metroplis-hastings algorithm. Here the move of the parameter in every step is given by an 'Gaussian kernel'. More precisely I have to add an small value to the parameter p: p^new=...
1
vote
2answers
110 views

Is that OK to have the same Prior and Proposal Distribution in MH?

Is this ok to choose the same proposal distribution as the prior in Metropolis algorithm? Perhaps it's a simple question and to me, it's totally fine but as I always see people choose different ...
1
vote
0answers
93 views

Efficiency in Metropolis Vs Gibbs sampling

I have read that Gibbs sampling is more efficient than Metropolis algorithm. Why? Is this due only to the fact the in Gibbs sampling the acceptance rate is $1$, so that the chain needs fewer ...
0
votes
2answers
92 views

Metropolis-Hastings fails when the loglikelihood is monotonically increasing with a parameter

I'm trying to estimate the parameters of a Pareto distribution (actually the paretian tail of a generic distribution) via Metropolis-Hastings. The problem is that the loglikelihood, $$ l(\alpha, ...
3
votes
1answer
48 views

Can Metropolis be considered as evolutionary algorithm?

If we compare simple 1+1 evolutionary algorithm (e.g. Droste, Jansen, and Wegener, 2002) 1+1 evolutionary algorithm Set $p_m := 1/n$. Choose randomly an initial bit string $x \in \{0,1\}^...
2
votes
0answers
60 views

Can I use Adaptive MCMC in any setting?

In time series econometrics and finance, most Bayesian authors approximate their models with a Gibbs Sampler, this is especial true for state space models, SV and so forth. The dimensionality of ...
4
votes
2answers
600 views

Metropolis Hastings Algorithm - Prior vs Proposal vs Numerator of Bayes Theorem

I've been using this technique in 'black-box' form for a little while as a physics student. I have been struggling to understand what's happening under the hood for some time and I think I almost ...
3
votes
1answer
124 views

Metropolis-Hastings with two dimensional target distribution

I'm confused in the following situation: I want to sample by writing code (Java) from the following distribution that is characterized by the mean vectors and covariance matrices: $$ p\left ( \...
0
votes
0answers
15 views

Loss Function to Calibrate Kernel - Metropolis Hastings

I was wondering if there is anything I can read about calibrating the variance for the proposal distribution in the random walk metropolis hastings. I was thinking of using an optimizer and creating ...
2
votes
1answer
103 views

Metropolis-Hastings acceptance rate confusion

I ran a Bayesian model that have about 2700 parameters. Among these parameters, Adaptive Metropolis algorithm was implemented to estimate ~790 parameters in the I-group and Metropolis algorithm was ...
15
votes
1answer
353 views

Metropolis-Hastings integration - why isn't my strategy working?

Assume I have a function $g(x)$ that I want to integrate $$ \int_{-\infty}^\infty g(x) dx.$$ Of course assuming $g(x)$ goes to zero at the endpoints, no blowups, nice function. One way that I've been ...
4
votes
1answer
86 views

Metropolis : Set first sample value instead of randomly generate an arbitary value

According to Metropolis-Hasting algorithm, the first sample is an arbitrary value generated randomly at the Initialization step. ( http://en.wikipedia.org/wiki/Metropolis%E2%80%93Hastings_algorithm ) ...
2
votes
1answer
196 views

In Bayesian analysis, how to sample from full conditional given uniform prior and normal data likelihood?

[EDIT] This question comes from the example of OpenBUGS manual: Stagnant: a changepoint problem and an illustration of how NOT to do MCMC! I also asked another question regarding this example. [...
2
votes
1answer
61 views

Bayesian Mixture Model Gibbs Sampler for two linear relationships

I am attempting to use a Gibbs Sampler to model a mixture of two groups, where the group membership is defined by a linear relationship conditional on x. Both groups have the same slope and intercept, ...