Markov Chain Monte Carlo (MCMC) refers to a class of methods for generating samples from a target distribution by generating random numbers from a Markov Chain whose stationary distribution is the target distribution. MCMC methods are typically used when more direct methods for random number ...

learn more… | top users | synonyms

0
votes
0answers
5 views

Calculating Marginal Data Density for VAR Model

I am currently estimating Bayesian vector autoregressive (BVAR) models and I would like to do model comparison with Bayes factors. I have read about the Gelfand-Day method, the Geweke (1999) modified ...
1
vote
1answer
34 views

MCMC/EM limitations? MCMC over EM?

I am currently learning hierarchical Bayesian models using JAGS from R, and also pymc using Python ("Bayesian Methods for Hackers"). I can get some intuition from this post: "you will end up with a ...
0
votes
0answers
32 views

GridWalk sampler with pymc3

I am trying to build a GridWalk sampler (actually PolicyWalk as in BIRL by Ramachandran et. al.. Edit More info: The distribution I am interested in is the reward posterior given by, $$ P(R|D) = ...
2
votes
0answers
14 views

Cluster Assignment in Bayesian perspective

I am going to study clustering methods in the Bayesian perspective. I understood how k-means works, and I found it pretty clear, due to the notion of distance and assignments to specific centers. I ...
0
votes
0answers
8 views

Explanations about the parallel tempering

I am reading this paper on parallel tempering but there are a few things I do not really understand. If I'm not mistaken, parallel tempering is a MCMC method which is quite convenient to sample from ...
0
votes
0answers
13 views

Sampling from a set of non-nested models

Consider a collection $\mathcal{M}$ of $m$ different model classes $\mathcal{M} = \{M_1,\dots,M_m\}$, where each model class has a parameter set $\Theta_i$, $i=1,\dots,m$. The model classes are not ...
2
votes
2answers
96 views

MCMC chain getting stuck

I am trying to use a Metropolis-within-Gibbs type algorithm to sample $\theta$ and $x$ from the following model. Starting with Bayes theorem I can write: $$ P(\theta, x | y) = \frac{P(y | x, \theta) ...
1
vote
1answer
33 views

Translate glmer (lme4) model specification into MCMCglmm

I am having some computational trouble estimating the following model with the glmer function in lme4: ...
0
votes
1answer
33 views

High dimensional sampling with low measurement noise

Assume that you have a model $$ Y = G(\Theta) + \varepsilon,$$ where $\Theta$ is a parameter vector with $\sim 8$ dimensions, $G$ is a highly nonlinear function of the parameters, $Y$ is observed ...
2
votes
0answers
25 views

Using MCMC to calculate log likelihood

I have an analytical solution to finding $p(y|\beta)$ and $p(\beta)$. My goal is to find $\log p(y)$. However, the integral $\int p(y|\beta)p(\beta)\,d\beta$ is not analytical. I did manage to use a ...
1
vote
2answers
31 views

Suspiciously high Multivariate PSRF from gelman.diag()

I am using "Multivariate PSRF" statistics from gelman.diag() function to analyze my MCMC chains. Now I analyzed convergence 471 variables (parameters for each ...
2
votes
1answer
55 views

How is ABC more computationally efficient than exact Bayesian Computation for parameter estimation in dynamical systems (ODE) models?

Approximate Bayesian Computation has been suggested as an approach to parameter estimation for computationally intensive simulations, most commonly in population genetics, but also in dynamical ...
0
votes
1answer
25 views

One-Step ahead predictive likelihood for time series forecasting

I am still new to Bayesian forecasting, so I am hoping to get some clarification on a simple concept (by the sounds of it). Suppose that we are interested in forecasting some time series one-step ...
0
votes
0answers
13 views

Monte Carlo for Multivariate Distribution with $x_i > 0$

I am trying to sample from a continuous posterior density $f$, where $f({\bf x}) = 0$ if any $x_i<0$ (can think of the marginals as looking like some sort of gamma distribution). I have coded a ...
0
votes
0answers
32 views

MCMC: The posteriors are too narrow

I have this very simple MCMC there I have ...
0
votes
1answer
55 views

Metropolis-Hastings using log of the density

Does Metropolis-Hastings work with the log of the proposal and the density to be sampled from? That is, say we want to sample from a density $\pi(x)$, using a proposal $q(x|x^{old})$, will the ...
2
votes
0answers
38 views

MCMC convergence: why Heidelberg's test says normal samples are non-stationary?

I am learning about and playing with Heidelberg's convergence test to automatically stop a MCMC sampling. I would have said that if I sample, for instance, from a normal distribution, the test ...
9
votes
2answers
539 views

When is MCMC useful?

I am having trouble in understanding in which situation the MCMC approach is actually useful. I am going through a toy example from the Kruschke book "Doing Bayesian Data Analysis: A Tutorial with R ...
0
votes
0answers
19 views

How many random effects to specify in glmmADMB?

I am not sure if I could use 4 random effects in a glmmADMB model. According to Bolker et al. (TREE 24: 127-131, 2009) when there are more than 3 random effects MCMC should be used. However, I do not ...
1
vote
0answers
15 views

Efficient generation of graph structured correlated random variables via MCMC/Gibbs

Sometime back I had asked this question about generating correlated random draws based on the correlation structure given by a graph. Link Here The solution there requires to create $n\times n$ ...
2
votes
0answers
20 views

Adaptive MCMC for discrete random variables

Let {$Y_1,Y_2, \cdots, Y_n$} be a sequence of random variables where $Y_t$ follows a multinomial distribution with $n=1$ (the number of trials) and $\pi_t = (\pi_{t1}, \pi_{t2}, \cdots, \pi_{tk})'$ be ...
1
vote
1answer
64 views

JAGS, cannot evaluate upper index of counter

I asked this question at the JAGS sourceforge help forum but didn't get response there. I have the following JAGS model: ...
0
votes
0answers
24 views

Base sales in multivariate time series | MCMC model

I have been looking around online for good resources that explain how one would go about calculating base sales when preforming marketing mix modeling. I was told by a colleague that essentially they ...
5
votes
2answers
164 views

Dirichlet Processes for clustering: how to deal with labels?

Q: What is the standard way to cluster data using a Dirichlet Process? When using Gibbs sampling clusters appear and dissapear during the sampling. Besides, we have a identifiability problem since ...
2
votes
0answers
53 views

Likelihood maximization: MCEM algorithm versus MCMC algorithm

Hello Everyone this is my first question. I am a particle physicist and I am doing some empirical studiues on parameters estimation using different methods (this might give me some handle to study on ...
1
vote
0answers
16 views

Symmetric Distribution for MCMC Continuous Distribution

I have a sampling distribution $g(X^{'}| X=x)$ such that $$ \log(X^{'})|X=x\sim N(\log(x), \sigma^2)$$ This ensures that our samples are in $(0, \infty)$. Now I would like to use the Metropolis ...
1
vote
1answer
25 views

Interpretation of absurdly large (but probably correct) Bayes Factors?

I estimated a Bayes factor to compare a hypothetical model against a null-model (which obviously by visual comparison of the posterior predictive with the data) fails to capture a certain aspect in ...
2
votes
0answers
60 views

how to determine if two dice are fair using pymc and roll data

My scenario is that I have two six-sided dice (D1 and D2), either of which may be fair or loaded (biased). I have samples of combined roll data (i.e. D1 + D2). I would like to view the posterior ...
-1
votes
1answer
47 views

Book recommendation on statistical computing? [closed]

Topics on importance sampling, Monte Carlo, MCMC etc.
0
votes
0answers
17 views

Bayesian inference of marginal likelihood using ABC

I have the following situation: suppose data $D = \{x_i\}$ iid are generated through some process with density function $f(x_i | \alpha, \beta)$ (which I think will be negative binomial) and we'd like ...
2
votes
0answers
21 views

How to evaluate a Bayesian forecast?

Suppose that I have a predictive posterior, which is an attempt to predict some one-step ahead forecasted value $\hat{y}_{T+1}$. How do I assess if my posterior has done a good job or not? If we had ...
2
votes
1answer
30 views

marginal conditional distribution from MCMC output [duplicate]

I have a MCMC sampler that targets $$\mathbb{P}(U_1,U_2,...U_n \mid G(U) \leq 0)$$ where $U=(U_1,U_2,...U_n)^T$. I realize now I am more interested in estimating the conditional density $$p_k = p(u_k ...
2
votes
2answers
426 views

Metropolis-Hastings Algorithm within Gibbs Sampling

I have this $f$ function below. $$ f(x_1,x_2)\propto \left(\dfrac{x_1}{x_2}\right)\left(\dfrac{\alpha}{x_2}\right)^{x_1-1}exp\left\{-\left(\dfrac{\alpha}{x_2}\right)^{x_1} \right\}I_{R^+}(x) $$ where ...
1
vote
1answer
69 views

Marginal Likelihood in PYMC

I am using the PYMC toolbox in python in order to carry out a model selection problem using MCMC. What I would like to have for each model is the marginal log-likelihood (i.e. model evidence). The ...
2
votes
0answers
43 views

Generating a skewed distribution given the median and left and right “$1\sigma$” limits [duplicate]

Edit: I found a solution to the problem, which is at the bottom of the post. I'm going to leave the post as it is in case someone else encounters a similar problem! I was banging my head to a ...
4
votes
1answer
95 views

What is the simple standard error for MCMC?

Simply put: suppose that we have observed $X=\left\{ X_{1},\ldots,X_{n}\right\}$. We then need to calculate some statistic $T$ using MCMC, using $M$ loops (By "loops" I mean the number of times the ...
1
vote
1answer
37 views

MCMC sampling with sum constraints

I'm interested in sampling a collection of variables with a sum constraint on them. For a simplified example: Prior: $X \sim \mathcal{N}(0, 1)$ $Y \sim \mathcal{N}(0, 1)$ Observation: $X + Y = 1$ ...
1
vote
1answer
71 views

Transformation of variables (Metropolis Hastings)

Say I have a bunch of data from a Poisson distribution and I want to find out my posterior i.e. I'm data fitting: $p(\lambda | X) \sim p(X|\lambda)p(\lambda)$ where $p(X|\lambda) = ...
0
votes
1answer
36 views

Gibbs Sampling for Boltzmann Machines

David Mac Kay, in his book on machine learning talks about Boltzmann machines, and on pg. 3 here http://www.inference.phy.cam.ac.uk/itprnn/ps/521.526.pdf He says "the second equation ...
8
votes
2answers
442 views

Is Markov chain based sampling the “best” for Monte Carlo sampling? Are there alternative schemes available?

Markov Chain Monte Carlo is a method based on Markov chains that allows us to obtain samples (in a Monte Carlo setting) from non-standard distributions from which we cannot draw samples directly. My ...
1
vote
0answers
40 views

“Observed node inconsistent ” when binomial success rate exactly one

I observed some curious behavior in JAGS (rjags). There probably is a good reason for it, but I can't figure it out: As part of a more complex model I have these statements: ...
3
votes
0answers
37 views

Fast Algorithm for Bayesian Measurement Model

I want to estimate a Bayesian Measurement model. That is I am concerned with the rating of each judge $j$ of the value of some trait $z$ for each observation $i$. Not all raters will have rated each ...
5
votes
1answer
63 views

How to derive Gibbs sampling?

I'm actually hesitating to ask this, because I'm afraid I will be referred to other questions or Wikipedia on Gibbs sampling, but I don't have the feeling that they describe what's at hand. Given a ...
0
votes
0answers
19 views

Overestimating variance with MCMC

I'm working with a very specific type of proposal distribution in MCMC algorithm. To validate it I use a simple multivariave Gaussian with $\mu=0$ and $\Sigma$ an identity matrix. The proposal ...
2
votes
0answers
24 views

Comparison of MCMC methods? [closed]

Where can I find a good comparison of Gibbs, Metropolis, and Hybrid MCMC in R or Python? I have thus far found this ...
4
votes
1answer
195 views

computing the posterior of two Gaussian probability distributions

I am a bit confused how to solve a Bayesian statistics problem. I have a parameter $\epsilon^s$ which is defined as following: $$\epsilon^s=\frac{\epsilon-g(\pi,z)}{1-g^*(\pi,z)\epsilon}$$ where ...
1
vote
1answer
45 views

Constructing Bayesian model for randomly picked points from a sine wave

I am trying to apply some data analysis on data which is generated by picking points from a sine wave with some noise added in. I am purposefully ignoring the time dependence, so just collecting data ...
0
votes
0answers
19 views

Gibbs sampler with beta posterior

Drive the Gibbs sampler for the joint posterior for a beta distribution X~Beta (a,b) with parameters alpha and phi, also stating the conditional distribution
0
votes
2answers
119 views

What is going on in an MCMC chain?

I'm currently running some data through the MCMCglmm package in R. I was wandering what actually happens in the chain that is created? I have a multiresponse model (as a quick overview of the data: ...
0
votes
0answers
45 views

how to integrate over product of two inverse-gammas

I'm trying to find a nice closed form for an integral which has the following form: $\int_{0}^{\infty} x^{-\theta_1 -1} \exp \bigl( - \frac{\beta_1}{x} \bigr) (ax +b)^{-\theta_2 -1} \exp \bigl( - ...