Markov Chain Monte Carlo (MCMC) refers to a class of methods for generating samples from a target distribution by generating random numbers from a Markov Chain whose stationary distribution is the target distribution. MCMC methods are typically used when more direct methods for random number ...

learn more… | top users | synonyms

0
votes
0answers
17 views

Metropolis-Hastings MCMC for Bayesian Regression in R

I am looking for a teaching example of a multivariate (not bivariate) implementation of Metropolis-Hastings for MCMC in R. I know several packages implement the algorithm more generally, but the code ...
2
votes
0answers
29 views

When should I be worried about the Jeffreys-Lindley paradox in Bayesian model choice?

I am considering a large (but finite) space of models of varying complexity which I explore using RJMCMC. The prior on the parameter vector for each model is fairly informative. In what cases (if ...
4
votes
0answers
25 views

Bayesian Probabilistic Matrix Factorization (BPMF) with PyMC3: PositiveDefiniteError using `NUTS` [migrated]

I've implemented the Bayesian Probabilistic Matrix Factorization algorithm using pymc3 in Python. I also implemented it's precursor, Probabilistic Matrix ...
2
votes
0answers
30 views

Is this how a Bayesian bootstrap works?

I am a bit new to the whole nonparametric and Bayesian idea, so tell me if this is correct: to estimate, say, the mean of a dataset's population we do the following: We define a function $f(x)$ that ...
0
votes
0answers
11 views

How to interpret a multilevel model that repeats variables for fixed and random effects (like in MCMhregress)?

I came across a specification of a multilevel model that I'd never seen before. It looks like the R function MCMChregress (see: ...
0
votes
1answer
19 views

Confused about how to run MCMC using OpenBUGS [on hold]

I am running the code below: ...
2
votes
1answer
71 views
+50

Bayesian meta analysis: implementation in BUGS/JAGS/STAN

I would like to conduct a meta analysis in order to collate the information from a number of studies. The parameter of interest is a probability $\theta$. In each of the studies, the observed data ...
0
votes
1answer
30 views
2
votes
2answers
124 views

What are the options when statisticians who only know R need more computing speed

Speed isn't a concern for many statistics projects but sometimes it is, for example MCMC. Assume that hardware improvement is not an option (looking for relatively free solutions that produce orders ...
3
votes
0answers
21 views

Simplification of my Hasting-Metropolis ratio

I am considering the following model : $$ y_{i,j} = f(x_{i,j},\beta,b_{i},d) + \varepsilon_{i,j} $$ where : $1 \leq i \leq s$ $1 \leq j \leq n_{s}$ $\beta$ is a (vector of) fixed effects $b_{i} ...
0
votes
2answers
36 views

Return value of uniform distributions for MCMC simulations

I am confused about how what value should be returned from a uniform distribution when using MCMC simulations. The proper normal distribution is define as $$ p(\theta) = \left\{ \begin{array}{cc} ...
1
vote
1answer
57 views

Gibbs sampling for reducible chain

I am new to Gibbs sampling and I ran into a problem with irreducibility. For the Gibbs sampler to work the Markov chain has to be irreducible. But that assumption is not satisfied in my probability ...
0
votes
0answers
40 views

How do I sample from a black-box model of a probability distribution?

I have a function 'P(x)' where we query for any 'x' it gives a probability value. This function 'P' does not have a closed form and the evaluation is costly. Now 'x' is a set of vectors(matrix whose ...
1
vote
1answer
30 views

How to incorporate parameter constraints in Metropolis Hastings

I am working on parameter estimation of GARCH model with Metropolis Hastings. But the results I have got doesn't look reasonable, actually it is quite different from what I have got from Gibbs ...
4
votes
1answer
32 views

Gibbs Sampler output: how many Markov chains?

When running a Gibbs sampler (for $n=200$ Iterations) with two full conditionals, I get the output $\mathbf{x} = (x_1^{(n)},x_2^{(n)})_{n =1,...,200}$. So $\mathbf{x}$ is the realizations of a Gibbs ...
0
votes
0answers
20 views

Calculating Marginal Data Density for VAR Model

I am currently estimating Bayesian vector autoregressive (BVAR) models and I would like to do model comparison with Bayes factors. I have read about the Gelfand-Day method, the Geweke (1999) modified ...
1
vote
1answer
58 views

MCMC/EM limitations? MCMC over EM?

I am currently learning hierarchical Bayesian models using JAGS from R, and also pymc using Python ("Bayesian Methods for Hackers"). I can get some intuition from this post: "you will end up with a ...
0
votes
0answers
48 views

GridWalk sampler with pymc3

I am trying to build a GridWalk sampler (actually PolicyWalk as in BIRL by Ramachandran et. al.. Edit More info: The distribution I am interested in is the reward posterior given by, $$ P(R|D) = ...
2
votes
0answers
17 views

Cluster Assignment in Bayesian perspective

I am going to study clustering methods in the Bayesian perspective. I understood how k-means works, and I found it pretty clear, due to the notion of distance and assignments to specific centers. I ...
0
votes
0answers
8 views

Explanations about the parallel tempering

I am reading this paper on parallel tempering but there are a few things I do not really understand. If I'm not mistaken, parallel tempering is a MCMC method which is quite convenient to sample from ...
0
votes
0answers
13 views

Sampling from a set of non-nested models

Consider a collection $\mathcal{M}$ of $m$ different model classes $\mathcal{M} = \{M_1,\dots,M_m\}$, where each model class has a parameter set $\Theta_i$, $i=1,\dots,m$. The model classes are not ...
2
votes
2answers
110 views

MCMC chain getting stuck

I am trying to use a Metropolis-within-Gibbs type algorithm to sample $\theta$ and $x$ from the following model. Starting with Bayes theorem I can write: $$ P(\theta, x | y) = \frac{P(y | x, \theta) ...
1
vote
1answer
43 views

Translate glmer (lme4) model specification into MCMCglmm

I am having some computational trouble estimating the following model with the glmer function in lme4: ...
0
votes
1answer
33 views

High dimensional sampling with low measurement noise

Assume that you have a model $$ Y = G(\Theta) + \varepsilon,$$ where $\Theta$ is a parameter vector with $\sim 8$ dimensions, $G$ is a highly nonlinear function of the parameters, $Y$ is observed ...
2
votes
0answers
29 views

Using MCMC to calculate log likelihood

I have an analytical solution to finding $p(y|\beta)$ and $p(\beta)$. My goal is to find $\log p(y)$. However, the integral $\int p(y|\beta)p(\beta)\,d\beta$ is not analytical. I did manage to use a ...
1
vote
2answers
34 views

Suspiciously high Multivariate PSRF from gelman.diag()

I am using "Multivariate PSRF" statistics from gelman.diag() function to analyze my MCMC chains. Now I analyzed convergence 471 variables (parameters for each ...
2
votes
1answer
56 views

How is ABC more computationally efficient than exact Bayesian Computation for parameter estimation in dynamical systems (ODE) models?

Approximate Bayesian Computation has been suggested as an approach to parameter estimation for computationally intensive simulations, most commonly in population genetics, but also in dynamical ...
0
votes
1answer
31 views

One-Step ahead predictive likelihood for time series forecasting

I am still new to Bayesian forecasting, so I am hoping to get some clarification on a simple concept (by the sounds of it). Suppose that we are interested in forecasting some time series one-step ...
0
votes
0answers
13 views

Monte Carlo for Multivariate Distribution with $x_i > 0$

I am trying to sample from a continuous posterior density $f$, where $f({\bf x}) = 0$ if any $x_i<0$ (can think of the marginals as looking like some sort of gamma distribution). I have coded a ...
0
votes
0answers
35 views

MCMC: The posteriors are too narrow

I have this very simple MCMC there I have ...
0
votes
1answer
59 views

Metropolis-Hastings using log of the density

Does Metropolis-Hastings work with the log of the proposal and the density to be sampled from? That is, say we want to sample from a density $\pi(x)$, using a proposal $q(x|x^{old})$, will the ...
2
votes
0answers
39 views

MCMC convergence: why Heidelberg's test says normal samples are non-stationary?

I am learning about and playing with Heidelberg's convergence test to automatically stop a MCMC sampling. I would have said that if I sample, for instance, from a normal distribution, the test ...
9
votes
2answers
582 views

When is MCMC useful?

I am having trouble in understanding in which situation the MCMC approach is actually useful. I am going through a toy example from the Kruschke book "Doing Bayesian Data Analysis: A Tutorial with R ...
0
votes
0answers
23 views

How many random effects to specify in glmmADMB?

I am not sure if I could use 4 random effects in a glmmADMB model. According to Bolker et al. (TREE 24: 127-131, 2009) when there are more than 3 random effects MCMC should be used. However, I do not ...
1
vote
0answers
16 views

Efficient generation of graph structured correlated random variables via MCMC/Gibbs

Sometime back I had asked this question about generating correlated random draws based on the correlation structure given by a graph. Link Here The solution there requires to create $n\times n$ ...
2
votes
0answers
20 views

Adaptive MCMC for discrete random variables

Let {$Y_1,Y_2, \cdots, Y_n$} be a sequence of random variables where $Y_t$ follows a multinomial distribution with $n=1$ (the number of trials) and $\pi_t = (\pi_{t1}, \pi_{t2}, \cdots, \pi_{tk})'$ be ...
1
vote
1answer
83 views

JAGS, cannot evaluate upper index of counter

I asked this question at the JAGS sourceforge help forum but didn't get response there. I have the following JAGS model: ...
0
votes
0answers
28 views

Base sales in multivariate time series | MCMC model

I have been looking around online for good resources that explain how one would go about calculating base sales when preforming marketing mix modeling. I was told by a colleague that essentially they ...
5
votes
2answers
184 views

Dirichlet Processes for clustering: how to deal with labels?

Q: What is the standard way to cluster data using a Dirichlet Process? When using Gibbs sampling clusters appear and dissapear during the sampling. Besides, we have a identifiability problem since ...
2
votes
0answers
56 views

Likelihood maximization: MCEM algorithm versus MCMC algorithm

Hello Everyone this is my first question. I am a particle physicist and I am doing some empirical studiues on parameters estimation using different methods (this might give me some handle to study on ...
1
vote
0answers
17 views

Symmetric Distribution for MCMC Continuous Distribution

I have a sampling distribution $g(X^{'}| X=x)$ such that $$ \log(X^{'})|X=x\sim N(\log(x), \sigma^2)$$ This ensures that our samples are in $(0, \infty)$. Now I would like to use the Metropolis ...
1
vote
1answer
28 views

Interpretation of absurdly large (but probably correct) Bayes Factors?

I estimated a Bayes factor to compare a hypothetical model against a null-model (which obviously by visual comparison of the posterior predictive with the data) fails to capture a certain aspect in ...
2
votes
0answers
66 views

how to determine if two dice are fair using pymc and roll data

My scenario is that I have two six-sided dice (D1 and D2), either of which may be fair or loaded (biased). I have samples of combined roll data (i.e. D1 + D2). I would like to view the posterior ...
-1
votes
1answer
48 views

Book recommendation on statistical computing? [closed]

Topics on importance sampling, Monte Carlo, MCMC etc.
0
votes
0answers
17 views

Bayesian inference of marginal likelihood using ABC

I have the following situation: suppose data $D = \{x_i\}$ iid are generated through some process with density function $f(x_i | \alpha, \beta)$ (which I think will be negative binomial) and we'd like ...
2
votes
0answers
22 views

How to evaluate a Bayesian forecast?

Suppose that I have a predictive posterior, which is an attempt to predict some one-step ahead forecasted value $\hat{y}_{T+1}$. How do I assess if my posterior has done a good job or not? If we had ...
2
votes
1answer
30 views

marginal conditional distribution from MCMC output [duplicate]

I have a MCMC sampler that targets $$\mathbb{P}(U_1,U_2,...U_n \mid G(U) \leq 0)$$ where $U=(U_1,U_2,...U_n)^T$. I realize now I am more interested in estimating the conditional density $$p_k = p(u_k ...
2
votes
2answers
443 views

Metropolis-Hastings Algorithm within Gibbs Sampling

I have this $f$ function below. $$ f(x_1,x_2)\propto \left(\dfrac{x_1}{x_2}\right)\left(\dfrac{\alpha}{x_2}\right)^{x_1-1}exp\left\{-\left(\dfrac{\alpha}{x_2}\right)^{x_1} \right\}I_{R^+}(x) $$ where ...
1
vote
1answer
90 views

Marginal Likelihood in PYMC

I am using the PYMC toolbox in python in order to carry out a model selection problem using MCMC. What I would like to have for each model is the marginal log-likelihood (i.e. model evidence). The ...
3
votes
0answers
46 views

Generating a skewed distribution given the median and left and right “$1\sigma$” limits [duplicate]

Edit: I found a solution to the problem, which is at the bottom of the post. I'm going to leave the post as it is in case someone else encounters a similar problem! I was banging my head to a ...