Markov Chain Monte Carlo (MCMC) refers to a class of methods for generating samples from a target distribution by generating random numbers from a Markov Chain whose stationary distribution is the target distribution. MCMC methods are typically used when more direct methods for random number ...

learn more… | top users | synonyms

2
votes
0answers
13 views

Comparison of MCMC methods? [on hold]

Where can I find a good comparison of Gibbs, Metropolis, and Hybrid MCMC in R or Python? I have thus far found this ...
5
votes
1answer
97 views
+50

computing the posterior of two Gaussian probability distributions

I am a bit confused how to solve a Bayesian statistics problem. I have a parameter $\epsilon^s$ which is defined as following: $$\epsilon^s=\frac{\epsilon-g(\pi,z)}{1-g^*(\pi,z)\epsilon}$$ where ...
1
vote
1answer
31 views

Constructing Bayesian model for randomly picked points from a sine wave

I am trying to apply some data analysis on data which is generated by picking points from a sine wave with some noise added in. I am purposefully ignoring the time dependence, so just collecting data ...
0
votes
0answers
15 views

Gibbs sampler with beta posterior

Drive the Gibbs sampler for the joint posterior for a beta distribution X~Beta (a,b) with parameters alpha and phi, also stating the conditional distribution
0
votes
2answers
85 views

What is going on in an MCMC chain?

I'm currently running some data through the MCMCglmm package in R. I was wandering what actually happens in the chain that is created? I have a multiresponse model (as a quick overview of the data: ...
0
votes
0answers
44 views

how to integrate over product of two inverse-gammas

I'm trying to find a nice closed form for an integral which has the following form: $\int_{0}^{\infty} x^{-\theta_1 -1} \exp \bigl( - \frac{\beta_1}{x} \bigr) (ax +b)^{-\theta_2 -1} \exp \bigl( - ...
0
votes
0answers
21 views

Gibbs Sampling with given Posterior Distribution

I'm trying to implement an algorithm from a paper which assigns three types of labels $m_i, m_d$ and $m_s$. Here $m_i$ labels a collection of documents $G_i , m_d$ a subcollection of them and $m_s$ ...
1
vote
1answer
45 views

Skewed posterior distribution on constrained parameter space for Bayesian inference of MCMC. Advice on what to do?

I am running a fully Bayesian MCMC procedure to estimate some time series models, and my model has a lot of parameter estimates. In particular, one of these parameters, $\phi$, is $\in [-1,1]$. The ...
0
votes
0answers
17 views

PANEL model (MCMC): Equivalent of system GMM in MCMC

I want to fit a PANEL model via MCMC. I am concerned, that I got some covariates that are endogenous. I use MCMC for various reasons, particularly cause I got some spatial dependencies in my model. I ...
0
votes
1answer
47 views

General questions on MCMC

This is a continuation of the following question. The previous link was related to rejection sampling. This is related to MCMC. General questions on rejection sampling 1a. As far as I understand, ...
1
vote
1answer
39 views

General questions on rejection sampling

I am new to Bayesian methods. I was going through a chapter on sampling. I have a few questions related to it. Please help me get these clarified. As far as I understand, rejection sampling will not ...
1
vote
2answers
32 views

Sampling from a portion of the normal distribution?

I have a a conditional distribution $p(X_1 | \theta) \propto MVN(\mu, \Omega) \pi(X_1)$ where $X_1=[x_1, x_2, \dots, x_n]'$ and $\pi(X_1)=1$ when all $x_i \in [0,a)$ and $0$ otherwise. Is there any ...
4
votes
2answers
64 views

Gibbs Sampling and Probability Notation

Problem 1 I am trying to implement Gibbs Sampling for the following problem: There is a grid measuring 3 x 3 sites, each "site" can be designated in a state, $X$, of 1 or -1. The sites are numbered ...
2
votes
0answers
33 views

Refining “good” mixing time estimate

Fix a Markov chain $\{ X_{t} \}_{t \in \mathbb{N}}$ with mixing time $\tau_{\mathrm{mix}}$. Assume that I know some finite bound on the mixing time $\tau_{\mathrm{mix}} < \tau < \infty$, and ...
2
votes
0answers
18 views

Using intermediate timesteps in Hamiltonian Monte Carlo

Hamiltonian Monte Carlo methods are usually built on a symplectic time-reversible ODE integrator such as the Leapfrog method. A HMC iteration uses the following sequence of steps: Sample new momenta ...
4
votes
1answer
43 views

What to do when rejecting a proposed point in MCMC?

I'm writing a simple Metropolis-Hastings MCMC algorithm. Every time a move gets accepted, the point is added to a list of accepted points. I wonder what exactly I should do when a proposed move has ...
0
votes
0answers
16 views

too large DIC improvment a problem?

I have a question regarding large changes in DIC difference across models. For some models, I am getting an improvement of about 70 000 DIC points. My mind is waving a red-flag here. My question is if ...
0
votes
0answers
22 views

Gibbs within Metropolis

Consider a model with two parameters, $\alpha$ and $\beta$. We want to sample these two parameters conditioning on two data points, $d_1$ and $d_2$. Is it possible to use an algorithm like this: 1) ...
1
vote
0answers
27 views

Fitting multiple models to a noisy measurement

I have measured a quantity for a set of data contains couple of thousands objects. Since the measurement is very noisy, I need a set of data contains a lot of objects. Then I have a model based on the ...
2
votes
1answer
82 views

Maximizing likelihood versus MCMC sampling: Comparing Parameters and Deviance

I am working in R. I use lm() for maximizing the likelihood in the first analysis, and STAN to sample from the posterior in a second analysis. ...
0
votes
0answers
14 views

What happens if you replace the sampling procedure in MCMC with maximization when fitting a HMM?

When using Markov chain monte carlo to fit hidden Markov models, after you use the forward algorithm to obtain the posterior distribution, you sample the hidden states for the current observation. ...
1
vote
1answer
82 views

Simple model selection example in PYMC

I am currently experimenting with PYMC and I am trying out a simple example so that I start learning how things work (I am also a Python beginner, but an experienced machine learner). I have set ...
2
votes
1answer
60 views

Sampling with Metropolis-Hastings

In Metropolis-Hastings sampling, if every draw of my proposal distribution (Q) is independent from the previous draw, is the convergence to the stationary distribution still guaranteed? To be more ...
0
votes
1answer
36 views

Systematic change of proposal density of metropolis algorithm - still a markov chain?

The problem is that I have a Gibbs-Sampler where one of the parameters has to be sampled via a Metropolis-Step from the respective FCD. I have problems finding a suitable scale for the Gaussian ...
2
votes
1answer
47 views

When to use skew normal regression via MCMC (mixture models)?

When do I use skew normal or skew t regression via MCMC? Do I use them when the data are heavily skewed, for example income data? Or do I fit a normal regression model first and inspect the residuals ...
0
votes
0answers
22 views

Error using “cengaussian” family with MCMCglmm

I'm trying to run a model using a cengaussian family distribution with the function MCMCglmm. The model is: ...
1
vote
0answers
24 views

Monte carlo integrations with metropolis hastings step

Consider the following problem: Suppose we want to compute the following integral $$ f(y_1|y_2) = \int_{\theta} \int_{x_1} \int_{x_2} f(y_1| x_1,x_2,\theta, y_2) f(x_1,x_2|\theta,y_2) f(\theta | ...
3
votes
2answers
134 views

What makes a GLM Hierarchical?

Wikipedia defines a Hierarchical GLM as: Hierarchical linear models (or multilevel regression) organizes the data into a hierarchy of regressions, for example where A is regressed on B, and B ...
4
votes
1answer
115 views

How to interpret autocorrelation plot in MCMC

I am getting familiar with Bayesian statistics by reading the book Doing Bayesian Data Analysis, by John K. Kruschke also known as the "puppy book". In chapter 9, hierarchical models are introduced ...
1
vote
0answers
13 views

Discrete time generator of stochastic process

While looking at one paper about Metropolic Hasting optimal convergence rates, I came accross a discrete time generator of Markov chain. It is defined as follows: $$G V(x)=nE\left [ \left( ...
6
votes
1answer
121 views

Does the Gibbs Sampling algorithm guarantee detailed balance?

I have it on supreme authority1 that Gibbs Sampling is a special case of the Metropolis-Hastings algorithm for Markov Chain Monte Carlo sampling. The MH algorithm always gives a transition probability ...
0
votes
0answers
19 views

Checking the mixing of an MCMC chain by testing its gaussianity

As I was running several Monte-Carlo runs and trying to find the mixing time by doing traceplots. I had the idea of instead running $N$ chains $X^{l}_i$ with $l$ the chain label (in $1,..., n$) and ...
1
vote
0answers
20 views

Verifying propriety of MCMC

I have a posterior I'd like to sample: $p(\theta\mid Y)\propto L(Y\mid \theta) p(\theta)$ where $p(\theta)$ is proper, so the posterior is proper. I can write $L(Y\mid\theta) = \int f(Y, Z\mid ...
1
vote
1answer
32 views

Metropolis-Hastings sample reusal

I have a posterior distribution from which I calculate some statistics using sampling, for example I calculate expectation. So I draw 1000 samples using Metropolis-Hastings and then I calculate their ...
1
vote
1answer
52 views

Using interaction terms in an MCMCglmm

I am using MCMCglmm models in R, with hierarchically nested data. The basic structure of the data is as follows - each dyad is a unique combination of focal/other: ...
0
votes
0answers
43 views

MCMCglmm interpretation of effective sample size

I'm running a selection of MCMCglmm models in R, and have a basic question about the outputs. Based on what I've been reading, one indication that the model is mixing well is a large effective ...
0
votes
1answer
59 views

Rstan model for a simple mixture of normals

This blog post from Rbloggers describes how to code a simple three-part normal mixture model with known mixing coefficients, means and standard deviations. While it describes the procedures in some ...
0
votes
0answers
27 views

validity of MCM proposal distribution? uniform prior?

I am writing a program to compute estimated values. Suppose I I have a prior (discrete) distribution T that I can sample from, but don't know the analytic PMF. That is to say I have a program that ...
0
votes
0answers
36 views

Inference methods for multimodality and label switching

Imagine that there are three professions in the world $a,b,c$ (astronauts, doctors and statisticians) and that the Gross Domestic Product (GDP) of a city can be modeled as a linear regression of its ...
3
votes
1answer
59 views

Bayesian inference MCMC model for interval censored data

I have interval censored data for incubation period and I suppose that the exact time Ti included in each interval of the data [L;U] follows a lognormal distribution (mu,sigma2). I would like to use a ...
2
votes
0answers
29 views

Computing a Metropolis-Hastings target distribution?

In implementations of the Metropolis-Hastings algorithm, how is the target distribution $\pi(\mathbf{x}) = P(\mathbf{x}|\mathbf{e})$ computed or estimated while ...
3
votes
2answers
99 views

Is my Bayesian analysis correct?

This is my first time doing a Bayesian analysis, so I'm not sure whether what I did makes perfect sense. I'm trying to tell if two samples come from the same distribution, more specifically, if they ...
0
votes
1answer
35 views

MCMC-Draws from different MCMC- chains

I have questions about the usage of draws from a MCMC. I estimate a hierarchical bayesian Multinomial Logit model (using bayesm in R). I am interested in the ratio of two coefficients 1 and 2, say ...
0
votes
0answers
24 views

Covariance for a multivariate Bayesian Additive Regression Tree

Chipman, George, and McCullogh (2010) state that: One can also extend the sum-of-trees model to a multivariate framework such as: $$ (29) \qquad\qquad Y_i = h_i\left( x_i \right) + ...
2
votes
1answer
34 views

MCMC sampling with noisy likelihood

A surprising results is that MCMC sampling remains unbiased if instead of computing the likelihood $p(x)$ ones compute an estimate $\hat{p}(x)$ as long as $\hat{p}$ is an unbiased and positive ...
0
votes
0answers
42 views

Univariate fixed effect Vs Multivariate model -Negative Covariance, positive parameter estimate, but why?

I am trying to compare the results of two models. The first model looks at y with x as a fixed effect. The second looks at the covariance between x and y. Both models have repeated measures for x ...
0
votes
0answers
17 views

How adapt MCMC when (constant) weights for oberservations are introduced?

I have the following problem: I have already set up a model and MCMC sampler for a mixed model without weights, i.e., every observation contributes the same amount of information. Now I would like to ...
4
votes
2answers
231 views

MCMC packages in R

Is there an R package for MCMC that can accept my self-defined (log)likelihood function (can be done in MCMCpack) and lets the user define contraints to the ...
2
votes
1answer
61 views

PYMC Confusion: are observed nodes fixed or stochastic?

I've been trying to gain a better understanding of factor potentials in PYMC. In reading this article by Cam Davidson-Pilon on Yhat, I got confused about how observed nodes are understood by PYMC. ...
4
votes
1answer
75 views

Is rstan or my grid approximation incorrect: deciding between conflicting quantile estimates in Bayesian inference

I have a model to achieve Bayesian estimates the population size $N$ and probability of detection $\theta$ in a binomial distribution solely based on the observed number of observed objects $y$: $$ ...