Tagged Questions

Markov Chain Monte Carlo refers to a class of methods for generating samples from a target distribution by generating random numbers from a Markov Chain whose stationary distribution is the target distribution. MCMC methods are typically used when less computationally expensive methods for random ...

learn more… | top users | synonyms

0
votes
0answers
7 views

Maximizing likelihood versus MCMC sampling: Comparing Parameters and Deviance

I am working in R. I use lm() for maximizing the likelihood in the first analysis, and STAN to sample from the posterior in a second analysis. ...
0
votes
0answers
13 views

What happens if you replace the sampling procedure in MCMC with maximization when fitting a HMM?

When using Markov chain monte carlo to fit hidden Markov models, after you use the forward algorithm to obtain the posterior distribution, you sample the hidden states for the current observation. ...
1
vote
1answer
26 views

Simple model selection example in PYMC

I am currently experimenting with PYMC and I am trying out a simple example so that I start learning how things work (I am also a Python beginner, but an experienced machine learner). I have set ...
2
votes
1answer
38 views

Sampling with Metropolis-Hastings

In Metropolis-Hastings sampling, if every draw of my proposal distribution (Q) is independent from the previous draw, is the convergence to the stationary distribution still guaranteed? To be more ...
0
votes
1answer
25 views

Systematic change of proposal density of metropolis algorithm - still a markov chain?

The problem is that I have a Gibbs-Sampler where one of the parameters has to be sampled via a Metropolis-Step from the respective FCD. I have problems finding a suitable scale for the Gaussian ...
1
vote
0answers
13 views

When to use skew normal regression via MCMC (mixture models)?

when do i use skew normal or skew t Regression via MCMC? Do I use them when the data are heavily skewed, for example income data? Or do I fit a normal Regression model first and inspect the residuals ...
0
votes
0answers
10 views

Error using “cengaussian” family with MCMCglmm

I'm trying to run a model using a cengaussian family distribution with the function MCMCglmm. The model is: ...
1
vote
0answers
21 views

Monte carlo integrations with metropolis hastings step

Consider the following problem: Suppose we want to compute the following integral $$ f(y_1|y_2) = \int_{\theta} \int_{x_1} \int_{x_2} f(y_1| x_1,x_2,\theta, y_2) f(x_1,x_2|\theta,y_2) f(\theta | ...
3
votes
2answers
118 views

What makes a GLM Hierarchical?

Wikipedia defines a Hierarchical GLM as: Hierarchical linear models (or multilevel regression) organizes the data into a hierarchy of regressions, for example where A is regressed on B, and B ...
4
votes
1answer
61 views

How to interpret autocorrelation plot in MCMC

I am getting familiar with Bayesian statistics by reading the book Doing Bayesian Data Analysis, by John K. Kruschke also known as the "puppy book". In chapter 9, hierarchical models are introduced ...
1
vote
0answers
9 views

Discrete time generator of stochastic process

While looking at one paper about Metropolic Hasting optimal convergence rates, I came accross a discrete time generator of Markov chain. It is defined as follows: $$G V(x)=nE\left [ \left( ...
5
votes
1answer
99 views

Does the Gibbs Sampling algorithm guarantee detailed balance?

I have it on supreme authority1 that Gibbs Sampling is a special case of the Metropolis-Hastings algorithm for Markov Chain Monte Carlo sampling. The MH algorithm always gives a transition probability ...
0
votes
0answers
17 views

Checking the mixing of an MCMC chain by testing its gaussianity

As I was running several Monte-Carlo runs and trying to find the mixing time by doing traceplots. I had the idea of instead running $N$ chains $X^{l}_i$ with $l$ the chain label (in $1,..., n$) and ...
1
vote
0answers
18 views

Verifying propriety of MCMC

I have a posterior I'd like to sample: $p(\theta\mid Y)\propto L(Y\mid \theta) p(\theta)$ where $p(\theta)$ is proper, so the posterior is proper. I can write $L(Y\mid\theta) = \int f(Y, Z\mid ...
1
vote
1answer
29 views

Metropolis-Hastings sample reusal

I have a posterior distribution from which I calculate some statistics using sampling, for example I calculate expectation. So I draw 1000 samples using Metropolis-Hastings and then I calculate their ...
1
vote
1answer
32 views

Using interaction terms in an MCMCglmm

I am using MCMCglmm models in R, with hierarchically nested data. The basic structure of the data is as follows - each dyad is a unique combination of focal/other: ...
0
votes
0answers
17 views

MCMCglmm interpretation of effective sample size

I'm running a selection of MCMCglmm models in R, and have a basic question about the outputs. Based on what I've been reading, one indication that the model is mixing well is a large effective ...
0
votes
1answer
33 views

Rstan model for a simple mixture of normals

This blog post from Rbloggers describes how to code a simple three-part normal mixture model with known mixing coefficients, means and standard deviations. While it describes the procedures in some ...
0
votes
0answers
12 views

validity of MCM proposal distribution? uniform prior?

I am writing a program to compute estimated values. Suppose I I have a prior (discrete) distribution T that I can sample from, but don't know the analytic PMF. That is to say I have a program that ...
0
votes
0answers
26 views

Inference methods for multimodality and label switching

Imagine that there are three professions in the world $a,b,c$ (astronauts, doctors and statisticians) and that the Gross Domestic Product (GDP) of a city can be modeled as a linear regression of its ...
3
votes
1answer
45 views

Bayesian inference MCMC model for interval censored data

I have interval censored data for incubation period and I suppose that the exact time Ti included in each interval of the data [L;U] follows a lognormal distribution (mu,sigma2). I would like to use a ...
2
votes
0answers
24 views

Computing a Metropolis-Hastings target distribution?

In implementations of the Metropolis-Hastings algorithm, how is the target distribution $\pi(\mathbf{x}) = P(\mathbf{x}|\mathbf{e})$ computed or estimated while ...
3
votes
2answers
94 views

Is my Bayesian analysis correct?

This is my first time doing a Bayesian analysis, so I'm not sure whether what I did makes perfect sense. I'm trying to tell if two samples come from the same distribution, more specifically, if they ...
0
votes
1answer
29 views

MCMC-Draws from different MCMC- chains

I have questions about the usage of draws from a MCMC. I estimate a hierarchical bayesian Multinomial Logit model (using bayesm in R). I am interested in the ratio of two coefficients 1 and 2, say ...
0
votes
0answers
21 views

Covariance for a multivariate Bayesian Additive Regression Tree

Chipman, George, and McCullogh (2010) state that: One can also extend the sum-of-trees model to a multivariate framework such as: $$ (29) \qquad\qquad Y_i = h_i\left( x_i \right) + ...
2
votes
1answer
21 views

MCMC sampling with noisy likelihood

A surprising results is that MCMC sampling remains unbiased if instead of computing the likelihood $p(x)$ ones compute an estimate $\hat{p}(x)$ as long as $\hat{p}$ is an unbiased and positive ...
0
votes
0answers
14 views

Univariate fixed effect Vs Multivariate model -Negative Covariance, positive parameter estimate, but why?

I am trying to compare the results of two models. The first model looks at y with x as a fixed effect. The second looks at the covariance between x and y. Both models have repeated measures for x ...
0
votes
0answers
16 views

How adapt MCMC when (constant) weights for oberservations are introduced?

I have the following problem: I have already set up a model and MCMC sampler for a mixed model without weights, i.e., every observation contributes the same amount of information. Now I would like to ...
4
votes
2answers
209 views

MCMC packages in R

Is there an R package for MCMC that can accept my self-defined (log)likelihood function (can be done in MCMCpack) and lets the user define contraints to the ...
2
votes
1answer
45 views

PYMC Confusion: are observed nodes fixed or stochastic?

I've been trying to gain a better understanding of factor potentials in PYMC. In reading this article by Cam Davidson-Pilon on Yhat, I got confused about how observed nodes are understood by PYMC. ...
4
votes
1answer
63 views

Is rstan or my grid approximation incorrect: deciding between conflicting quantile estimates in Bayesian inference

I have a model to achieve Bayesian estimates the population size $N$ and probability of detection $\theta$ in a binomial distribution solely based on the observed number of observed objects $y$: $$ ...
3
votes
2answers
155 views

MCMC of a mixture and the label switching problem

I generated some data according to a mixture of two lognormals: $f(x) = p \cdot \mathcal LN(\mu_1, \sigma) + (1-p) \cdot \mathcal LN (\mu_2, \sigma)$. given $p$ and $\sigma$, this is my code to find ...
0
votes
1answer
107 views

what to chose for prior and proposal function for MCMC of a mixture

I generated some data according to a mixture of two lognormals: $f(x) = p \cdot \mathcal LN(\mu_1, \sigma) + (1-p) \cdot \mathcal LN (\mu_2, \sigma)$ Now I want to use MCMC to find the parameters $p, ...
1
vote
0answers
35 views

Point Estimate of normally distributed threshold parameter with unknown mean and variance

I'm new to Bayesian analysis an applied what I learned in John Kruschke's book to simplified versions of a model I previously fitted with non-Bayesian methods. For those simplified versions, even ...
0
votes
1answer
77 views

Defining the Stochastic and Deterministic variables with pymc3

I am trying to use write my own stochastic and deterministic variables with pymc3, but old ...
0
votes
0answers
30 views

Fitting Model with MCMC?

In fitting a model with several variable I have found extremely useful a method involving the minimization of the Chi Square using a MCMC approach. In particular, I followed this tutorial ...
1
vote
0answers
55 views

Understanding a measure of convergence of MCMC simulations

I am trying to better understand better the Gelman/Rubin measure of convergence of MCMCs. The method starts off by defining two quantities: $B$ and $W$. $B$ is said to be the between chain variance ...
0
votes
2answers
54 views

What does drawing sample using Metropolis-Hastings algorithm mean?

I am confused with the word "draw samples from any probability distribution P(x)", mean I apologize for my ignorance, but, drawing sample as i understand, is for example, tossing a coin and writing ...
3
votes
0answers
30 views

Sampling methods and parallelization

A couple of years ago I learned about recent work in parallelizing slice sampling methods. More recently, I have read great things about NUTS and Hamiltonian Monte Carlo methods (HMC) in general (e.g. ...
0
votes
1answer
45 views

MCMC algorithm to generate samples

I read that MCMC algorithm is used to draw samples from a distribution. The example mentioned in the text book is about a 6x6 matrix which after 1000 iterations will converge to a steady state 1x6 ...
3
votes
1answer
77 views

Is there an R package for MCMC estimation of Generalized Method of Moments?

I'm looking for an R package (or a combination of packages) that would allow me to perform MCMC estimation of a GMM model, with a user-specified moments function. I've looked at the CRAN Bayesian ...
1
vote
1answer
117 views

Gibbs Sampling Detecting Change point in time series

I was reading through this one page paper on using Gibbs sampling for detecting a change point in a time series like data. While I understand the part where the $\lambda$ and $\phi$ are chosen from a ...
1
vote
0answers
65 views

Effective Sample Size for posterior

I am trying to implement unsuccessfully a function in matlab, to compute the effective sample size after a MCMC chain, with a posterior with 3 coefficients. Source: Sims MCMC $ VAR(1) / Y_t=\mu ...
0
votes
0answers
18 views

How to find the pdf of a time-dependent parameter in a linear regression model with drift?

Suppose there exists a simple linear equation, where the dependent variable y(time series; measurements available) depends on x1 and x2. If the parameter multiplier of x2 is time dependent and the ...
2
votes
0answers
36 views

Gibbs sample from AR(1) of exogenous input

I am trying to fit a model where there is a sequence of exogenous "shocks", $X_1, X_2, ..., X_T$, and a AR(1) of these shocks explain $Y_1, Y_2, ..., Y_T$. Specifically, Data (known): $X_1, X_2, ...
3
votes
1answer
93 views

Use only the last sample as the posterior in MCMC

I am new to statistics. After an MCMC sampler warmed up, the posterior is better estimated as the mean of several samples. (e.g. related question: http://stats.stackexchange.com//questions/56077) ...
1
vote
0answers
42 views

HMM-forward backward algorithm

Let $x_t, \, t=1, \dots ,T$ be a time series and suppose that $x_t | \xi_t \sim N(\mu_{\xi_t},\sigma^2_{\xi_t})$, where $\xi_t \in [1, \dots,K]$ is a group indicator (or regime or state), the ...
2
votes
2answers
95 views

Why is it desirable to have low auto-correlation in MCMC?

I keep reading about the need to check for autocorrelation in MCMC. Why is it important that the autocorrelation is low? What does it measure in the context of MCMC?
0
votes
0answers
29 views

What is the best ensemble sampler for highly correlated parameter space?

I have a likelihood that I want to estimate the free parameters for it and I am using MCMC to estimate the parameters. Two of the free parameters are positions and I defined uniform priors and one has ...
0
votes
2answers
84 views

Binary version of Probabilistic Matrix Factorization in pymc?

I'm a newby in statistics, this is my first post, sorry for any possible mistake. There is a good Bayesian Probabilistic Matrix Factorization model introduced in: Bayesian Probabilistic Matrix ...