Markov Chain Monte Carlo (MCMC) refers to a class of methods for generating samples from a target distribution by generating random numbers from a Markov Chain whose stationary distribution is the target distribution. MCMC methods are typically used when more direct methods for random number ...

learn more… | top users | synonyms

2
votes
2answers
62 views

Why is MCMC needed when estimating a parameter using MAP

Given the formula for MAP estimation of a parameter Why is a MCMC (or similar) approach needed, couldn't I just take the derivative, set it to zero and then solve for the parameter?
1
vote
0answers
16 views

Why is the posterior the stationary distribution of a Gibbs chain?

I'm having trouble understanding the setup here. I'm follow Probabilistic Graphical Models by Koller and Friedman. They say that we wish to generate samples from the posterior distribution ...
0
votes
0answers
11 views

Fitting to a sum (mixture?) of components

I have a curve-fitting problem in which I'm fitting my data to a sum of components with identical functional forms but different parameters. That is $d_i = f(x_i | \theta_j) + f(x_i | \phi_j) + ...
0
votes
2answers
49 views

Metropolis-Hastings fails when the loglikelihood is monotonically increasing with a parameter

I'm trying to estimate the parameters of a Pareto distribution (actually the paretian tail of a generic distribution) via Metropolis-Hastings. The problem is that the loglikelihood, $$ l(\alpha, ...
0
votes
0answers
5 views

RJAGS Prediction Envelope for Distributions

I am using RJAGS to estimate the parameters of my model from data. My model contains diverse range of parameters for known distributions. I get the credibility intervals for my distribution parameters ...
0
votes
0answers
12 views

How to get the step size right for an MCMC run

I'm trying to constrain data using MCMC but the results I'm getting out are sensitive to MCMC step size (the widths of the posteriors are smaller for smaller step sizes but the chains don't converge ...
0
votes
1answer
48 views

R: linear mixed effects plus MCMC estimation

In a paper I wrote a few years ago, I wrote the following: All results were analyzed using linear mixed models effects, with Subjects and Items as random effects. I present p-values estimated from ...
2
votes
0answers
40 views

Can I use Adaptive MCMC in any setting?

In time series econometrics and finance, most Bayesian authors approximate their models with a Gibbs Sampler, this is especial true for state space models, SV and so forth. The dimensionality of ...
0
votes
0answers
11 views

How does Shuffled-Complex-Evolution-Metropolis algorithm compare to other adaptive samplers (e.g. NUTS)?

I recently heard of the Shuffled-Complex-Evolution-Metropolis algorithm and am curious how it compares to other adaptive MCMC sampling algorithms. Unfortunately I am still learning about optimizing ...
0
votes
0answers
16 views

Slice sampler in PyMC gives values outside support of prior

When sampling variables with Uniform or Beta priors in PyMC I'm getting some very strange behavior. The sampler ends up drawing values far, far outside of the support of the prior. Here's a simple ...
0
votes
1answer
33 views

Could anyone help me check my gibbs sampling code? [closed]

I am now trying to write a Gibbs Sampling code based on the posteriors from a paper "Bayesian Regularization via Graph Laplacian", writer: Fei Liu, et. When I run the code, it always show the error: ...
2
votes
0answers
36 views

Test-retest correlation for panel data

I've run an experiment in which subjects rate how much they like six different objects on a 1-5 scale on two occasions. I'd like to obtain a summary measure of how consistent are the subjects in their ...
3
votes
2answers
68 views

Metropolis Hastings Algorithm - Prior vs Proposal vs Numerator of Bayes Theorem

I've been using this technique in 'black-box' form for a little while as a physics student. I have been struggling to understand what's happening under the hood for some time and I think I almost ...
1
vote
0answers
23 views

Derivation of formulas for Boltzmann machines - MCMC

With interest i read the latest post on https://theclevermachine.wordpress.com/ on Boltzmann machines, and the derivation of the underlying formulas. The derivation (per below) shows that the ...
4
votes
1answer
50 views

Managing high autocorrelation in MCMC

I'm building a rather complex hierarchical Bayesian model for a meta-analysis using R and JAGS. Simplifying a bit, the two key levels of the model have $$ y_{ij} = \alpha_j + \epsilon_i$$ $$\alpha_j ...
1
vote
1answer
53 views

Conditions on transformation function in Monte Carlo expectation

If I have an i.i.d. set of samples $\theta_1, \ldots, \theta_n$ from my posterior $p(\theta | y)$ then: $ E(f(\theta | y)) = \int f(\theta) p(\theta | y)\, \mathrm{d}\theta \approx \frac{1}{n} ...
0
votes
1answer
37 views

Weighted log-probabilities in generalised gamma distribution

This question is related to the problems I mentioned in this question. I am not sure if there is a good solution, but am hoping someone more experienced with this type of thing can help out. I am ...
2
votes
1answer
44 views

MCMC Estimation of Multidimensional 2PL IRT Model Using JAGS

I'm trying to prepare for some more advanced work involving MIRT models I'll be doing later this year by fitting a very simple multidimensional 2PL model to some simulated data using MCMC methods in ...
3
votes
1answer
38 views

Why do we want low autocorrelation for MCMC convergence?

Usually, autocorrelation is one diagnostical tool for judging the convergence of a MCMC trail. Low autocorrelation is desired as this would mean that the parameter space is well explored. I have a ...
1
vote
0answers
29 views

Using Markov Chain Monte Carlo to compute the chances that a particular solitaire laid out with 52 cards would come out successfully

Based on some references I got from another question I learned that: While convalescing from an illness in 1946, Stan Ulam was playing solitaire. It, then, occurred to him to try to compute the ...
2
votes
0answers
24 views

Uniform Sampling from Intersections of Faces of Simplices

I'm trying to sample uniformly on the intersections of faces of several simplicies, with all coordinates being non-negative. That is, given constraints $$A\vec{w}=\vec{b} \ \ and \ \ \vec{w} \geq ...
0
votes
0answers
22 views

Significant variables in MCMCglmm

I am trying to run MCMCglmm with all my independent variables against a binary variable (Y). When none of the variables came out to be significant I deleted the one with the highest pMCMC value and ...
1
vote
1answer
69 views

Weighting observations and measurement uncertainty in bayes

I am working on using MCMC (via STAN) to estimate model parameters for a bunch of observations with measurement uncertainty. I'm having problems with weighting each observation, and have reduced the ...
1
vote
1answer
57 views

In a Boltzmann machine, why isn't there a simple expression for the optimal edge weights in terms of correlations between variables?

Suppose I have a fully connected, fully visible Boltzmann machine (no hidden variables) with binary variables $x_i\in \{+1, -1\}$ that defines the probability distribution $$ p(\mathbf{x} ; ...
1
vote
0answers
17 views

Can I use an unknown number of variables to model my time-series?

I have a bunch of data-sets showing the relationship between two observables, "force" and "time". See example plot You see the regularity of the features: There is a region of linearly increasing ...
3
votes
1answer
37 views

Metropolis-Hastings with two dimensional target distribution

I'm confused in the following situation: I want to sample by writing code (Java) from the following distribution that is characterized by the mean vectors and covariance matrices: $$ p\left ( ...
2
votes
0answers
37 views

Using empirical priors in PyMC

I'm using PyMC to sample the posterior distribution and I've run into a roadblock with using priors from samples, not models. My situation is as follows: I have some empirical data for a parameter ...
0
votes
0answers
30 views

Conditional density of topic assignment in A Split-Merge MCMC Algorithm for the Hierarchical Dirichlet Process

I'm trying to implement the algorithm described in A Split-Merge MCMC Algorithm for the Hierarchical Dirichlet Process by Chong Wang and David Blei. Equation (7) on page 4 has the terms ...
2
votes
0answers
112 views

When and why do I have to use “trait” for multinomial multilevel models with MCMCglmm in R?

I want to estimate a multilevel multinomial logit model but I am struggling with the terminology and notation used by the R-package MCMCglmm. There is documentation ...
2
votes
0answers
45 views

Metropolis-Hastings acceptance rate confusion

I ran a Bayesian model that have about 2700 parameters. Among these parameters, Adaptive Metropolis algorithm was implemented to estimate ~790 parameters in the I-group and Metropolis algorithm was ...
2
votes
0answers
115 views

PyMC for Categorical Latent Model

I'm learning PyMC and am trying to fit a simple categorical mixture model but the sampling estimates don't converge to the true values. I'm wondering if I've specified the model incorrectly or am ...
1
vote
1answer
36 views

Marginal effects from Bayesian probit

I'm trying to run a standard Bayesian probit model, and I can't find any packages in R that will give me marginal effects (the most common way to interpret probit results in my field), nor do they ...
1
vote
0answers
37 views

Discrete MCMC JAGS chains get stuck

I have been running a model where one of the parameters is discrete. I can't think of a simple way to represent this model, so I won't (unless necessary) post it here. My issue is, that when I look ...
0
votes
0answers
15 views

Heidelberg & Welch tests, some are passed and some are failed for multiple chains

I am using mcmc(specifically jags) sampling to get posterior dist'n. The problem is that some variables are passing the stationarity test of Heidelberg & Welch test... BUT, if I run multiple ...
4
votes
0answers
36 views

What is the equivalent for cdfs of MCMC for pdfs?

In conjunction with a Cross Validated question on simulating from a specific copula, that is, a multivariate cdf $C(u_1,\ldots,u_k)$ defined on $[0,1]^k$, I started wondering about the larger picture, ...
1
vote
2answers
29 views

JAGS choosing a random subset of a vector

I would like to allow the subset of a vector I am summing over to be a random quantity. My model is of the form (albeit more complex): ...
4
votes
1answer
60 views

Metropolis : Set first sample value instead of randomly generate an arbitary value

According to Metropolis-Hasting algorithm, the first sample is an arbitrary value generated randomly at the Initialization step. ( http://en.wikipedia.org/wiki/Metropolis%E2%80%93Hastings_algorithm ) ...
0
votes
0answers
10 views

Intraspecific variation in MCMCglmm

I want to use MCMCglmm to account for phylogenetic autocorrelation on my GLMM. However I have more than one trait measurement for certain species. Is there a way ...
1
vote
0answers
14 views

Is it possible to extend dynamic spatial (space-time) panel model to a SARAR (SAC) specification?

I have panel data of 200 regions over 20 years. My goal is to estimate a dynamic spatial (space-time) panel model. I would like to employ an extension of model used in Debarsy/Ertur/LeSage (2009): ...
0
votes
0answers
28 views

Probability that a large corpus of text is generated with the same parameters as a subset

Let's say I have a process which generates different words at a set (unknown) frequency per word. I sample this process X times, generating the word "yo" Y times. I then look at a subset of my ...
2
votes
0answers
47 views

pymc implementation of ThinkBayes 1.3 cookie problem

This is obviously overkill for this problem, but I thought it would help cement the concepts for me. The problem: Suppose there are two bowls of cookies. Bowl 1 contains 30 vanilla cookies and 10 ...
0
votes
0answers
24 views

Combining independent MCMC samplers from different models

I am interested in sampling from the joint posterior $p(\theta_k,k \mid y)$ where $\theta_k$ belongs to the parameter space of model with index $k$. One way of doing this is with the reversible jump ...
5
votes
0answers
56 views

Hamiltonian Monte-Carlo with piecewise differentiable log likelihood

This is a bit of a curious situation. I have an energy function $E=S+N$ which is the sum of a smooth differentiable function $S$ and a piecewise constant "noise" function $N$. This means that on ...
2
votes
1answer
44 views

Combining multiple parallel MCMC chains into one longer chain

Let's say that one has run $m$ parallel MCMC chains where each chain has had burn-in. Let the resulting chains be denoted by $$ x_1^{(i)},\dots,x_N^{(i)} \quad \text{ for } i=1,\dots,m,$$ where $N$ is ...
2
votes
1answer
41 views

Gibbs sampler for a particular distribution

I'm trying to implement Gibbs Sampler for the distribution: $$\pi(x,y)=e^{-10(x^2-y)^2-(y-1/4)^4}$$ So, like the first step, I need to find: $$\phi(t) = \int_{-\infty}^{t} e^{-10(x^2-y)^2-(y-0.25)^4} ...
0
votes
1answer
25 views

What is this bivariate distribution called and how to make it posterior?

I am trying to make this bivariate density function as posterior f(x,y) = k x^2 exp( - x y^2 - y^2 + 2y - 4x) and try jags instead of implementing in R as in ...
-1
votes
2answers
98 views

Large? Number of parameters in MCMC model [closed]

I am implementing a Hierarchical Bayesian Modeling in order to model the relation between the independent and dependent parameters $(x, y)$. I assume the relation is: $$ y_i = \alpha + \beta x_i + ...
1
vote
1answer
72 views

In Bayesian analysis, how to sample from full conditional given uniform prior and normal data likelihood?

[EDIT] This question comes from the example of OpenBUGS manual: Stagnant: a changepoint problem and an illustration of how NOT to do MCMC! I also asked another question regarding this example. ...
1
vote
0answers
34 views

OpenBUGS example: Stagnant, a changepoint problem and an illustration of how NOT to do MCMC! - Why is the second parameterization better?

I am working on an Bayesian problem from an OpenBugs example: Stagnant, a changepoint problem and an illustration of how NOT to do MCMC!. This is a changepoint problem. Basically we assume a model ...
1
vote
0answers
42 views

In MCMC simulation, how to deal with very small likelihood values that couldn't be represented by computer? [duplicate]

I am working on a Bayesian project based on Stagnant data from a OpenBugs example, which is a changepoint problem. Basically we assume a model with two straight lines that meet at a certain ...