Markov Chain Monte Carlo (MCMC) refers to a class of methods for generating samples from a target distribution by generating random numbers from a Markov Chain whose stationary distribution is the target distribution. MCMC methods are typically used when more direct methods for random number ...

learn more… | top users | synonyms

1
vote
1answer
48 views

In a Boltzmann machine, why isn't there a simple expression for the optimal edge weights in terms of correlations between variables?

Suppose I have a fully connected, fully visible Boltzmann machine (no hidden variables) with binary variables $x_i\in \{+1, -1\}$ that defines the probability distribution $$ p(\mathbf{x} ; ...
1
vote
0answers
10 views

Can I use an unknown number of variables to model my time-series?

I have a bunch of data-sets showing the relationship between two observables, "force" and "time". See example plot You see the regularity of the features: There is a region of linearly increasing ...
3
votes
1answer
28 views

Metropolis-Hastings with two dimensional target distribution

I'm confused in the following situation: I want to sample by writing code (Java) from the following distribution that is characterized by the mean vectors and covariance matrices: $$ p\left ( ...
2
votes
0answers
23 views

Using empirical priors in PyMC

I'm using PyMC to sample the posterior distribution and I've run into a roadblock with using priors from samples, not models. My situation is as follows: I have some empirical data for a parameter ...
0
votes
0answers
20 views

Conditional density of topic assignment in A Split-Merge MCMC Algorithm for the Hierarchical Dirichlet Process

I'm trying to implement the algorithm described in A Split-Merge MCMC Algorithm for the Hierarchical Dirichlet Process by Chong Wang and David Blei. Equation (7) on page 4 has the terms ...
1
vote
0answers
54 views
+50

Notation of MCMCglmm for multinomial multilevel models

I want to estimate a multilevel multinomial logit model but I am struggling with the terminology and notation used by the R-package MCMCglmm. There is documentation ...
2
votes
0answers
37 views

Metropolis-Hastings acceptance rate confusion

I ran a Bayesian model that have about 2700 parameters. Among these parameters, Adaptive Metropolis algorithm was implemented to estimate ~790 parameters in the I-group and Metropolis algorithm was ...
0
votes
0answers
8 views

Save and Restore current state in PYMC [migrated]

Recently, I launched a Bayesian model run that are written in PYMC. Due to power outage, the results generated during halfway of the run are gone. So, the logical step is to look for ways to save the ...
2
votes
0answers
76 views

PyMC for Categorical Latent Model

I'm learning PyMC and am trying to fit a simple categorical mixture model but the sampling estimates don't converge to the true values. I'm wondering if I've specified the model incorrectly or am ...
1
vote
1answer
14 views

Marginal effects from Bayesian probit

I'm trying to run a standard Bayesian probit model, and I can't find any packages in R that will give me marginal effects (the most common way to interpret probit results in my field), nor do they ...
1
vote
0answers
27 views

Discrete MCMC JAGS chains get stuck

I have been running a model where one of the parameters is discrete. I can't think of a simple way to represent this model, so I won't (unless necessary) post it here. My issue is, that when I look ...
0
votes
0answers
9 views

Heidelberg & Welch tests, some are passed and some are failed for multiple chains

I am using mcmc(specifically jags) sampling to get posterior dist'n. The problem is that some variables are passing the stationarity test of Heidelberg & Welch test... BUT, if I run multiple ...
4
votes
0answers
29 views

What is the equivalent for cdfs of MCMC for pdfs?

In conjunction with a Cross Validated question on simulating from a specific copula, that is, a multivariate cdf $C(u_1,\ldots,u_k)$ defined on $[0,1]^k$, I started wondering about the larger picture, ...
1
vote
2answers
22 views

JAGS choosing a random subset of a vector

I would like to allow the subset of a vector I am summing over to be a random quantity. My model is of the form (albeit more complex): ...
4
votes
1answer
50 views

Metropolis : Set first sample value instead of randomly generate an arbitary value

According to Metropolis-Hasting algorithm, the first sample is an arbitrary value generated randomly at the Initialization step. ( http://en.wikipedia.org/wiki/Metropolis%E2%80%93Hastings_algorithm ) ...
0
votes
0answers
7 views

Intraspecific variation in MCMCglmm

I want to use MCMCglmm to account for phylogenetic autocorrelation on my GLMM. However I have more than one trait measurement for certain species. Is there a way ...
1
vote
0answers
8 views

Is it possible to extend dynamic spatial (space-time) panel model to a SARAR (SAC) specification?

I have panel data of 200 regions over 20 years. My goal is to estimate a dynamic spatial (space-time) panel model. I would like to employ an extension of model used in Debarsy/Ertur/LeSage (2009): ...
0
votes
0answers
24 views

Probability that a large corpus of text is generated with the same parameters as a subset

Let's say I have a process which generates different words at a set (unknown) frequency per word. I sample this process X times, generating the word "yo" Y times. I then look at a subset of my ...
2
votes
0answers
32 views

pymc implementation of ThinkBayes 1.3 cookie problem

This is obviously overkill for this problem, but I thought it would help cement the concepts for me. The problem: Suppose there are two bowls of cookies. Bowl 1 contains 30 vanilla cookies and 10 ...
0
votes
0answers
22 views

Combining independent MCMC samplers from different models

I am interested in sampling from the joint posterior $p(\theta_k,k \mid y)$ where $\theta_k$ belongs to the parameter space of model with index $k$. One way of doing this is with the reversible jump ...
5
votes
0answers
52 views

Hamiltonian Monte-Carlo with piecewise differentiable log likelihood

This is a bit of a curious situation. I have an energy function $E=S+N$ which is the sum of a smooth differentiable function $S$ and a piecewise constant "noise" function $N$. This means that on ...
2
votes
1answer
27 views

Combining multiple parallel MCMC chains into one longer chain

Let's say that one has run $m$ parallel MCMC chains where each chain has had burn-in. Let the resulting chains be denoted by $$ x_1^{(i)},\dots,x_N^{(i)} \quad \text{ for } i=1,\dots,m,$$ where $N$ is ...
2
votes
1answer
38 views

Gibbs sampler for a particular distribution

I'm trying to implement Gibbs Sampler for the distribution: $$\pi(x,y)=e^{-10(x^2-y)^2-(y-1/4)^4}$$ So, like the first step, I need to find: $$\phi(t) = \int_{-\infty}^{t} e^{-10(x^2-y)^2-(y-0.25)^4} ...
0
votes
1answer
25 views

What is this bivariate distribution called and how to make it posterior?

I am trying to make this bivariate density function as posterior f(x,y) = k x^2 exp( - x y^2 - y^2 + 2y - 4x) and try jags instead of implementing in R as in ...
-1
votes
2answers
90 views

Large? Number of parameters in MCMC model [closed]

I am implementing a Hierarchical Bayesian Modeling in order to model the relation between the independent and dependent parameters $(x, y)$. I assume the relation is: $$ y_i = \alpha + \beta x_i + ...
1
vote
1answer
57 views

In Bayesian analysis, how to sample from full conditional given uniform prior and normal data likelihood?

[EDIT] This question comes from the example of OpenBUGS manual: Stagnant: a changepoint problem and an illustration of how NOT to do MCMC! I also asked another question regarding this example. ...
1
vote
0answers
32 views

OpenBUGS example: Stagnant, a changepoint problem and an illustration of how NOT to do MCMC! - Why is the second parameterization better?

I am working on an Bayesian problem from an OpenBugs example: Stagnant, a changepoint problem and an illustration of how NOT to do MCMC!. This is a changepoint problem. Basically we assume a model ...
1
vote
0answers
39 views

In MCMC simulation, how to deal with very small likelihood values that couldn't be represented by computer? [duplicate]

I am working on a Bayesian project based on Stagnant data from a OpenBugs example, which is a changepoint problem. Basically we assume a model with two straight lines that meet at a certain ...
0
votes
0answers
22 views

How to interpret the results of geweke.diag() function present in Coda package of R?

I am using geweke.diag() to check the convergence of an MCMC chain, I am using following R-Code for the purpose ...
0
votes
0answers
38 views

Is there good introduction to scala for MCMC simulation

https://darrenjw.wordpress.com/2011/07/16/gibbs-sampler-in-various-languages-revisited/ It seems scala is the way to go for MCMC for complex models, Is there any good introduction for scala to get ...
4
votes
1answer
48 views

Analyzing output in MCMC

I am using emcee to do inference on some data. I am trying to fit my data to a line of equation $ y = mx + b $. ...
0
votes
0answers
70 views

Fitting a Bayesian Hierarchical Poisson Regression in R

I'm trying to fit a Bayesian hierarchical poisson regression. To do so, I'm using MCMChpoisson function from MCMCpack in R. Based on this package, the model is: $$Y_i \sim Poisson(\lambda_i)$$ ...
0
votes
0answers
15 views

How can I smooth a set of distributions?

I have a set of results from MCMC modelling of a variable at discrete time points and I would like to know what kind of approaches I could take to smooth the results, given I would expect some kind of ...
-2
votes
2answers
1k views

Priors in Bayesian MCMC

I am trying to understand how the choice of priors affects a Bayesian model estimated using MCMC. At a basic level I understand that the product of the prior and the likelihood are proportional to the ...
1
vote
0answers
24 views

What are good values for autocorrelation, Gelman, and cross-correlation in rjags?

I don't want to post my whole code since it is long, so I will only post part of it: ...
2
votes
1answer
32 views

Bayesian Mixture Model Gibbs Sampler for two linear relationships

I am attempting to use a Gibbs Sampler to model a mixture of two groups, where the group membership is defined by a linear relationship conditional on x. Both groups have the same slope and intercept, ...
2
votes
1answer
162 views

Metropolis-Hastings MCMC for Bayesian Regression in R

I am looking for a teaching example of a multivariate (not bivariate) implementation of Metropolis-Hastings for MCMC in R. I know several packages implement the algorithm more generally, but the code ...
6
votes
0answers
99 views

When should I be worried about the Jeffreys-Lindley paradox in Bayesian model choice?

I am considering a large (but finite) space of models of varying complexity which I explore using RJMCMC. The prior on the parameter vector for each model is fairly informative. In what cases (if ...
2
votes
0answers
39 views

Is this how a Bayesian bootstrap works?

I am a bit new to the whole nonparametric and Bayesian idea, so tell me if this is correct: to estimate, say, the mean of a dataset's population we do the following: We define a function $f(x)$ that ...
0
votes
0answers
21 views

How to interpret a multilevel model that repeats variables for fixed and random effects (like in MCMhregress)?

I came across a specification of a multilevel model that I'd never seen before. It looks like the R function MCMChregress (see: ...
2
votes
1answer
142 views

Bayesian meta analysis: implementation in BUGS/JAGS/STAN

I would like to conduct a meta analysis in order to collate the information from a number of studies. The parameter of interest is a probability $\theta$. In each of the studies, the observed data ...
1
vote
1answer
61 views
2
votes
2answers
159 views

What are the options when statisticians who only know R need more computing speed

Speed isn't a concern for many statistics projects but sometimes it is, for example MCMC. Assume that hardware improvement is not an option (looking for relatively free solutions that produce orders ...
3
votes
0answers
26 views

Simplification of my Hasting-Metropolis ratio

I am considering the following model : $$ y_{i,j} = f(x_{i,j},\beta,b_{i},d) + \varepsilon_{i,j} $$ where : $1 \leq i \leq s$ $1 \leq j \leq n_{s}$ $\beta$ is a (vector of) fixed effects $b_{i} ...
0
votes
2answers
41 views

Return value of uniform distributions for MCMC simulations

I am confused about how what value should be returned from a uniform distribution when using MCMC simulations. The proper normal distribution is define as $$ p(\theta) = \left\{ \begin{array}{cc} ...
1
vote
1answer
64 views

Gibbs sampling for reducible chain

I am new to Gibbs sampling and I ran into a problem with irreducibility. For the Gibbs sampler to work the Markov chain has to be irreducible. But that assumption is not satisfied in my probability ...
0
votes
0answers
50 views

How do I sample from a black-box model of a probability distribution?

I have a function 'P(x)' where we query for any 'x' it gives a probability value. This function 'P' does not have a closed form and the evaluation is costly. Now 'x' is a set of vectors(matrix whose ...
1
vote
1answer
45 views

How to incorporate parameter constraints in Metropolis Hastings

I am working on parameter estimation of GARCH model with Metropolis Hastings. But the results I have got doesn't look reasonable, actually it is quite different from what I have got from Gibbs ...
5
votes
1answer
39 views

Gibbs Sampler output: how many Markov chains?

When running a Gibbs sampler (for $n=200$ Iterations) with two full conditionals, I get the output $\mathbf{x} = (x_1^{(n)},x_2^{(n)})_{n =1,...,200}$. So $\mathbf{x}$ is the realizations of a Gibbs ...
0
votes
0answers
30 views

Calculating Marginal Data Density for VAR Model

I am currently estimating Bayesian vector autoregressive (BVAR) models and I would like to do model comparison with Bayes factors. I have read about the Gelfand-Day method, the Geweke (1999) modified ...