Refers to a class of methods for generating samples from a target distribution by generating random numbers from a Markov Chain whose stationary distribution is the target distribution. MCMC methods are typically used when less computationally expensive methods for random number generation (e.g. ...
1
vote
0answers
49 views
Predicting the next value
I've recently read a blogpost where someone tries to predict the outcome of eurovision.
Quoting from the summary of the site:
Essentially, we can look at people’s voting preferences in the
...
0
votes
0answers
26 views
Calculating HPD for marginal effects of interactions of lmer models
I am fitting a model with lmer, using the following code:
...
0
votes
0answers
21 views
How to calculate Hightest Posterior Density (HPD) of coeficients in a simple regression (lm) in R?
I am trying to calculate HPD for the coeficients of regression models fitted with lm or lmrob in R, pretty much in the same way that can be accomplished by the association of mcmcsamp and HPDinterval ...
1
vote
1answer
63 views
PyMC beginner: how to actually sample from the fitted model
I'm trying a very simple model: fitting a Normal where I assume I know the precision, and I just want to find the mean. The code below seems to fit the Normal correctly. But after fitting, I want to ...
4
votes
1answer
161 views
Understanding MCMC and the Metropolis-Hastings algorithm
Over the past few days I have been trying to understand how Markov Chain Monte Carlo (MCMC) works. In particular I have been trying to understand and implement the Metropolis-Hastings algorithm. So ...
0
votes
0answers
39 views
How many chains should we choose in multi-treatment meta-analysis using MCMC and WinBugs
I am learning to undertake multiple treatment comparison meta-analysis (aka network meta-analysis or MTM) using WinBugs. One of the specifications are choosing the number of chains to run in the MCMC ...
2
votes
1answer
88 views
Most efficient way of sampling a product of distributions $k \cdot f(x) \cdot g(x)$, given that PDFs $f(x)$ and $g(x)$ can easily sampled?
Is there an efficient approach tailored to sampling from a PDF $k \cdot f(x) \cdot g(x)$ (where $k$ is a normalizing constant) that would perform better than naive Metropolis, slice sampling, etc.? ...
0
votes
1answer
52 views
Given MCMC samples, what are the options for estimating posterior of parameters?
Markov chain Monte Carlo (MCMC) is a class of algorithms for sampling from probability distributions. In the end, given a parametric model explaining the data, MCMC could also be used for parameter ...
7
votes
1answer
163 views
Given that one can sample $X \sim f(x)$, is there an easy way to sample $Y \sim k \cdot f(g(y))$ (such as $k \cdot f(e^y)$)?
Say I'm able to sample an RV $X$ from a PDF $f(x)$, can I exploit this to efficiently sample another RV $Y \sim k \cdot f(g(y))$ (where $k$ is a normalizing constant)?
I'm interested in something ...
2
votes
1answer
78 views
Efficient MCMC using the normal approximation of the posterior
I can usually quickly get the normal approximation of the posterior distribution, but I sometimes struggle with setting up an efficient MCMC of the same model. Can I somehow use the results of the ...
0
votes
0answers
24 views
Gibbs sampling product of normals as conditional
I am deriving a gibbs sampler for a joint distribution, where the conditionals of various parameters are product of two non-standard normal distributions. Usually, I have seen that in Gibbs sampling ...
0
votes
1answer
108 views
Metropolis-Hastings algorithm, using a proposal distribution other than a Gaussian in Matlab
I am currently working on my final year project for my mathematics degree which is based on giving an overview of the Metropolis-Hastings algorithm and some numerical examples. So far I have got some ...
2
votes
2answers
128 views
Where can i find a good book that teaches MCMC in R?
I am looking for a good book that will teach me MCMC in R , in particular Block Gibbs and Collapsed MCMC. Preferably with R pseudocode supplemented within the book as well.
Does anyone have any ...
3
votes
3answers
77 views
Analogous measure of AIC which uses the posterior distribution for model selection?
Suppose the following problem: I have $n$ models, $M_k$, each with parameters $\mathbf{\theta}_k$ for a data set $D$. There where previous observations of a subset of the parameters which are common ...
1
vote
0answers
43 views
How to implement multiple GP submodels in PYMC
I'm hoping someone can give me some guidance on implementing
Gaussian processes (GP) with PYMC. In particular, I'm not sure how to use
multiple GP submodels properly within a single pymc model.
More ...
-1
votes
1answer
29 views
CODA gleman.diag, Error in chol.default(W): [closed]
I'd like to use gelman.diag for an MCMC chain I ran in JAGS. It is very large, so I can't provide it.
The chain contains several MVN distributions, and I use a wishart prior on the precision matrix.
...
0
votes
0answers
111 views
How to generate prediction with MCMCglmm package [closed]
I'm new to R and wanna ask if predict.MCMCglmm is working properly or not?
I've used MCMCglmm package for my data with two responses rvu_d and rvu_nm as follows:
...
1
vote
1answer
97 views
Generation of a random vector on an affine hyperplane
I would like to design a proposal of the form:
$$
p(t=(t_i)|\hat{t}=(\hat{t}_i))
$$
where $t$ (and $\hat{t}$) lies in an affine hyperplane $T \subset R^n$:
$$
t \in T \Leftrightarrow
\sum_i t_i=1
...
5
votes
2answers
52 views
Metropolis Sampling and invalid states
I have a short question about Monte Carlo integration with Metropolis sampling. I have a continuous state space, but only certain parts of this state space are valid. It is possible that the ...
1
vote
1answer
129 views
Using STAN (related to BUGS/JAGS) to do linear regression with with ARMA(1,1) noise?
EDIT: I've modified my STAN code and it looks like I am getting numbers close to using R's arima. The original code, now moved to the end, was incorrect.
I've been ...
1
vote
0answers
61 views
Can I adapt a MCMC proposal using a parallel chain?
I am running two MCMC chains (say chain A and chain B) in parallel, using the Metropolis-Hastings algorithm with acceptance probability:
$P(accept\ x_t) = \min\{1, f(x_t)/f(x_{t-1})\}$.
I would like ...
1
vote
0answers
66 views
Can someone point me to a good example of using bayesian models for making marketing decisions?
I am tasked to build a bayesian model to support decision making for paid search marketing. I've researched online and found several scholarly articles on using Hierarchical Bayesian model or MCMC in ...
0
votes
0answers
49 views
proposal distribution on a manifold
I try to sample a set of parameters $(x_i)$ from a posterior
$p((x_i)|(o_j))$ using a metropolis-hastings algorithm.
The fact is that a simple independent random walk on each of the $x_i$ as
proposal ...
0
votes
0answers
47 views
Gibbs sampling for the latent variables in Sample Selected Probit
Consider the following model of data generation:
$\ y_1 = I(y_1^*>0) \ where\ y_1^* = \beta_1X_1+\epsilon_1 $
$\ y_2 = I(y_2^*>0)*y_1 \ where\ y_2^* = \beta_2X_2+\epsilon_2 $
$\ y_3 = ...
1
vote
1answer
103 views
Estimated error variance $\sigma^2$ for MCMC estimation in a high-dimensional space
Let $f$ be a function such that:
$$f~:~(x,~\theta)\in\mathbb{R}^{3}\times\mathbb{R}^{12} \rightarrow f(x,~\theta)\in\mathbb{R}^3$$
My observations $y$ are noisy values taken by the function $f(\cdot ...
2
votes
0answers
90 views
Gibbs sampling from full conditionals
I have the following joint density:
$p(x_1,x_2,y_1,y_2) \propto \exp\left(−\left(x_1^2+x_2^2+c_1(y_2-y_1)^2+c_2(y_2-y_1)^4\right)\right)$
Can I use Gibbs sampling to sample from that? How can I get ...
1
vote
1answer
172 views
Effective Sample Size for posterior inference from MCMC sampling
When obtaining MCMC samples to make inferences on a particular parameter, what are good guides for the minimum number of effective samples that one should aim for?
And, does this advice change as the ...
1
vote
1answer
90 views
Thinning chains in BUGS/JAGS
Hi I have a quick question about the details of running a model in JAGS and BUGS.
Say I run a model with n.burnin=5000, n.iter=5000 and thin=2. Does this mean that the program will
run 5,000 ...
0
votes
0answers
44 views
Best practices for MCMC early stopping?
What are best and / or standard practices for MCMC early stopping?
I have an algorithm which I want to compare with existing non-MCMC algorithms for accuracy and speed. When assessing the speed it's ...
2
votes
2answers
56 views
Regarding the convergence assessment on Markov chain monte carlo (MCMC)
During studying the convergence assessment on Markov Chain Monte Carlo, I once read the following statement:
A slowly converging sampler may be indistinguishable from one that will never ...
1
vote
0answers
53 views
Sampling Stationary Vector Autoregression coefficients while Gibbs Sampling
I have been estimating a Bayesian Vector Autoregression using Gibbs Sampling. When constructing the posterior predictive distribution, I have noticed that when the simulated coefficients from the MCMC ...
3
votes
2answers
73 views
Estimating covariance of the difference of directional distributions derived from Gaussian mixtures
Given Gaussian mixtures $X_1, X_2 \in \mathbb{R}^p$ defined as $$P(X_i = x) = \sum_s \omega^{(s)}_i \mathcal{N}(x; \mu^{(s)}_i, \Sigma_i)$$ where the superscript $(s)$ indexes the $s$-th component of ...
1
vote
1answer
82 views
Computing conditional expectation of ordered normal random variables
There are $m$ normally distributed, independent random variables $N_1, \ldots, N_m$ with distinct means $\mu_1, \ldots \mu_m$ and standard deviations $\sigma_1, \ldots, \sigma_m$. Then, we observe a ...
1
vote
1answer
54 views
When is blocked Metropolis sampling more efficient?
Consider the problem of sampling from $p(\mathbf{x}, \mathbf{y})$ using the Metropolis or Metropolis-Hastings (MH) algorithm. I can either propose samples for $p(\mathbf{x}, \mathbf{y})$ directly, or ...
4
votes
1answer
169 views
Elementary MCMC pseudocode
My aim relates to the project described here, but I've tried to make this question self-contained.
I'm trying to write the MCMC pseudocode for the following inference problem:
Given two observed ...
2
votes
1answer
196 views
Basic Bayesian MCMC to estimate two parameters from binomial distributions given unknown number of trials
This is a very basic question about Bayesian inference. I'm not grasping one or more fundamental concepts.
Let's say I have two observed outcomes, X and Y. I want to infer the probabilities (px and ...
8
votes
1answer
237 views
What are some well known improvements over textbook MCMC algorithms that people use for bayesian inference?
When I'm coding a Monte Carlo simulation for some problem, and the model is simple enough, I use a very basic textbook Gibbs sampling. When it's not possible to use Gibbs sampling, I code the textbook ...
2
votes
0answers
120 views
How do Metropolis-Hastings algorithms ensure the constructed MC has the limiting distribution?
When looking back at the Metropolis-Hastings algorithms, I think I have been missing the most important point:
How do Metropolis-Hastings algorithms ensure the constructed MC has a
limiting ...
4
votes
1answer
145 views
Satisfaction of detailed balance equation in Metropolis-Hastings algorithms?
To sample a target distribution, it suffices to construct a Markov
chain with the target distribution being its limiting distribution.
Note that such a MC may not satisfy the detailed balance ...
1
vote
1answer
180 views
Modelling a mixed model in JAGS/BUGS
I am currently in the process of implementing a model for soccer result prediction in JAGS. Actually, I have implemented several, but I have reached my most difficult challenge yet: A model described ...
2
votes
1answer
142 views
Sampling in discrete distribution
At the moment I'm working with an algorithm that calculates probabilities $Z$ for a finite sequence of data points $X= \left\{ x_1,x_2,...,x_n \right\}$ using an expression like:
$p(Z_i| X_i,\theta) = ...
2
votes
1answer
67 views
choosing an initial state and finding multiple sample points in MCMC?
In a discrete-time Markov chain, having constructed the transition
distributions, the initial distribution should not affect the
limiting distribution (when it exists) in theory. So in MCMC, how
to ...
2
votes
0answers
33 views
Sampling from elliptic pde solution in high dimensions
What is known about sampling from solutions to elliptic PDE's in high dimensions, where it is computationally infeasible to construct or store the actual solution?
For example, let $u$ solve the ...
3
votes
0answers
112 views
Deriving priors for MCMC implementation
I have been working on an assignment lately wherein the object is to implement an MCMC approach to simulate from a generated posterior distribution.
The posterior distribution is generated from a ...
1
vote
0answers
49 views
Is it possible to compute integrals in JAGS?
Is it possible to specify a model that uses an integral in the JAGS dialect of BUGS? (I believe this is possible in *BUGS, but I use a Mac, on which it seems JAGS is better supported.)
2
votes
0answers
123 views
Gibbs sampler on the precision (with a gamma prior) in a hierarchical Bayesian model doesn't converge
I am deriving a Gibbs sampler with a model similar to the model in this paper (a graphical model is shown in page 4). To put it simple, my question only concerns $w_i$ (a $K$-dimensional vector drawn ...
2
votes
1answer
161 views
MCMC and data augmentation
I have been looking at an MCMC data augmentation question; the general form of the question is as follows:
Suppose data gathered on a process suggests $X_{i} \sim \text{Pois}(\lambda)$ and a prior ...
6
votes
1answer
191 views
Reference on examples with R codes for Bayesian simulation based methods of posterior approximation
I have been trying to learn the Bayesian simulation based methods of posterior approximation. Although the theories are now quite clear but I am seeking for some examples with R codes so that I can ...
1
vote
0answers
80 views
What books to read for MCMC theory?
Suppose one is interested in stochastic processes for the purpose getting a theoretical understanding of MCMC. They already have a decent understanding of probability theory (let's say at the level ...
3
votes
2answers
122 views
Non-conventional form for a full conditional
I have a full-conditional of the form :
$$
p(X|...) \propto \exp(-(aX^2 +bX +c/X)),
$$
All of the other full-conditionals of my model simplify to a simpler form and for a sake of simplicity, I would ...
