# Tag Info

25

This paper by Christian (Xi'an) Robert and George Casella provides a nice summary of the history of MCMC. From the paper (emphasis is mine). What can be reasonably seen as the first MCMC algorithm is what we now call the Metropolis algorithm, published by Metropolis et al. (1953). It emanates from the same group of scientists who produced the Monte Carlo ...

15

Some possible generic explanations for this perceived discrepancy, assuming of course there is no issue with code or likelihood definition or MCMC implementation or number of MCMC iterations or convergence of the likelihood maximiser (thanks, Jacob Socolar): in large dimensions $N$, the posterior does not concentrate on the maximum but something of a ...

15

With flat priors, the posterior is identical to the likelihood up to a constant. Thus MLE (estimated with an optimizer) should be identical to the MAP (maximum a posteriori value = multivariate mode of the posterior, estimated with MCMC). If you don't get the same value, you have a problem with your sampler or optimiser. For complex models, it is very ...

12

The excellent answer by knrumsey gives some history on the progression of important academic work in MCMC. One other aspect worth examining is the development of software to facilitate MCMC by the ordinary user. Statistical methods are often used mostly by specialists until they are implemented in software that allows the ordinary user to implement them ...

8

The confusion stems from a misunderstanding of the notation $$V \sim f_V$$ which means both (a) $V$ is a random variable with density $f_V$ and (b) $V$ is created by a PRNG algorithm that reproduces a generation of a random variable with density $f_V$. Each time a generation $V_i\sim f_V$ occurs in the algorithm from Casella and Berger, a new ...

8

In my view, MCMC/bootstrapping/permutation methods all fall under the category of computational techniques. They aren't tied down to a specific approach or way of thinking about a problem but rather an algorithmic approach to a class of problems. Techniques that involve resampling and iteration don't arise from a machine learning framework, they come out of ...

7

A divergent transition in Stan tells you that the region of the posterior distribution around that divergent transition is geometrically difficult to explore. For example here is a quote from the manual: The primary cause of divergent transitions in Euclidean HMC (other than bugs in the code) is highly varying posterior curvature, for which small step ...

6

As a preliminary, let me point out that the issue of reconstituting the joint from the marginals is a constant theme on this forum, the answer being invariably that it is not possible without further assumptions. "Suppose we know the marginals of $X$ and $Y$ and the covariance matrix between $X$ and $Y$." This information is not enough for simulating $(... 6 Strictly speaking, you have to rerun your MCMC algorithm from scratch to approximate the new posterior. MCMC algorithms are not sequential, which means that you cannot update their output with new data to update your estimate of the posterior. You just have to redo it. However, you can use importance sampling to recursively update your posterior ... 6 Recycling proposed values in a Metropolis-Hastings algorithm goes under the name of Rao-Blackwellisation. For instance, we made such a proposal in Casella and Robert (1996) Rao-Blackwellisation of sampling schemes. Douc and Robert (2013) A vanilla Rao-Blackwellisation of Metropolis-Hastings algorithms Note that the weighting you propose in the question:$...

6

does using the log-density not break any of the assumptions of MH or other algorithms used for sampling from densities? And can one just take the exponential of the samples obtained via MH to get transform them back to samples from the original density? The issue with underflows and upperflows can be treated through logarithms to some extent, without ...

5

We addressed this problem in our 2011 vanilla Rao-Blackwellisation paper. The limiting distribution of the unique simulations in the Metropolis-Hastings sequence is associated with the density $$\tilde\pi(x)\propto\pi(x)\bar{\alpha}(x)\quad\text{where}\quad\bar{\alpha}(x)=\int_{\mathcal X}\alpha(x,y)q(y|x)\,\text{d}y$$if $\pi(\cdot)$ is the original target ...

5

Are you wondering how GR works, or why 1.1 seems to be the accepted cut-off. If the latter, you're not alone: arXiv paper questioning 1.1 cutoff argues that 1.1 is too high. They also propose a revised version of GR that is improved and can even evaluate a single chain. The Stan folks are also working on a revised version of Stan's Rhat, which I believe is ...

5

As I explained earlier this week in my introductory lecture to Monte Carlo methods, the founding principle of such numerical methods is the Law of Large Numbers, or the stabilisation of empirical frequencies to their expectations. Markov chain Monte Carlo algorithms are a special case of methods implementing the Monte Carlo principle, in that Markov chains ...

5

Since $$[\{1-\alpha(X_{i-1},Y_i)\}f(X_{i-1})+\alpha(X_{i-1},Y_i)f(Y_i)]= \overbrace{\mathbb E[f(X_i)|X_{i-1},Y_i]}^\text{integrating out U_i}=H(X_{i-1},Y_i)$$ the sum$$A_nf=\frac{1}{n}\sum_{i=1}^n H(X_{i-1},Y_i)$$satisfies the standard ergodic theorem for the Markov chain $(X_{i-1},Y_i)$ and under geometric ergodicity satsfies a Central Limit theorem with ...

5

You have performed MCMC in order to generate a sample from the posterior distribution $$\pi(\theta_1,\theta_2,\theta_3 \lvert y)$$ Because it is a random sample $\{ (\theta_{i1},\theta_{i2},\theta_{i3})\}_{i=1}^n$ anything that holds for random samples apply. This means that Law of Large Numbers will apply such that a consistent estimator of $\mathbb E[\... 4 This is an interesting idea, but I see several difficulties with it: contrary to standard importance sampling, or even Metropolised importance sampling the proposal is not acting in the same space as the target distribution, but in a space of smaller dimension so validation is unclear [and may impose to keep weights across iterations, hence facing ... 4 The biggest problem with drawing from the prior is if a user is using a rather flat prior. For example, if a user is using a logistic regression model and they don't want the prior to have much of an effect on the posterior, they may choose to make the prior a normal distribution with a standard deviation of 100. Taking a draw from this prior will, with very ... 4 This looks similar to "local" importance sampling. In the literature, constructing estimators of this sort seem to termed as "waste-recyling", and quick search yields a few papers: This thesis by Murray PNAS paper by Freknel here Does waste-recyling help by Delmas and Jourdain here Their use in parallel MCMC here Further search using the keyword "waste-... 4 Stan computes a log-posterior density and uses its gradient to do sampling. It does this by incrementing a variable storing the log probability (really, the log kernel. Ben Goodrich points out that Stan only needs to care about the log probability up to constant terms, which are neglected). At each iteration, each sampling statement in the model block ... 4 Short answer: this is a code error. If I use the log scale, should I also use a log-proposal? The probability of acceptance is always $$\min\left\{1,\dfrac{\pi(x^\text{new})}{\pi(x^\text{old})}\times\dfrac{q(x^\text{old}|x^\text{new})}{q(x^\text{new}|x^\text{old})} \right\}$$ which can also be written as $$\min\left\{1,\exp[\ln\pi(x^\text{new})-\ln\... 4 For what you described, I cannot see any direct relationship with MCMC. What you needed is just a forward sampling. Here is how it works (suppose we have discrete binary random variables): Step 1. get a sample for X_1. In order to do this step, we need to have the distribution P(X_1). (Something like$$ P(X_1)=\left\{ \begin{array}... 4 Once you have the form of the PDF, there are various techniques for sampling. Some easy forms can be handled via Inverse Transform Sampling. Some special forms can be handled via methods special methods, e.g. sampling from normal distribution via Box-Müller. Other general methods exist for PDFs with non-easy/non-special forms (i.e. inverse transform sampling ... 3 Hamiltonian Monte Carlo (HMC), originally called Hybrid Monte Carlo, is a form of Markov Chain Monte Carlo with a momentum term and corrections. The "Hamiltonian" refers to Hamiltonian mechanics. The use-case is stochastically (randomly) exploring high dimensions for numeric integration over a probability space. Contrast with MCMC Plain/vanilla Markov ... 3 You are right - the conditioning data is not necessary: The Gibbs sampler is an MCMC method designed to sample from an arbitrary joint distribution, in cases where it is simpler to get the conditional distribution of each element (conditional on the other elements) than it is so get the marginal distribution of the elements. The Gibbs sampler is applicable ... 3 Multiplying the likelihood by a constant makes no difference, as you state. You can take$\tilde L(x)=\frac1n L(x)$with$K>0$and obtain the same result. When you take the log, you will then get$\log\tilde L(x) = \log L(x) -n$: on the log scale, you can add a constant, but not multiply by a constant. Multiplying the log-likelihood by a constant$k$(... 3 I assume the authors are talking about the "batch means estimators" which are very popular in steady state simulation and MCMC. Suppose$X_1, X_2, \dots X_N$are from a Markov chain with stationary distribution$\pi$with mean$\theta$and variance$\tau^2$. Let$\bar{\theta}$be the sample average. Then if the samples had been iid, the variance of$\bar{\...

3

Without proposing here a complete recap of Markov chain Monte Carlo methods, let me point out that the fundamental concept behind these methods is that they converge to the distribution $\pi^\star(\cdot)$ they aim at simulating, rather than simulating values from that distribution from the start of the method (as in, e.g., accept-reject algorithms). This ...

3

MCMC relies on building a Markov chain whose stationary distribution is a joint distribution you wish to sample from. But you don't start at the stationary distribution, you start at some initial value (in multivariate space). It may take some time for the process to "wash out" the initial conditions. Under suitable conditions the approach to the ...

3

Personally, I find it very hard to draw a line between the two, as there is clearly some overlapping. Machine Learning is a field that is based on classical statistics and USES statistic models heavily. Also, the mathematics behind Machine Learning can get extremely complicated, so I really would not use the mathematical argument as a discriminant. One ...

Only top voted, non community-wiki answers of a minimum length are eligible