Hot answers tagged

20

Good examples so far but consider $$X_i \sim Bernoulli(.5)$$ In that case the distribution of the data will only have support on 0 and 1. But the sample mean will have an ever decreasing probability of taking a value of 0 or 1 as the sample size gets larger and larger. That alone should show that the mean isn't being sampled from the original distribution....


12

As an even more pathological example, consider a sample from the distribution which is uniform on the union of $[0,1]$ and $[3,4]$. As the sample size increases, the mean will tend to 2 which isn't even in the support of the distribution. Another similar example is the uniform distribution on the boundary of the unit sphere (in any number of dimensions)


12

No, it is only valid in cases as the Cauchy distribution, the means of samples of the Cauchy follow the same Cauchy dstribution.


3

Since you are interested in simulating $$\pi(\theta_1|\mathbf x) = \int \pi(\theta_1,\theta_2|\mathbf x)\,\text{d}\theta_2$$ you are essentially seeking a manageable approximation to this integral that does not involve simulating the joint $\pi(\theta_1,\theta_2|\mathbf x)$. The MAP proposal is stating that $$\int \pi(\theta_1,\theta_2|\mathbf x)\,\text{d}\...


3

You basically answered your question. We use multiple chains to diagnose problems with convergence (see e.g. Roy, 2020; Vehtari et al, 2019). If one of the chains explores different area of the posterior then the others, this is a clear sign of problems with convergence. This would often mean that you cannot trust the results. Try running the simulation ...


3

If you could take the mode (you can't here), its called MAP (Maximum a Posteriori) estimate. It's a common point estimator (may not be the best sometimes). If you take the expected value/mean it's called conditional expectation given data, $\mathbb E[X|\mathcal D]$ or the posterior mean, which is a commonly used point estimator as well. The advantage of ...


2

The mode probably isn't a great statistic to take in that case. How about the median or the mean (particularly since you aren't affected by extremely large values, as you are bounded between 0 and 1)?


2

It is sometimes useful to simulate various kinds of probability models, especially when direct mathematical analysis is difficult or involves computational difficulties. Example 1. Suppose a manufacturing process requires two steps in sequence: The first takes a length of time $X$ that is distributed normally with mean 20 and SD 2 hours and, independently; ...


1

As you said, MCMC estimates the posterior distribution. To get a point estimate for $\theta$, it's your decision to use MAP, posterior expectation or something else. MCMC creates samples from the unnormalized version on the RHS, i.e. $p(\theta)p(y|\theta)$. You'll need to choose a prior for $\theta$ and also express the likelihood of the data, i.e. $p(y|\...


1

Rejecting until an acceptance without repeating the current value induces a bias in the algorithm since it does not simulate from the proposal but from another distribution. Many questions on this forum address this issue What is the probability of rejection in rejection sampling? What is the difference between Metropolis-Hastings, Gibbs, Importance, and ...


1

No. For the average to be a sample of the distribution, it must belong to the support of the distribution. Below are two examples where that is not the case (which is sufficient to show that the statement is not true in general). Discrete The distribution p(x=1) = 0.5; p(x=-1) = 0.5 has support $$S=\{-1,1\}$$ but average $0\notin S$. Continuous The ...


1

While the EM algorithm does not involve arbitrary importance distributions, in that $q(z|x)$ is usually defined as $p(z|x,\theta^{(t)})$, if $\theta^{(t)}$ denotes the value of the parameter at the $t$-th step of the algorithm, it happens that, as detailed in the Wikipedia page on the EM algorithm, [where I copied the description and HTML code,] following ...


1

The JSS paper for this package specifies: If not defined, default priors are used which are not proper and this can lead to both inferential and numerical problems. you can access the paper here: https://www.jstatsoft.org/article/view/v033i02 Note that default isn't the same thing as an uninformative prior but is indicative of the challenges that arise ...


1

MCMC is not needed (or suggested) in order to compute a MAP estimate. In general, you won't be able to solve for the root of the MAP objective's derivative. However, you can use numerical optimization to approximately maximize the objective. As noted, MCMC is a sampling method, whereas MAP estimation is an optimization problem. MAP estimation only requires ...


Only top voted, non community-wiki answers of a minimum length are eligible