Hot answers tagged posterior
7
The clue that was in my answer to the previous answer is to look at how I integrated out the parameters - because you will do exactly the same integrals here. You question assumes the variance parameters known, so they are constants. You only need to look at the $\alpha,\mu$ dependence on the numerator. To see this, note that we can write:
...
6
It's always possible to create a prior that will overwhelm your data, no matter how many observations you have. However, for any fixed prior, as the number of observations grows, the influence of the prior shrinks (except for the 0-mass case that Macro pointed out in his comment).
For some prior distributions there's a concept of "prior sample size": if ...
6
To give a more narrow response than the excellent ones that have already been posted, and focus on the advantage in interpretation - the Bayesian interpretation of a, e.g., "95% credible interval" is that the probability that the true parameter value lies within the interval equals 95%. One of the two common frequentist interpretations of a, e.g., "95% ...
6
I find it surprising that a flat likelihood produces convergence issues: it is usually the opposite case that causes problems! The usual first check for such situations is to make sure that your posterior is proper: if not it would explain for endless excursions in the "tails". If the posterior is indeed proper, you could use fatter tail proposals like a ...
5
First note that with the specified hidden Markov model the "B" "B" "B" "B" state sequence has the same conditional probability as the "A" "A" "A" "A" sequence given the emissions and the implementation just happens to pick one arbitrarily over the other. However, introducing a small asymmetry in the emission probabilities the A-sequence will be the unique ...
5
Consider the expression:
$$\frac{exp(A)}{exp(A)+exp(B)}$$
The generic strategy to compute the above expression when $exp(A)$ overflows would be to transform as follows:
$$\frac{1}{1+exp(B-A)}$$
For example R chokes on:
$$\frac{exp(1100)}{exp(1100)+exp(1104)}$$
But, happily computes the following transformation to yield a value of 0.01798621:
...
4
In my opinion, the reason that Bayesian statistics are "better" for intepretation is nothing to do with the priors, but is due to the definition of a probability. The Bayesian definition (the relative plausibility of the truth of some proposition) is more closely in accord with our everyday usage of the word than is the frequentist definition (the long run ...
4
It is high time to convert my comment to an answer.
Bayesian inference has a theoretical fundation, namely Bayesian decision theory. The key tools of Bayesian decision theory are the loss functions.
A Bayesian estimate is derived as follows from the Bayesian decision theory. Define a "distance" $d(\theta,\theta')$ on the parameter space (I will come back ...
3
You can do Bayesian updating for the covariance structure in much the same spirit as you updated the mean. The conjugate prior for the covariance matrix of the multivariate-normal is the Inverse-Wishart distribution, so it makes sense to start there,
$P(\Sigma) \sim W^{-1}(\mathbf{\Psi}, \nu)$
Then when you get your sample $X$ of length $n$ you can ...
3
The convergence depends on several things: the number of parameters, the model itself, the sampling algorithm, the data ...
I would suggests to avoid any general rule and to employ a couple of convergence diagnostics tools to detect appropriate burn-in and thinning number of iterations in each specific example. See also ...
3
I assume, you are refering to the estimation of marginal likelihood. Laplace approximation is used to approximate a marginal likelihood based on normal distribution. Resulting estimate is as follows:
$$f(y)\approx (2\pi)^{d}|\widetilde{\Sigma}|^{1/2}f(y|\widetilde{\theta})f(\widetilde{\theta})$$
Where $\widetilde{\theta}$ is a posterior mode (can be ...
3
One approach to choosing the cutoff value $\epsilon$ for ABC rejection sampling is the following (similar to Aniko's answer). Simulate several test data sets from known parameter values which are vaguely similar to your observed data (e.g. by performing ABC with a relatively large $\epsilon$). From the ABC output for a test data set, some criterion of ...
3
You can avoid this problem altogether by sampling from the untruncated distribution of Y[k], then (in R) discarding all samples for which Y[k] doesn't lie within the constraint bounds. This is a perfectly valid operation, however, if you have few posterior observations in the feasible region, you'll naturally have a large simulation error associated with ...
3
Just from reading your question, it seems to me you are mixing computational performances and statistical inference. The posterior distribution on the parameters does not have to be symmetric, so this is not an indicator of poor MCMC convergence. My advice is to try your code on simulated data (meaning simulated from the very semiparametric Cox model you are ...
3
Your posterior is correct (I edited a missing $\sqrt{}$). Because you are using conjugate priors you actually do not need Gibbs sampling, you can derive exactly the posterior distribution. This is shown for instance in our book, Bayesian Core, where the entire chapter 3 is dedicated to Gaussian linear regression.
If you really want to run a Gibbs sampler ...
2
Your formulas look a bit strange because $\theta=x$. $X$ and $x$ are usually used to denote the random variable and the data, not the parameter.
Anyway, if $X$ is the number of mistakes among 10 iid items then $X\sim Bin(10,x)$, i.e. $X$ follows the binomial distribution. Thus $p(X|x)=\binom{10}{X} x^2(1-x)^8$, and you can compute the posterior using your ...
2
I think you are getting the problem wrong and this has nothing to do with ABC. If your posterior distribution is such that the parameter $\theta$ is not entirely identifiable from one dataset, $x_1$, but is identifiable from a $N$ sample, $x_1,\ldots,x_N$, what you need to consider is the "single" posterior conditional on all of those datasets:
$$
...
2
Ok, I'm assuming you mean the fellow researcher's opinion is that the mean of the process is $\mu_0$ with a standard deviance of that opinion of $\sigma_0$ (rather than the researcher thinks the standard deviation of the process you are modelling is $\sigma_0$, since according to your question you know the process has standard deviation $\sigma$). In that ...
2
Are you looking for techniques, or might you be looking for packages, such as R's coda?
As you note, you'd ignore the first N data points in each chain, which are called "burn-in". How large N is depends on how quickly your model converged and there are a whole host of methods that you can use to give an idea of convergence. (Some of these methods require ...
1
I assume you are trying to conduct Bayesian linear regression using a conjugate prior. The calculations are explained in the wikipedia entry Bayesian linear regression.
The only difference is that you are working with the precision $\beta$ instead of the variance $\sigma^2$, which are equivalent up to a change of variable.
1
Ok, I found the real solution for my problem. I am posting it even if the correct answer to my (misplaced) question is the one selected.
Basically, my question explains how to estimate the mean knowing the covariance, and the answer how to estimate the covariance knowing the mean. But my actual problem was estimating with both parameters unknown.
I found ...
1
A beta distribution with $\alpha$ = 1 and $\beta$ = 1 is the same as a uniform distribution. So it is in fact, uniformative. You're trying to find information about a parameter of a distribution (in this case, percentage of left handed people in a group of people). Bayes formula states:
$P(r|Y_{1,...,n})$ = $\frac{P(Y_{1,...,n}|r)*P(r)}{\int ...
1
I think the most appropriate summary for this application is just to simply state that the years were 15% different, ± some uncertainty. Here's why: first, the concept of statistical significance per se doesn't fit naturally into the Bayesian framework; second, in this application it's not really plausible that the parameter takes the exact same value in the ...
1
This is a common problem with computation of likelihoods for all manner of models; the kinds of things that are commonly done are to work on logs, and to use a common scaling factor that bring the values into a more reasonable range.
In this case, I'd suggest:
Step 1: Pick a fairly "typical" $\theta$, $\theta_0$. Divide the formula for both numerator and ...
1
Try capitalizing on the properties of using the logarithms and summation rather than taking the product of decimal numbers. Following the summation just use the anti-log to put it back into your more natural form. I think something like this should do the trick
...
1
Can you write down the distribution of your first parameter conditional on your second parameter and vice-versa? If so, Gibbs sampling would be a viable option. It's only a couple of lines of code and it can mix almost instantly in many cases.
1
You missed the point that the distribution is a mixture of Gaussians:
each sample $y_i$ is either distributed as per $p(y_i | x)$ with probability $1-w$ and as $p_c(y)$ (clutter distribution for $y$, independent of $x$) with probability $w$.
Let $c_i$ be the indicator variable indicating that sample $i$ was draw from
the clutter distribution; thus, if ...
1
Using a simplified notation, you have $\textit{a priori}$ that $\theta\sim Beta(a,b)$, and $$
f(x\mid\theta)={n\choose x}\theta^x (1-\theta)^{n-x} \, ,
$$
for $x=0,1,\dots,n$ and $\theta\in[0,1]$.
As you already know, the posterior $\pi(\theta\mid x)$ is proportional to
$$
Likelihood \times Prior \, ,
$$
where the $Likelihood$ is just ...
1
The Bayesian framework has a big advantage over frequentist because it does not depend on having a "crystal ball" in terms of knowing the correct distributional assumptions to make. Bayesian methods depend on using what information you have, and knowing how to encode that information into a probability distribution.
Using Bayesian methods is basically ...
1
I have typically seen the uniform prior used in either "instructive" type examples, or in cases in which truly nothing is known about a particular hyperparameter. Typically, I see uninformed priors that provide little information about what the solution will be, but which encode mathematically what a good solution probably looks like. For example, one ...
Only top voted, non community-wiki answers of a minimum length are eligible
