# Tag Info

22

Let me first explain what a conjugate prior is. I will then explain the Bayesian analyses using your specific example. Bayesian statistics involve the following steps: Define the prior distribution that incorporates your subjective beliefs about a parameter (in your example the parameter of interest is the proportion of left-handers). The prior can be ...

8

To give a more narrow response than the excellent ones that have already been posted, and focus on the advantage in interpretation - the Bayesian interpretation of a, e.g., "95% credible interval" is that the probability that the true parameter value lies within the interval equals 95%. One of the two common frequentist interpretations of a, e.g., "95% ...

7

The clue that was in my answer to the previous answer is to look at how I integrated out the parameters - because you will do exactly the same integrals here. You question assumes the variance parameters known, so they are constants. You only need to look at the $\alpha,\mu$ dependence on the numerator. To see this, note that we can write: ...

6

I find it surprising that a flat likelihood produces convergence issues: it is usually the opposite case that causes problems! The usual first check for such situations is to make sure that your posterior is proper: if not it would explain for endless excursions in the "tails". If the posterior is indeed proper, you could use fatter tail proposals like a ...

6

It's always possible to create a prior that will overwhelm your data, no matter how many observations you have. However, for any fixed prior, as the number of observations grows, the influence of the prior shrinks (except for the 0-mass case that Macro pointed out in his comment). For some prior distributions there's a concept of "prior sample size": if ...

6

In my opinion, the reason that Bayesian statistics are "better" for intepretation is nothing to do with the priors, but is due to the definition of a probability. The Bayesian definition (the relative plausibility of the truth of some proposition) is more closely in accord with our everyday usage of the word than is the frequentist definition (the long run ...

5

Consider the expression: $$\frac{exp(A)}{exp(A)+exp(B)}$$ The generic strategy to compute the above expression when $exp(A)$ overflows would be to transform as follows: $$\frac{1}{1+exp(B-A)}$$ For example R chokes on: $$\frac{exp(1100)}{exp(1100)+exp(1104)}$$ But, happily computes the following transformation to yield a value of 0.01798621: ...

5

First note that with the specified hidden Markov model the "B" "B" "B" "B" state sequence has the same conditional probability as the "A" "A" "A" "A" sequence given the emissions and the implementation just happens to pick one arbitrarily over the other. However, introducing a small asymmetry in the emission probabilities the A-sequence will be the unique ...

5

Yes. And actually this is the interesting invariance property: it means that two Bayesians using a different parameterization of the model but both using the Jeffreys prior, obtain the same posterior distribution (up to change-of-variables) to draw inference. Conceptually, there's no prior predictive distribution based on the Jeffreys prior. The goal of the ...

4

It is high time to convert my comment to an answer. Bayesian inference has a theoretical fundation, namely Bayesian decision theory. The key tools of Bayesian decision theory are the loss functions. A Bayesian estimate is derived as follows from the Bayesian decision theory. Define a "distance" $d(\theta,\theta')$ on the parameter space (I will come back ...

4

I assume, you are refering to the estimation of marginal likelihood. Laplace approximation is used to approximate a marginal likelihood based on normal distribution. Resulting estimate is as follows: $$f(y)\approx (2\pi)^{d}|\widetilde{\Sigma}|^{1/2}f(y|\widetilde{\theta})f(\widetilde{\theta})$$ Where $\widetilde{\theta}$ is a posterior mode (can be ...

4

You can do it more explicitly if the "by inspection" trick bothers you. Suppose that $X\mid\theta\sim\mathrm{Bin}(n,\theta)$ with prior $\mathrm{Beta}(a,b)$. Then, the posterior density is proportional to $$\theta^{x+a-1} (1-\theta)^{n-x+b-1} \, .$$ The full posterior density is $$\pi(\theta\mid x) = A\;\theta^{x+a-1} (1-\theta)^{n-x+b-1} ... 4 There is no generic way to get the boxplot of the differences from two boxplot summaries. The same marginal boxplots are compatible with quite different sets of differences. Here is a simple example: Values X = 1,2,3,4,5 and Y = 5,4,3,2,1 produce the same box plot, but the differences are 4,2,0,-2,-4. Let Z be another copy of either X or Y; ... 3 You can do Bayesian updating for the covariance structure in much the same spirit as you updated the mean. The conjugate prior for the covariance matrix of the multivariate-normal is the Inverse-Wishart distribution, so it makes sense to start there, P(\Sigma) \sim W^{-1}(\mathbf{\Psi}, \nu) Then when you get your sample X of length n you can ... 3 The convergence depends on several things: the number of parameters, the model itself, the sampling algorithm, the data ... I would suggests to avoid any general rule and to employ a couple of convergence diagnostics tools to detect appropriate burn-in and thinning number of iterations in each specific example. See also ... 3 I think you are getting the problem wrong and this has nothing to do with ABC. If your posterior distribution is such that the parameter \theta is not entirely identifiable from one dataset, x_1, but is identifiable from a N sample, x_1,\ldots,x_N, what you need to consider is the "single" posterior conditional on all of those datasets:$$ ...

3

Your posterior is correct (I edited a missing $\sqrt{}$). Because you are using conjugate priors you actually do not need Gibbs sampling, you can derive exactly the posterior distribution. This is shown for instance in our book, Bayesian Core, where the entire chapter 3 is dedicated to Gaussian linear regression. If you really want to run a Gibbs sampler ...

3

One approach to choosing the cutoff value $\epsilon$ for ABC rejection sampling is the following (similar to Aniko's answer). Simulate several test data sets from known parameter values which are vaguely similar to your observed data (e.g. by performing ABC with a relatively large $\epsilon$). From the ABC output for a test data set, some criterion of ...

3

You can avoid this problem altogether by sampling from the untruncated distribution of Y[k], then (in R) discarding all samples for which Y[k] doesn't lie within the constraint bounds. This is a perfectly valid operation, however, if you have few posterior observations in the feasible region, you'll naturally have a large simulation error associated with ...

3

Just from reading your question, it seems to me you are mixing computational performances and statistical inference. The posterior distribution on the parameters does not have to be symmetric, so this is not an indicator of poor MCMC convergence. My advice is to try your code on simulated data (meaning simulated from the very semiparametric Cox model you are ...

2

The Bayesian framework has a big advantage over frequentist because it does not depend on having a "crystal ball" in terms of knowing the correct distributional assumptions to make. Bayesian methods depend on using what information you have, and knowing how to encode that information into a probability distribution. Using Bayesian methods is basically ...

2

Your formulas look a bit strange because $\theta=x$. $X$ and $x$ are usually used to denote the random variable and the data, not the parameter. Anyway, if $X$ is the number of mistakes among 10 iid items then $X\sim Bin(10,x)$, i.e. $X$ follows the binomial distribution. Thus $p(X|x)=\binom{10}{X} x^2(1-x)^8$, and you can compute the posterior using your ...

2

You are right that the paper is saying the wrong thing. You certainly can evaluate the posterior distribution of $x$ at a known location using $O(n)$ operations. The problem is when you want to compute moments of the posterior. To compute the posterior mean of $x$ exactly, you would need $2^N$ operations. This is the problem that the paper is trying to ...

2

Ok, I'm assuming you mean the fellow researcher's opinion is that the mean of the process is $\mu_0$ with a standard deviance of that opinion of $\sigma_0$ (rather than the researcher thinks the standard deviation of the process you are modelling is $\sigma_0$, since according to your question you know the process has standard deviation $\sigma$). In that ...

2

Are you looking for techniques, or might you be looking for packages, such as R's coda? As you note, you'd ignore the first N data points in each chain, which are called "burn-in". How large N is depends on how quickly your model converged and there are a whole host of methods that you can use to give an idea of convergence. (Some of these methods require ...

2

I think the most appropriate summary for this application is just to simply state that the years were 15% different, ± some uncertainty. Here's why: first, the concept of statistical significance per se doesn't fit naturally into the Bayesian framework; second, in this application it's not really plausible that the parameter takes the exact same value in the ...

2

This is a common problem with computation of likelihoods for all manner of models; the kinds of things that are commonly done are to work on logs, and to use a common scaling factor that bring the values into a more reasonable range. In this case, I'd suggest: Step 1: Pick a fairly "typical" $\theta$, $\theta_0$. Divide the formula for both numerator and ...

Only top voted, non community-wiki answers of a minimum length are eligible