Questions tagged [posterior]
In Bayesian statistics, the term 'posterior' refers to the probability distribution of a parameter conditioned on the observed data.
110
questions
154
votes
3
answers
242k
views
Help me understand Bayesian prior and posterior distributions
In a group of students, there are 2 out of 18 that are left-handed. Find the posterior distribution of left-handed students in the population assuming uninformative prior. Summarize the results. ...
- 1,541
34
votes
3
answers
17k
views
Why is it necessary to sample from the posterior distribution if we already KNOW the posterior distribution?
My understanding is that when using a Bayesian approach to estimate parameter values:
The posterior distribution is the combination of the prior distribution and the likelihood distribution.
We ...
- 2,481
56
votes
1
answer
27k
views
What are posterior predictive checks and what makes them useful?
I understand what the posterior predictive distribution is, and I have been reading about posterior predictive checks, although it isn't clear to me what it does yet.
What exactly is the posterior ...
20
votes
2
answers
11k
views
Effective Sample Size for posterior inference from MCMC sampling
When obtaining MCMC samples to make inference on a particular parameter, what are good guides for the minimum number of effective samples that one should aim for?
And, does this advice change as the ...
- 3,319
12
votes
2
answers
3k
views
Computation of likelihood when $n$ is very large, so likelihood gets very small?
I am trying to compute this posterior distribution:
$$
(\theta|-)=\frac{\prod_{i=1}^{n}p_i^{y_i}(1-p_i)^{1-y_i}}{\sum_{\text{all}\,\theta,p_i|\theta}\prod_{i=1}^{n}p_i^{y_i}(1-p_i)^{1-y_i}}
$$
The ...
- 4,520
5
votes
4
answers
2k
views
How are artificially balanced datasets corrected for?
I came across the following in Pattern Recognition and Machine Learning by Christopher Bishop -
A balanced data set in which we have selected equal numbers of examples from each of the classes would ...
- 283
26
votes
6
answers
8k
views
Does the Bayesian posterior need to be a proper distribution?
I know that priors need not be proper and that the likelihood function does not integrate to 1 either. But does the posterior need to be a proper distribution? What are the implications if it is/is ...
- 1,851
26
votes
3
answers
16k
views
How can an improper prior lead to a proper posterior distribution?
We know that in the case of a proper prior distribution,
$P(\theta \mid X) = \dfrac{P(X \mid \theta)P(\theta)}{P(X)}$
$ \propto P(X \mid \theta)P(\theta)$.
The usual justification for this step is ...
- 3,102
5
votes
2
answers
4k
views
What to do when your likelihood function has a double product with small values near zero - log transform doesn't work?
I currently have a likelihood function defined as the following:
$$
L=\prod_{i=1}^{N}\left[\prod_{s=1}^{S_i}L_{is}(y\space|\space \rho_A)\times\phi + \prod_{s=1}^{S_i}L_{is}(y\space|\space \rho_B)\...
- 2,057
2
votes
1
answer
4k
views
Bayesian updating with conjugate priors using the closed form expressions
I have one two data sets of scalar values: one large data set (about 700 data points) and one small data set (80 data points). I would like to update the large data set with the small one using the ...
- 133
4
votes
2
answers
2k
views
What is an example of a transformation on a posterior distribution such that the MAP estimate will be non-invariant?
Suppose that we have a posterior distribution $p(\theta\mid y)$ and we wish to define a transformation on $\theta$ such that $\phi = f(\theta)$. I know that generally such transformations will not ...
- 3,892
28
votes
1
answer
30k
views
Multivariate normal posterior
This is a very simple question but I can't find the derivation anywhere on the internet or in a book. I would like to see the derivation of how one Bayesian updates a multivariate normal distribution....
- 843
20
votes
4
answers
4k
views
How is the bayesian framework better in interpretation when we usually use uninformative or subjective priors?
It is often argued that the bayesian framework has a big advantage in interpretation (over frequentist), because it computes the probability of a parameter given the data - $p(\theta|x)$ instead of $p(...
- 6,133
9
votes
2
answers
1k
views
When approximating a posterior using MCMC, why don't we save the posterior probabilities but use the parameter value frequencies afterwards?
I'm currently estimating parameters of a model defined by several ordinary differential equations (ODEs). I try this with a bayesian approach by approximating the posterior distribution of the ...
- 656
8
votes
2
answers
15k
views
What exactly does it mean to and why must one update prior?
I'm still trying to understand prior and posterior distributions in Bayesian inference.
In this question, one flips a coin. Priors:
unfair is 0.1, and being fair is 0.9
Coin is flipped 10x and is ...
- 2,394
4
votes
1
answer
942
views
marginal conditional distribution from MCMC output [duplicate]
I have a MCMC sampler that targets $$\mathbb{P}(U_1,U_2,...U_n \mid G(U) \leq 0)$$ where $U=(U_1,U_2,...U_n)^T$. I realize now I am more interested in estimating the conditional density $$p_k = p(u_k \...
- 105
3
votes
1
answer
115
views
Predictive distributions for data from common distributions
I have data that I assume comes from some distribution, such as normal. I don't care about estimating the parameters of this distribution. Instead, I'd like to know the distribution that future ...
- 1,211
5
votes
1
answer
636
views
Are the mean of samples taken from Metropolis-Hastings MCMC normally distributed?
I've come across the following theorem while studying MCMC. It seems to suggest that the sample mean taken from the MCMC – the posterior marginal expectation – should be normally distributed, using ...
- 591
3
votes
1
answer
2k
views
Relation Between Bayesian Estimation and Maximum a posteriori estimation
Is maximum a posteriori estimation some kind of Bayesian Estimation?
If yes, can you point out other Bayesian estimators?
Edit:
So I've come to know the following (don't know if they are correct):
...
- 213
3
votes
1
answer
219
views
Why does $P(\theta_1\mid D, \theta_2) \propto P(D \mid \theta_1, \theta_2)P(\theta_1)$ hold?
Suppose that in a Bayesian framework we have observed data $D$, using independent prior distributions on the parameters of the model, denoted by $\theta_1, \theta_2$. Then, the joint posterior ...
- 3,892
3
votes
0
answers
1k
views
How do I perform an actual "posterior predictive check"?
This question is the follow-up of this previous question: Bayesian inference and testable implications.
For concreteness, consider the following bayesian model. This model is not to be taken ...
user272422
63
votes
3
answers
41k
views
What is the difference between posterior and posterior predictive distribution?
I understand what a Posterior is, but I'm not sure what the latter means?
How are the 2 different?
Kevin P Murphy indicated in his textbook, Machine Learning: a Probabilistic Perspective, that it is ...
- 2,444
20
votes
4
answers
7k
views
Posterior distribution and MCMC [duplicate]
I have read something like 6 articles on Markov Chain Monte carlo methods, there are a couple of basic points I can't seem to wrap my head around.
How can you "draw samples from the posterior ...
- 671
12
votes
1
answer
731
views
Steps to figure out a posterior distribution when it might be simple enough to have an analytic form?
This was also asked at Computational Science.
I am trying to compute a Bayesian estimate of some coefficients for an autoregression, with 11 data samples: $$ Y_{i} = \mu + \alpha\cdot{}Y_{i-1} + \...
- 2,312
12
votes
2
answers
718
views
Can a proper prior and exponentiated likelihood lead to an improper posterior?
(This question is inspired by this comment from Xi'an.)
It is well known that if the prior distribution $\pi(\theta)$ is proper and the likelihood $L(\theta | x)$ is well-defined, then the posterior ...
- 2,067
12
votes
1
answer
28k
views
Binomial uniform prior bayesian statistics
Suppose to have a binomial distribution where the prior of the parameter is uniform. How can I get the posterior distribution of the parameter?
- 3,129
8
votes
3
answers
1k
views
Dirichlet conjugate update derivation
I am attempting to derive the update equations for the conjugate to the Dirichlet distribution, as outlined here: https://mathoverflow.net/questions/20399/conjugate-prior-of-the-dirichlet-distribution
...
- 89
6
votes
1
answer
673
views
Can the posterior mean always be expressed as a weighted sum of the maximum likelihood estimate and the prior mean?
See this question.
Is this always true? Can the posterior mean always be expressed as a weighted sum of the maximum likelihood estimate and the prior mean (after choosing some appropriate prior)?
- 356
5
votes
1
answer
1k
views
Interpreting prior and posterior
I am bit puzzled on how we can interpret the posterior. Assume a coin which is 0.1 probable to be unfair. So our prior probability on the coin being unfair is 0.1, and being fair is 0.9. Also by ...
- 1,432
4
votes
2
answers
459
views
How does Prior Variance Affect Discrepancy between MLE and Posterior Expectation
Suppose that $\theta\in R$ is a parameter of interest, $p(\theta)$ is our prior belief regarding $\theta$, and $\hat \theta$ is the MLE for theta derived from the data $x$. It is my understanding that ...
- 1,006
4
votes
1
answer
3k
views
Conjugate priors for Gamma distribution of unknown $\alpha$ and $\beta$
Per the Wikipedia on conjugate priors link, the conjugate prior for a Gamma of unknown $\alpha$ and $\beta$ is proportional to an expression involving both $\alpha$ and $\beta$ as well as a $\Gamma$ ...
- 305
3
votes
1
answer
2k
views
Combining multiple posterior distributions
I am new to Bayesian statistics, and thus have problems to come up with a solution for the following problem:
Using Approximate Bayesian Computation (ABC), I generate a posterior distribution from ...
- 133
2
votes
3
answers
770
views
Where is wrong with my formulation of estimating the probability of a biased coin?
After a year and a half I've realized that the question is very ill posed and the notations are confusing and misleading, I tried to fix it but it seems irredeemable. Actually it took myself half an ...
- 15.9k
2
votes
1
answer
5k
views
Show posterior mean can be written as a weighted average of the prior mean and MLE
Suppose $Y_1, \dots Y_n$ are exponentially distributed: $Y_i | \lambda \sim Exp(\lambda)$.
Find the conjugate prior for $\lambda$, and the corresponding posterior distribution.
Show that the posterior ...
- 43
1
vote
0
answers
281
views
Classification of Bayesian posterior probabilities
I have run a series of Bayesian models with flat priors in which I obtain a posterior probability distribution for my coefficient of interest. The reviewer of my paper wishes us to classify these ...
- 21
1
vote
1
answer
176
views
Is this posterior probability integral right?
From Wiki:
where
, k is binomially distributed, and I'm not sure about u.
I'm thinking that the second line should be:
I mean, if we let X represent the toss of a die, then $P(X = 1, 2, 3, 4, 5)...
- 2,394
27
votes
8
answers
7k
views
Posterior very different to prior and likelihood
If the prior and the likelihood are very different from each other, then sometimes a situation occurs where the posterior is similar to neither of them. See for example this picture, which uses normal ...
- 661
23
votes
2
answers
3k
views
What is/are the implicit priors in frequentist statistics?
I've heard the notion that Jaynes claims frequentists operate with an "implicit prior".
What is or are these implicit priors? Does this mean frequentist models are all special cases of Bayesian ...
- 235
20
votes
2
answers
5k
views
How to apply Bayes' theorem to the search for a fisherman lost at sea
The article The Odds, Continually Updated mentions the story of a Long Island fisherman who literally owes his life to Bayesian Statistics. Here's the short version:
There are two fishermen on a ...
- 493
15
votes
4
answers
3k
views
Why is the normalisation constant in Bayesian not a marginal distribution
The formula for Baye's rule is as follows $$p(\theta |D) = \frac{p(D|\theta)p(\theta)}{\int p(D|\theta)p(\theta)d\theta}$$
where $\int p(D|\theta)p(\theta)d\theta$ is the normalising constant $z$. ...
- 1,116
14
votes
4
answers
2k
views
Are MCMC based methods appropriate when Maximum a-posteriori estimation is available?
I have been noticing that in many practical applications, MCMC-based methods are used to estimate a parameter even though the posterior is analytical (for example because the priors were conjugate). ...
- 273
13
votes
1
answer
1k
views
When can't frequentist sampling distribution be interpreted as Bayesian posterior in regression settings?
My actual questions are in the last two paragraphs, but to motivate them:
If I am attempting to estimate the mean of a random variable that follows a Normal distribution with a known variance, I've ...
- 483
12
votes
1
answer
20k
views
How does the beta prior affect the posterior under a binomial likelihood
I have two questions,
Question 1: How can I show that the posterior distribution is a beta distribution if the likelihood is binomial and the prior is a beta
Question 2: How does choices the prior ...
- 123
12
votes
2
answers
10k
views
Evaluate posterior predictive distribution in Bayesian linear regression
I'm confused on how to evaluate the posterior predictive distribution for Bayesian linear regression, past the basic case described here on page 3, and copied below.
$$ p(\tilde y \mid y) = \int p(\...
- 2,801
11
votes
1
answer
8k
views
Example of maximum a posteriori estimation
I have been reading about maximum likelihood estimation and maximum a posteriori estimation and so far I have met concrete examples only with maximum likelihood estimation. I have found some abstract ...
- 5,727
10
votes
3
answers
8k
views
How to interpret Bayesian (posterior predictive) p-value of 0.5?
In the following paper found here and reference below, the author suggests that "if the model is true or close to true, the posterior predictive p-value will almost certainly be very close to 0.5" . ...
- 309
9
votes
1
answer
5k
views
MAP estimation as regularisation of MLE
Going through the Wikipedia article on Maximum a posteriori estimation, it got confusing after reading this:
It is closely related to the method of maximum likelihood (ML) estimation, but employs ...
- 1,039
8
votes
1
answer
4k
views
Gibbs sampling to produce posterior pdf
Suppose we have the following classical normal linear regression model:
$$y_i = \beta_1 x_{1i} + \beta_2x_{2i} + \beta_3x_{3i} + e_i$$
where $e_{i} \sim iid.N(0, \sigma^2)$ for all $i = 1, 2, \cdots,...
- 889
8
votes
1
answer
1k
views
Why not use Beta(1,1) as boundary avoiding prior on a transformed correlation parameter?
In Bayesian Data Analysis, chapter 13, page 317, second full paragraph, in the modal and distributional approximations, Gelman et al. write:
If the plan is to summarize inference by the posterior ...
- 3,813
7
votes
1
answer
2k
views
What is the difference between 'Laplace approximation' and 'Modified harmonic mean'?
this question is about Bayesian and computational statistics. I am learning them right now, I have two very common output from my software, one is Laplace approximation and the other is Modified ...
- 6,209