8
$\begingroup$

When talking about events, there is the following formula called Bayes' rule, where $A$ and $B$ are random events: $$P(A|B)=\frac{P(B|A)P(A)}{P(B)}$$

Now let's say that for now only $A$ happened. I suppose that $P(B|A)$ is called posterior probability (beside conditional probability) since there is an evidence, $A$.

But is $P(A|B)$ also called posterior probability?

Edit: An (artificial) example:

$A$ = "Last year I had no accidents."

$B$ = "This year I will have no accidents."

Edit 2:

Simply put: $P(\text{something}|\text{evidence})$ = posterior probability

But $P(\text{evidence}|\text{something})$ = posterior?

Doesn't "posterior" have something to do with the chronology of events (their order of happening in time)?

$\endgroup$
3
  • $\begingroup$ Posterior is a conditional probability. Could you try making your question more clear? What are $A$ and $B$? Maybe you could illustrate it with an example? $\endgroup$
    – Tim
    Commented May 22, 2018 at 9:32
  • $\begingroup$ $A$ and $B$ are random events. $\endgroup$
    – user_anon
    Commented May 22, 2018 at 9:37
  • $\begingroup$ What my question was about: P(something|evidence) = posterior pb. But P(evidence|something) = posterior? $\endgroup$
    – user_anon
    Commented May 22, 2018 at 9:39

1 Answer 1

17
$\begingroup$

TLDR; Posterior probability is just the conditional probability that is outputted by the Bayes theorem. There is nothing special about it, it does not differ anyhow from any other conditional probability, it just has it's own name.

The Bayes theorem is about obtaining one conditional probability $P(A|B)$, given another one $P(B|A)$ and the prior $P(A)$,

$$ \underbrace{P(A|B)}_\text{posterior}=\frac{P(B|A)\,\overbrace{P(A)}^\text{prior}}{P(B)} $$

So in the equation we have two random variables $A$ and $B$ and their conditional and marginal probabilities, that's all. Prior $P(A)$ is the probability of $A$ "before" learning about $B$, while posterior $P(A|B)$ is the probability of $A$ "after" learning about $B$, where the "before" and "after" refer to your procedure of calculating the probabilities, not any chronological order. The naming convention is that the left hand side is the posterior, while the prior appears in the right hand side part. Using Bayes theorem you can easily switch the sides back and forth (that's the point of the theorem). The usual use case is when you know only $P(B|A)$ and $P(A)$, but you don't know $P(A|B)$ and want to learn about it. Here you can find nice example of such usage of Bayes theorem.

The specific case is Bayesian inference, where we use Bayes theorem to learn about the distribution of the parameter of interest $\theta$ given the data $X$, i.e. obtain the posterior distribution $f(\theta|X)$. This is achieved by looking at the likelihood function (that looks at the "evidence" you gathered) and the prior (the distribution of $\theta$ that is assumed before looking at the data).

$$ \underbrace{f(\theta|X)}_\text{posterior}=\frac{\overbrace{f(X|\theta)}^\text{likelihood}\,\overbrace{f(\theta)}^\text{prior}}{\underbrace{f(X)}_\text{normalizing constant}} $$

The most basic example of such usage of Bayes theorem is the beta-binomial model, but Bayesian inference is not only about probabilities of coin tosses, for example you can use it to do regression analysis and many other, even much more complicated, models.

To comment on your last question:

Doesn't "posterior" have something to do with the chronology of events (their order of happening in time)?

Not at all. You can ask a valid probabilistic question is reverse time order, e.g. "What is the probability that it was raining in the morning given that the ground is wet?".

$\endgroup$
14
  • $\begingroup$ So, when not doing Bayesian inference, the posterior and prior "exist" only when talking about Bayes theorem? Without considering the Bayes theorem with $P(A|B)=\ldots$, I cannot state that $P(A|B)$ is a posterior probability, right? $\endgroup$
    – user_anon
    Commented May 22, 2018 at 11:02
  • 2
    $\begingroup$ @user_anon posterior probability is just the conditional probability that is outputted by the Bayes theorem. There is nothing special about it, it does not differ anyhow from any other conditional probability, it just has it's own name. $\endgroup$
    – Tim
    Commented May 22, 2018 at 11:19
  • 1
    $\begingroup$ @user_anon It's the same since $P(A, B) = P(B|A)P(A)$ see en.wikipedia.org/wiki/Conditional_probability $\endgroup$
    – Tim
    Commented May 22, 2018 at 11:26
  • 1
    $\begingroup$ @user_anon this happens when you chain together many Bayes theorem equations, but it's a different story and unrelated to your question. $\endgroup$
    – Tim
    Commented May 22, 2018 at 12:04
  • 1
    $\begingroup$ @user_anon Because it doesn't. You can condition on time preceding event but do not have to. You can calculate probability that it was raining given that the ground is wet now. $\endgroup$
    – Tim
    Commented Aug 13, 2018 at 15:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.