Podcast #128: We chat with Kent C Dodds about why he loves React and discuss what life was like in the dark days before Git. Listen now.

 6 added 5 characters in body edited May 22 '18 at 11:38 Tim♦ 71.4k1414 gold badges152152 silver badges254254 bronze badges TLDR; Posterior probability is just the conditional probability that is outputted by the Bayes theorem. There is nothing special about it, it does not differ anyhow from any other conditional probability, it just has it's own name. The Bayes theorem is about obtaining one conditional probability $$P(A|B)$$, given another one $$P(B|A)$$ and the prior $$P(A)$$, $$\underbrace{P(A|B)}_\text{posterior}=\frac{P(B|A)\,\overbrace{P(A)}^\text{prior}}{P(B)}$$ So in the equation we have two random variables $$A$$ and $$B$$ and their conditional and marginal probabilities, that's all. Prior $$P(A)$$ is the probability of $$A$$ "before" learning about $$B$$, while posterior $$P(A|B)$$ is the probability of $$A$$ "after" learning about $$B$$, where the "before" and "after" refer to your procedure of calculating the probabilities, not any chronological order. The naming convention is that the left hand side is the posterior, while the prior appears in the right hand side part. Using Bayes theorem you can easily switch the sides back and forth (that's the point of the theorem). The usual use case is when you know only $$P(B|A)$$ and $$P(A)$$, but you don't know $$P(A|B)$$ and want to learn about it. Here you can find nice example of such usage of Bayes theorem. The specific case is Bayesian inference, where we use Bayes theorem to learn about the distribution of the parameter of interest $$\theta$$ given the data $$X$$, i.e. obtain the posterior distribution $$f(\theta|X)$$. This is achieved by looking at the likelihood function (that looks at the "evidence" you gathered) and the prior (the distribution of $$\theta$$ that is assumed before looking at the data). $$\underbrace{f(\theta|X)}_\text{posterior}=\frac{\overbrace{f(X|\theta)}^\text{likelihood}\,\overbrace{f(\theta)}^\text{prior}}{\underbrace{f(X)}_\text{normalizing constant}}$$ The most basic example of such usage of Bayes theorem is the beta-binomial model, but Bayesian inference is not only about probabilities of coin tosses, for example you can use it to do regression analysis and many other, even much more complicated, models. To comment on your last question: Doesn't "posterior" have something to do with the chronology of events (their order of happening in time)? Not at all. You can ask a valid probabilistic question is reverse time order, e.g. "What is the probability that it was raining in the morning given that the ground is wet?". TLDR; Posterior probability is just the conditional probability that is outputted by the Bayes theorem. There is nothing special about it, it does not differ anyhow from any other conditional probability, it just has it's own name. The Bayes theorem is about obtaining one conditional probability $$P(A|B)$$, given another one $$P(B|A)$$ and the prior $$P(A)$$, $$\underbrace{P(A|B)}_\text{posterior}=\frac{P(B|A)\,\overbrace{P(A)}^\text{prior}}{P(B)}$$ So in the equation we have two random variables $$A$$ and $$B$$ and their conditional and marginal probabilities, that's all. Prior $$P(A)$$ is the probability of $$A$$ "before" learning about $$B$$, while posterior $$P(A|B)$$ is the probability of $$A$$ "after" learning about $$B$$, where the "before" and "after" refer to your procedure of calculating the probabilities, not any chronological order. The naming convention is that the left hand side is the posterior, while the prior appears in the right hand side part. Using Bayes theorem you can easily switch the sides back and forth (that's the point of the theorem). The usual use case is when you know $$P(B|A)$$ and $$P(A)$$, but you don't know $$P(A|B)$$ and want to learn about it. Here you can find nice example of such usage of Bayes theorem. The specific case is Bayesian inference, where we use Bayes theorem to learn about the distribution of the parameter of interest $$\theta$$ given the data $$X$$, i.e. obtain the posterior distribution $$f(\theta|X)$$. This is achieved by looking at the likelihood function (that looks at the "evidence" you gathered) and the prior (the distribution of $$\theta$$ that is assumed before looking at the data). $$\underbrace{f(\theta|X)}_\text{posterior}=\frac{\overbrace{f(X|\theta)}^\text{likelihood}\,\overbrace{f(\theta)}^\text{prior}}{\underbrace{f(X)}_\text{normalizing constant}}$$ The most basic example of such usage of Bayes theorem is the beta-binomial model, but Bayesian inference is not only about probabilities of coin tosses, for example you can use it to do regression analysis and many other, even much more complicated, models. To comment on your last question: Doesn't "posterior" have something to do with the chronology of events (their order of happening in time)? Not at all. You can ask a valid probabilistic question is reverse time order, e.g. "What is the probability that it was raining in the morning given that the ground is wet?". TLDR; Posterior probability is just the conditional probability that is outputted by the Bayes theorem. There is nothing special about it, it does not differ anyhow from any other conditional probability, it just has it's own name. The Bayes theorem is about obtaining one conditional probability $$P(A|B)$$, given another one $$P(B|A)$$ and the prior $$P(A)$$, $$\underbrace{P(A|B)}_\text{posterior}=\frac{P(B|A)\,\overbrace{P(A)}^\text{prior}}{P(B)}$$ So in the equation we have two random variables $$A$$ and $$B$$ and their conditional and marginal probabilities, that's all. Prior $$P(A)$$ is the probability of $$A$$ "before" learning about $$B$$, while posterior $$P(A|B)$$ is the probability of $$A$$ "after" learning about $$B$$, where the "before" and "after" refer to your procedure of calculating the probabilities, not any chronological order. The naming convention is that the left hand side is the posterior, while the prior appears in the right hand side part. Using Bayes theorem you can easily switch the sides back and forth (that's the point of the theorem). The usual use case is when you know only $$P(B|A)$$ and $$P(A)$$, but you don't know $$P(A|B)$$ and want to learn about it. Here you can find nice example of such usage of Bayes theorem. The specific case is Bayesian inference, where we use Bayes theorem to learn about the distribution of the parameter of interest $$\theta$$ given the data $$X$$, i.e. obtain the posterior distribution $$f(\theta|X)$$. This is achieved by looking at the likelihood function (that looks at the "evidence" you gathered) and the prior (the distribution of $$\theta$$ that is assumed before looking at the data). $$\underbrace{f(\theta|X)}_\text{posterior}=\frac{\overbrace{f(X|\theta)}^\text{likelihood}\,\overbrace{f(\theta)}^\text{prior}}{\underbrace{f(X)}_\text{normalizing constant}}$$ The most basic example of such usage of Bayes theorem is the beta-binomial model, but Bayesian inference is not only about probabilities of coin tosses, for example you can use it to do regression analysis and many other, even much more complicated, models. To comment on your last question: Doesn't "posterior" have something to do with the chronology of events (their order of happening in time)? Not at all. You can ask a valid probabilistic question is reverse time order, e.g. "What is the probability that it was raining in the morning given that the ground is wet?". 5 added 127 characters in body edited May 22 '18 at 11:25 Tim♦ 71.4k1414 gold badges152152 silver badges254254 bronze badges First thing to notice,TLDR; Posterior probability is just the factconditional probability that you useis outputted by the Bayes theorem. There is nothing special about it, it does not make the calculation Bayesiandiffer anyhow from any other conditional probability, it just has it's own name.  The Bayes theorem is a theorem about obtaining one conditional probability $$P(A|B)$$, given another one $$P(B|A)$$ and athe prior $$P(A)$$, $$\underbrace{P(A|B)}_\text{posterior}=\frac{P(B|A)\,\overbrace{P(A)}^\text{prior}}{P(B)}$$ So in the equation we have two random variables $$A$$ and $$B$$ and their conditional and marginal probabilities, that's all. Prior $$P(A)$$ is the probability of $$A$$ "before" learning about $$B$$, while posterior $$P(A|B)$$ is the probability of $$A$$ "after" learning about $$B$$, where the "before" and "after" refer to your procedure of calculating the probabilities, not any chronological order. The naming convention is that the left hand side is the posterior, while the prior appears in the right hand side part. Using Bayes theorem you can easily switch the sides back and forth (that's the point of the theorem). The usual use case is when you know $$P(B|A)$$ and $$P(A)$$, but you don't know $$P(A|B)$$ and want to learn about it. Here you can find nice example of such usage of Bayes theorem. The specific case is Bayesian inference, where we use Bayes theorem to learn about the distribution of the parameter of interest $$\theta$$ given the data $$X$$, i.e. obtain the posterior distribution $$f(\theta|X)$$. This is achieved by looking at the likelihood function (that looks at the "evidence" you gathered) and the prior (the distribution of $$\theta$$ that is assumed before looking at the data). $$\underbrace{f(\theta|X)}_\text{posterior}=\frac{\overbrace{f(X|\theta)}^\text{likelihood}\,\overbrace{f(\theta)}^\text{prior}}{\underbrace{f(X)}_\text{normalizing constant}}$$ The most basic example of such usage of Bayes theorem is the beta-binomial model, but Bayesian inference is not only about probabilities of coin tosses, for example you can use it to do regression analysis and many other, even much more complicated, models. To comment on your last question: Doesn't "posterior" have something to do with the chronology of events (their order of happening in time)? Not at all. You can ask a valid probabilistic question is reverse time order, e.g. "What is the probability that it was raining in the morning given that the ground is wet?". First thing to notice, is the fact that you use Bayes theorem does not make the calculation Bayesian. The Bayes theorem is a theorem about obtaining one conditional probability $$P(A|B)$$, given another one $$P(B|A)$$ and a prior $$P(A)$$, $$\underbrace{P(A|B)}_\text{posterior}=\frac{P(B|A)\,\overbrace{P(A)}^\text{prior}}{P(B)}$$ So in the equation we have two random variables $$A$$ and $$B$$ and their conditional and marginal probabilities, that's all. Prior $$P(A)$$ is the probability of $$A$$ "before" learning about $$B$$, while posterior $$P(A|B)$$ is the probability of $$A$$ "after" learning about $$B$$, where the "before" and "after" refer to your procedure of calculating the probabilities, not any chronological order. The naming convention is that the left hand side is the posterior, while the prior appears in the right hand side part. Using Bayes theorem you can easily switch the sides back and forth (that's the point of the theorem). The usual use case is when you know $$P(B|A)$$ and $$P(A)$$, but you don't know $$P(A|B)$$ and want to learn about it. Here you can find nice example of such usage of Bayes theorem. The specific case is Bayesian inference, where we use Bayes theorem to learn about the distribution of the parameter of interest $$\theta$$ given the data $$X$$, i.e. obtain the posterior distribution $$f(\theta|X)$$. This is achieved by looking at the likelihood function (that looks at the "evidence" you gathered) and the prior (the distribution of $$\theta$$ that is assumed before looking at the data). $$\underbrace{f(\theta|X)}_\text{posterior}=\frac{\overbrace{f(X|\theta)}^\text{likelihood}\,\overbrace{f(\theta)}^\text{prior}}{\underbrace{f(X)}_\text{normalizing constant}}$$ The most basic example of such usage of Bayes theorem is the beta-binomial model, but Bayesian inference is not only about probabilities of coin tosses, for example you can use it to do regression analysis and many other, even much more complicated, models. To comment on your last question: Doesn't "posterior" have something to do with the chronology of events (their order of happening in time)? Not at all. You can ask a valid probabilistic question is reverse time order, e.g. "What is the probability that it was raining in the morning given that the ground is wet?". TLDR; Posterior probability is just the conditional probability that is outputted by the Bayes theorem. There is nothing special about it, it does not differ anyhow from any other conditional probability, it just has it's own name.  The Bayes theorem is about obtaining one conditional probability $$P(A|B)$$, given another one $$P(B|A)$$ and the prior $$P(A)$$, $$\underbrace{P(A|B)}_\text{posterior}=\frac{P(B|A)\,\overbrace{P(A)}^\text{prior}}{P(B)}$$ So in the equation we have two random variables $$A$$ and $$B$$ and their conditional and marginal probabilities, that's all. Prior $$P(A)$$ is the probability of $$A$$ "before" learning about $$B$$, while posterior $$P(A|B)$$ is the probability of $$A$$ "after" learning about $$B$$, where the "before" and "after" refer to your procedure of calculating the probabilities, not any chronological order. The naming convention is that the left hand side is the posterior, while the prior appears in the right hand side part. Using Bayes theorem you can easily switch the sides back and forth (that's the point of the theorem). The usual use case is when you know $$P(B|A)$$ and $$P(A)$$, but you don't know $$P(A|B)$$ and want to learn about it. Here you can find nice example of such usage of Bayes theorem. The specific case is Bayesian inference, where we use Bayes theorem to learn about the distribution of the parameter of interest $$\theta$$ given the data $$X$$, i.e. obtain the posterior distribution $$f(\theta|X)$$. This is achieved by looking at the likelihood function (that looks at the "evidence" you gathered) and the prior (the distribution of $$\theta$$ that is assumed before looking at the data). $$\underbrace{f(\theta|X)}_\text{posterior}=\frac{\overbrace{f(X|\theta)}^\text{likelihood}\,\overbrace{f(\theta)}^\text{prior}}{\underbrace{f(X)}_\text{normalizing constant}}$$ The most basic example of such usage of Bayes theorem is the beta-binomial model, but Bayesian inference is not only about probabilities of coin tosses, for example you can use it to do regression analysis and many other, even much more complicated, models. To comment on your last question: Doesn't "posterior" have something to do with the chronology of events (their order of happening in time)? Not at all. You can ask a valid probabilistic question is reverse time order, e.g. "What is the probability that it was raining in the morning given that the ground is wet?". 4 deleted 2 characters in body edited May 22 '18 at 11:11 Tim♦ 71.4k1414 gold badges152152 silver badges254254 bronze badges First thing to notice, is the fact that you use Bayes theorem does not make the calculation Bayesian. The Bayes theorem is a theorem about obtaining one conditional probability $$P(A|B)$$, given another one $$P(B|A)$$ and a prior $$P(A)$$, $$\underbrace{P(A|B)}_\text{posterior}=\frac{P(B|A)\,\overbrace{P(A)}^\text{prior}}{P(B)}$$ So in the equation we have two random variables $$A$$ and $$B$$ and their conditional and marginal probabilities, that's all. Prior $$P(A)$$ is the probability of $$A$$ "before" learning about $$B$$, while posterior $$P(A|B)$$ is the probability of $$A$$ "after" learning about $$B$$, where the "before" and "after" refer to your procedure of calculating the probabilities, not any chronological order. The naming convention is that the left hand side is the posterior, while the prior appears in the right hand side part. Using Bayes theorem you can easily switch the sides back and forth (that's the point of the theorem). The usual use case is when you know $$P(B|A)$$ and $$P(A)$$, but you don't know $$P(A|B)$$ and want to learn about it. Here you can find nice example of such usage of Bayes theorem. The different storyspecific case is Bayesian inference, where we use Bayes theorem to learn about the distribution of the parameter of interest $$\theta$$ given the data $$X$$, i.e. obtain the posterior distribution $$f(\theta|X)$$. This is achieved by looking at the likelihood function (that looks at the "evidence" you gathered) and the prior (the distribution of $$\theta$$ that is assumed before looking at the data). $$\underbrace{f(\theta|X)}_\text{posterior}=\frac{\overbrace{f(X|\theta)}^\text{likelihood}\,\overbrace{f(\theta)}^\text{prior}}{\underbrace{f(X)}_\text{normalizing constant}}$$ The most basic example of such usage of Bayes theorem is the beta-binomial model, but Bayesian inference is not only about probabilities of coin tosses, for example you can use it to do regression analysis and many other, even much more complicated, models. To comment on your last question: Doesn't "posterior" have something to do with the chronology of events (their order of happening in time)? Not at all. You can ask a valid probabilistic question is reverse time order, e.g. "What is the probability that it was raining in the morning given that the ground is wet?". First thing to notice, is the fact that you use Bayes theorem does not make the calculation Bayesian. The Bayes theorem is a theorem about obtaining one conditional probability $$P(A|B)$$, given another one $$P(B|A)$$ and a prior $$P(A)$$, $$\underbrace{P(A|B)}_\text{posterior}=\frac{P(B|A)\,\overbrace{P(A)}^\text{prior}}{P(B)}$$ So in the equation we have two random variables $$A$$ and $$B$$ and their conditional and marginal probabilities, that's all. Prior $$P(A)$$ is the probability of $$A$$ "before" learning about $$B$$, while posterior $$P(A|B)$$ is the probability of $$A$$ "after" learning about $$B$$, where the "before" and "after" refer to your procedure of calculating the probabilities, not any chronological order. The naming convention is that the left hand side is the posterior, while the prior appears in the right hand side part. Using Bayes theorem you can easily switch the sides back and forth (that's the point of the theorem). The usual use case is when you know $$P(B|A)$$ and $$P(A)$$, but you don't know $$P(A|B)$$ and want to learn about it. Here you can find nice example of such usage of Bayes theorem. The different story is Bayesian inference, where we use Bayes theorem to learn about the distribution of the parameter of interest $$\theta$$ given the data $$X$$, i.e. obtain the posterior distribution $$f(\theta|X)$$. This is achieved by looking at the likelihood function (that looks at the "evidence" you gathered) and the prior (the distribution of $$\theta$$ that is assumed before looking at the data). $$\underbrace{f(\theta|X)}_\text{posterior}=\frac{\overbrace{f(X|\theta)}^\text{likelihood}\,\overbrace{f(\theta)}^\text{prior}}{\underbrace{f(X)}_\text{normalizing constant}}$$ The most basic example of such usage of Bayes theorem is the beta-binomial model, but Bayesian inference is not only about probabilities of coin tosses, for example you can use it to do regression analysis and many other, even much more complicated, models. To comment on your last question: Doesn't "posterior" have something to do with the chronology of events (their order of happening in time)? Not at all. You can ask a valid probabilistic question is reverse time order, e.g. "What is the probability that it was raining in the morning given that the ground is wet?". First thing to notice, is the fact that you use Bayes theorem does not make the calculation Bayesian. The Bayes theorem is a theorem about obtaining one conditional probability $$P(A|B)$$, given another one $$P(B|A)$$ and a prior $$P(A)$$, $$\underbrace{P(A|B)}_\text{posterior}=\frac{P(B|A)\,\overbrace{P(A)}^\text{prior}}{P(B)}$$ So in the equation we have two random variables $$A$$ and $$B$$ and their conditional and marginal probabilities, that's all. Prior $$P(A)$$ is the probability of $$A$$ "before" learning about $$B$$, while posterior $$P(A|B)$$ is the probability of $$A$$ "after" learning about $$B$$, where the "before" and "after" refer to your procedure of calculating the probabilities, not any chronological order. The naming convention is that the left hand side is the posterior, while the prior appears in the right hand side part. Using Bayes theorem you can easily switch the sides back and forth (that's the point of the theorem). The usual use case is when you know $$P(B|A)$$ and $$P(A)$$, but you don't know $$P(A|B)$$ and want to learn about it. Here you can find nice example of such usage of Bayes theorem. The specific case is Bayesian inference, where we use Bayes theorem to learn about the distribution of the parameter of interest $$\theta$$ given the data $$X$$, i.e. obtain the posterior distribution $$f(\theta|X)$$. This is achieved by looking at the likelihood function (that looks at the "evidence" you gathered) and the prior (the distribution of $$\theta$$ that is assumed before looking at the data). $$\underbrace{f(\theta|X)}_\text{posterior}=\frac{\overbrace{f(X|\theta)}^\text{likelihood}\,\overbrace{f(\theta)}^\text{prior}}{\underbrace{f(X)}_\text{normalizing constant}}$$ The most basic example of such usage of Bayes theorem is the beta-binomial model, but Bayesian inference is not only about probabilities of coin tosses, for example you can use it to do regression analysis and many other, even much more complicated, models. To comment on your last question: Doesn't "posterior" have something to do with the chronology of events (their order of happening in time)? Not at all. You can ask a valid probabilistic question is reverse time order, e.g. "What is the probability that it was raining in the morning given that the ground is wet?". 3 added 378 characters in body edited May 22 '18 at 10:14 Tim♦ 71.4k1414 gold badges152152 silver badges254254 bronze badges 2 added 627 characters in body edited May 22 '18 at 10:03 Tim♦ 71.4k1414 gold badges152152 silver badges254254 bronze badges 1 answered May 22 '18 at 9:51 Tim♦ 71.4k1414 gold badges152152 silver badges254254 bronze badges