Skip to main content
added 219 characters in body
Source Link
Tim
  • 141.1k
  • 26
  • 270
  • 512

No, the code is not correct. The likelihood function for "the event where 8 heads observed after 10 coin tossing" is the binomial distribution with $n=10$ and unknown probability of success $\theta$:

$$ y \sim \mathsf{Binom}(n=10,\, p=\theta) $$

this is your $p(y|\theta)$. Notice that this already is a distribution of all the tosses, not a single toss.

In your code you used binom.pmf(n=10,k=i,p=0.8), so you assumed (or knew) $p$ to be $0.8$. This is not a likelihood, but rather a distribution of $y$ when all the parameters are known. Alternatively, maybe you wanted to show likelihood function evaluated on $p=0.8$ value, then this is the case.

Likelihood is a distribution of the data given the parameter $\theta$. Prior is the distribution of the unknown parameter $\theta$ that is assumed a priori, before seeing the data. In your code the first is, incorrectly, also a binomial distribution. The most common, "textbook", prior for $\theta$ is the beta distribution:

$$ \theta \sim \mathsf{Beta}(\alpha, \beta) $$

this is $p(\theta)$. Notice that $\theta$ can be any real number in $[0, 1]$ interval (it is a probability), so the Bayes theorem is

$$ p(\theta|y) = \frac{p(y|\theta)\,p(\theta)}{\int_0^1 p(y|\theta)\,p(\theta)\, d\theta} $$

So in the denominator you cannot list all the possible values and sum, since there is infinite number of possible values for $\theta$. You need to take integral, that is why we use simulation-based approaches as MCMC, to approximate those integrals.

There is also a mathematical shortcut, since beta distribution is a conjugate prior for $p$ parameter of the binomial distribution, there is a closed-form solution and the posterior distribution is available right away as

$$ \theta|y \sim \mathsf{Beta}(\alpha+y,\,\beta+n-y) $$

this is $p(\theta|y)$.

No, the code is not correct. The likelihood function for "the event where 8 heads observed after 10 coin tossing" is the binomial distribution with $n=10$ and unknown probability of success $\theta$:

$$ y \sim \mathsf{Binom}(n=10,\, p=\theta) $$

this is your $p(y|\theta)$. Notice that this already is a distribution of all the tosses, not a single toss. Likelihood is a distribution of the data given the parameter $\theta$. Prior is the distribution of the unknown parameter $\theta$ that is assumed a priori, before seeing the data. In your code the first is, incorrectly, also a binomial distribution. The most common, "textbook", prior for $\theta$ is the beta distribution:

$$ \theta \sim \mathsf{Beta}(\alpha, \beta) $$

this is $p(\theta)$. Notice that $\theta$ can be any real number in $[0, 1]$ interval (it is a probability), so the Bayes theorem is

$$ p(\theta|y) = \frac{p(y|\theta)\,p(\theta)}{\int_0^1 p(y|\theta)\,p(\theta)\, d\theta} $$

So in the denominator you cannot list all the possible values and sum, since there is infinite number of possible values for $\theta$. You need to take integral, that is why we use simulation-based approaches as MCMC, to approximate those integrals.

There is also a mathematical shortcut, since beta distribution is a conjugate prior for $p$ parameter of the binomial distribution, there is a closed-form solution and the posterior distribution is available right away as

$$ \theta|y \sim \mathsf{Beta}(\alpha+y,\,\beta+n-y) $$

this is $p(\theta|y)$.

No, the code is not correct. The likelihood function for "the event where 8 heads observed after 10 coin tossing" is the binomial distribution with $n=10$ and unknown probability of success $\theta$:

$$ y \sim \mathsf{Binom}(n=10,\, p=\theta) $$

this is your $p(y|\theta)$. Notice that this already is a distribution of all the tosses, not a single toss.

In your code you used binom.pmf(n=10,k=i,p=0.8), so you assumed (or knew) $p$ to be $0.8$. This is not a likelihood, but rather a distribution of $y$ when all the parameters are known. Alternatively, maybe you wanted to show likelihood function evaluated on $p=0.8$ value, then this is the case.

Likelihood is a distribution of the data given the parameter $\theta$. Prior is the distribution of the unknown parameter $\theta$ that is assumed a priori, before seeing the data. In your code the first is, incorrectly, also a binomial distribution. The most common, "textbook", prior for $\theta$ is the beta distribution:

$$ \theta \sim \mathsf{Beta}(\alpha, \beta) $$

this is $p(\theta)$. Notice that $\theta$ can be any real number in $[0, 1]$ interval (it is a probability), so the Bayes theorem is

$$ p(\theta|y) = \frac{p(y|\theta)\,p(\theta)}{\int_0^1 p(y|\theta)\,p(\theta)\, d\theta} $$

So in the denominator you cannot list all the possible values and sum, since there is infinite number of possible values for $\theta$. You need to take integral, that is why we use simulation-based approaches as MCMC, to approximate those integrals.

There is also a mathematical shortcut, since beta distribution is a conjugate prior for $p$ parameter of the binomial distribution, there is a closed-form solution and the posterior distribution is available right away as

$$ \theta|y \sim \mathsf{Beta}(\alpha+y,\,\beta+n-y) $$

this is $p(\theta|y)$.

Source Link
Tim
  • 141.1k
  • 26
  • 270
  • 512

No, the code is not correct. The likelihood function for "the event where 8 heads observed after 10 coin tossing" is the binomial distribution with $n=10$ and unknown probability of success $\theta$:

$$ y \sim \mathsf{Binom}(n=10,\, p=\theta) $$

this is your $p(y|\theta)$. Notice that this already is a distribution of all the tosses, not a single toss. Likelihood is a distribution of the data given the parameter $\theta$. Prior is the distribution of the unknown parameter $\theta$ that is assumed a priori, before seeing the data. In your code the first is, incorrectly, also a binomial distribution. The most common, "textbook", prior for $\theta$ is the beta distribution:

$$ \theta \sim \mathsf{Beta}(\alpha, \beta) $$

this is $p(\theta)$. Notice that $\theta$ can be any real number in $[0, 1]$ interval (it is a probability), so the Bayes theorem is

$$ p(\theta|y) = \frac{p(y|\theta)\,p(\theta)}{\int_0^1 p(y|\theta)\,p(\theta)\, d\theta} $$

So in the denominator you cannot list all the possible values and sum, since there is infinite number of possible values for $\theta$. You need to take integral, that is why we use simulation-based approaches as MCMC, to approximate those integrals.

There is also a mathematical shortcut, since beta distribution is a conjugate prior for $p$ parameter of the binomial distribution, there is a closed-form solution and the posterior distribution is available right away as

$$ \theta|y \sim \mathsf{Beta}(\alpha+y,\,\beta+n-y) $$

this is $p(\theta|y)$.