I have the following example:
Assume that we have an observation $Y$ from a Binomial distribution with parameter $n = 20$ and success probability $p: [Y \sim \mathrm{Bin}(20, p)]$.
Further assume that we observed $y = 12$ and want to make inference about the value of $p$ using Zellner's prior, i.e., we want to use the following PDF as prior for $p$: $[f(p) = c p^p (1-p)^{(1-p)}]$, where $c$ is such that $\int_0^1 f(p) \ dp = 1$. We can determine this constant by numeric integration in R:
integrand <- function(p) p^p * (1-p)^(1-p) res <- integrate(integrand, lower=0, upper=1) res (c <- 1/res$val)
It then continues as follows:
Given that we have observed $y = 12$, the likelihood $L(p \mid y)$ is $[ L(p \mid y) = \binom{20}{12} p^{12} (1 - p)^8]$. The posterior of $p$ is $[f(p \mid y) \propto L(p \mid y) f(p) = \binom{20}{12} p^{12} (1 - p)^8 c p^p (1 - p)^{(1 - p)}]$.
Whence $[f(p \mid y) \propto p^{12 + p} (1 - p)^{9 - p}]$.
Again, we could find the normalising constant for this density by numerical intergration:
integrand <- function(p) p^(12+p) * (1-p)^(9-p) res1 <- integrate(integrand, lower=0, upper=1) res1 (c1 <- 1/res1$val) > > We could also use numerical integration to calculate the posterior > mean and posterior variance of $p$. The following calculates > $\mathbb{E}[p \mid y]$, the expectation of the posterior distribution: > > integrand <- function(p) c1 * p * p^(12+p) * (1-p)^(9-p) > (resEp <- integrate(integrand, lower=0, upper=1)) > Ep <- resEp$val
Now we calculate $\mathbb{E}[p^2 \mid y]$, from which we can determine $\mathrm{Var}[p \mid y] = \mathbb{E}[p^2 \mid y]- (\mathbb{E}[p \mid y])^2$, the variance of the posterior distribution:
integrand <- function(p) c1 * p^2 * p^(12+p) * (1-p)^(9-p) (resEp2 <- integrate(integrand, lower=0, upper=1)) Ep2 <- resEp2$val (Varp <- Ep2 - Ep*Ep)
I have only just started learning Bayesian probability, so please be patient with me.
I understand that the $p^{12 + p} \cdot (1 - p)^{9 - p}$ is $f(p \mid y)$ -- the posterior of $p$.
I understand that, in accordance with the formula for the expected value of a continuous random variable $E[X] = \int_\mathbb{R} xf(x)$, we multiply $f(p \mid y)$, which is essentially the density function $f(x)$ in this case, by $p$, which is essentially $x$, the realisations/outcomes/observations of our random variable, since $p$ is the random variable we are trying to find a posterior distribution for, as is in accordance with the Bayesian methodology.
I understand that the normalising constant is used to reduce any probability function to a probability density function between $0$ and $1$.
However, I don't understand the following:
- Why is the normalising constant needed here (in general)?
- Why do we need to multiply by the normalising constant in the formula for expectation? It has that $\mathbb{E}[p \mid y] = c_1 \cdot p \cdot p^{12 + p} \cdot (1 - p)^{9 - p}$, and, as stated above, I think I understand all of it except for the normalising constant.
I would greatly appreciate it if people could please take take the time to clarify this.