Finding posterior probability mass function of binomial parameter

Question

Question:

Suppose a lot containing 1000 items is received from a supplier containing parameter (unknown) defective items. The past experiences with this supplier suggest that 5% of items in a lot are defective. Suppose we are told that each item he produces has probability 0.05 of being defective and defectives occur independently.

The natural option would be to use a conjugate prior to use for $\theta$ parameter is a beta distribution $$\theta \sim Beta(\alpha, \beta)$$ where $\alpha, \beta$ could be set to model so that $$E [\theta] = \frac{5}{100}=5\%$$ so that the prior reflects the expected number of defects.

Suppose we select a random sample of 10 items from this lot and let X be the number of defective item in the sample: Find the posterior probability mass function of the parameter.

What I know:

As posterior probability is required, I assume it is related to Bayes theorem as Bayes theorem holds the concept of posterior and priors.

I know the Bayes theorem, but cannot formulate the problem accordingly.

Any kind of assistance regarding this problem would be of great help.

Thank You.

Answer 1

As a prior distribution on the probability $\theta$ of an item from this supplier being defective, you might use $\theta \sim \mathsf{Beta}(\alpha_0=1,\beta_0=19),$ with density function

$$f(\theta) \propto \theta^{\alpha_0-1}(1-\theta)^{\beta_0-1},$$

where the symbol $\propto$ (read as "proportional to") indicates that we have omitted the norming constant of the density function. This distribution has $E(\theta) = \frac{\alpha_0}{\alpha_0+\beta_0} = \frac{1}{20} = 0.05$ and has $P(\theta < 0.1) \approx 0.86$ and $P(0.0013 < \theta < 0.1765) = 0.95.$ In R:

pbeta(0.1, 1, 19)
[1] 0.8649148
qbeta(c(.025,.975), 1, 19)
[1] 0.001331629 0.176466912

As @ChristopHanck has said, there are many beta distributions that would give $E(\theta) = 0.05.$ For example, if you feel more sure about $\theta \approx 0.05,$ then you could choose $\alpha_0$ and $\beta_0$ larger and in about the same ratio. In particular, the distribution $\mathsf{Beta}(5,95)$ has $E(\theta) = 0.05,$ but $P(0.02,0.09)\approx 0.95.$ However, that may represent a stronger opinion about $\theta \approx 0.05$ than you really have, based on past experience with the supplier. Also, such a 'highly informative' prior distribution will have a very strong influence on the posterior distribution and the conclusions we may draw from it.

    qbeta(c(.05,.95), 5,95)
    [1] 0.02010876 0.09007356

Now suppose you take a random sample of $n = 10$ items from the lot at hand and observe $x = 1$ defective. The resulting binomial likelihood function is $$g(x|\theta) \propto \theta^x(1-\theta)^{n-x} = \theta(1-\theta)^9.$$

Then, according to Bayes' Theorem, the posterior distribution has density

$$h(\theta|x) \propto f(\theta) \times g(x|\theta) \propto \theta^{\alpha_0-1}(1-\theta)^{\beta_0-1} \times \theta^x(1-\theta)^{n-x}\\ = \theta^{a_0+x-1}(1-\theta)^{\beta_0 +n-x -1} = \theta^{2-1}(1-\theta)^{28 - 1},$$

which we recognize as the 'kernel' (density without constant) of the distribution $\mathsf{Beta}(\alpha_n=2,\beta_n=28).$

In this case we have been able to find the the posterior distribution, without having to compute its norming constant, because the beta prior and binomial likelihood are 'conjugate' (mathematically compatible).

This particular posterior distribution has posterior mean $E(\theta|x) = \frac{2}{30} = 0.0667$ and a 95% Bayesian posterior interval estimate of $\theta$ is $(0.0085,0.1776).$

qbeta(c(.025,.975), 2,28)
[1] 0.008463962 0.177644295

Notes: (1) If we had used the stronger prior distribution mentioned above, then the posterior distribution would have been very little different from the prior distribution. (2) A frequentist Agresti-Coull 95% confidence interval for $\theta$ based only on one failure in a sample of ten is approximately $(0, 0.429).$