# What is the Bayesian Prior Predictive distribution from two normal populations?

The question goes as follows:

A shoe factory produces brown shoes and black shoes. They look the same but differ only in their weight characteristics. Brown shoes have their weight distributed as Normal$$(\mu = 7 \space\text{lbs}, \sigma^2)$$ and black shoes as Normal$$(\mu = 8 \space\text{lbs}, \sigma^2)$$. They are produced in equal proportions and the black shoes are, on average, 1 lbs heavier than brown shoes.

a) Write an equation for the prior predictive distribution of the weight of a randomly selected pair of shoes.

b) You are provided a shoe, and find that it weighs 7 lbs. Find the conditional probability that it's a brown shoe, when $$\sigma = 1$$.

c) Explain how the probability that a 7 lbs shoe is brown will vary as $$\sigma$$ increases and decreases.

I am mostly interested in parts (a) and (b), as I think for (c), the answer is that as $$\sigma$$ goes down, the probability that a shoe is brown will increase and vice versa.

For (a), my initial thoughts are that it would be $$\text{Normal}(\mu = 7.5, 2\sigma^2)$$ as I think the variances will add. However, I'm not quite sure of whether this is the correct mean.

Fort part (b), I'm really confused and would appreciate any detailed help. Many thanks!

For these tasks I always try to figure out the common probability density/distribution first because the answers to all the succeeding questions can be easily answered using this single common distribution.

For every of the single shoes produced by the family we have a pair of random variables, namely $$T_i$$ (the type of the shoe $$i$$) and $$S_i$$ (the size of the shoe $$i$$). Let us rename 'brown' to 'small' and 'black' to 'large', i.e. the random variable $$T_i$$ takes values in the set $$\{\text{small}, \text{large}\}$$ and the variable $$S_i$$ takes values in $$\mathbb{R}$$ (since it is normally distributed). Notice that this is unrealistic because the normal distribution will also assign probability mass (i.e. $$\neq 0$$) to intervals in the negative numbers (i.e. it is unlikely but possible that shoes have a negative size) but this is how the task is formulated so we stick to it.

Let us assume that there is only one single shoe, i.e. we deal with the random variables $$T, S$$. What they mean ba saying that 'Brown shoes have their weight distributed as ...' is that they actually want to tell you how the density $$p(s|t)$$ looks like, namely \begin{align*} p(s|\text{small}) &= \mathcal{N}(7, 1)(s) \\ p(s|\text{large}) &= \mathcal{N}(8, 1)(s) \end{align*} (notice that $$\mathcal{N}(\mu, \sigma^2)$$ is an ordinary deterministic function, namely the density function as given on wikipedia (see pdf on the right). Furthermore they tell you that $$P[T=\text{small}] = P[T=\text{large}] = 0.5$$

Now let us compute the total probability: \begin{align*} p(s,t) &= p(s|t) \cdot p(t) = \begin{cases} \frac{1}{2}\mathcal{N}(7, 1)(s) & \text{if t=\text{small}} \\ \frac{1}{2}\mathcal{N}(8, 1)(s) & \text{if t=\text{large}}\end{cases} \\ &= \frac{1}{2} \left(\mathbf{1}_{t=\text{small}}\mathcal{N}(7, 1)(s) + \mathbf{1}_{t=\text{large}}\mathcal{N}(8, 1)(s) \right) \end{align*}

I think that (a) asks you to write down an expression for $$p(s)$$. Notice that if you have two random variables $$X,Y$$ with a common density $$p(x,y)$$ then the density of a single one of them (say $$Y$$) can be computed by something called 'marginalization', i.e. $$p(y) = \sum_{x} p(x,y)$$

(b) asks you to write down an expression for $$p(\text{small} | 7) = p(t|s)$$ for $$t=\text{small}, s = 7$$. Notice that one can say a lot about the sum of normally distributed random variables if they are independent (which they are in this case), see wikipedia.

If you treat $$\sigma^2=1$$ as a variable in (b) instead of the fixed value $$1$$ then in (b) you will get an expression that still depends on one thing: $$\sigma$$. Your job now is to draw a curve $$\sigma \mapsto \text{formula that you got in (b) for \sigma}$$. Intuitively you should expect that for a small $$\sigma$$ (i.e. the size small or brown shoes is very tightly distributed around $$7$$) then if you actually observe $$7$$, the probability or the confidence of the ''model'' should be very high that the type is brown (or small as we call it). However, the more loosely the size is distributed, the bigger the effect of the big (black) shoes will come into play and the model gets more and more inconfident that it actually is a pair of small shoes although you have observed the perfect mean value for their distribution.