what does p( y | μ,σ²) really mean?

Question

Just started to study Bayesian Statistics. I am very confused the concept of having a conditional probability on a distribution. Specifically:

I understand what p( A | B ) where A="I am sick" and B = "Took a flu shot" means. (probability that I am sick, given that I am from the probability of the population that took a flue shot)

But what does p( y | μ,σ²) mean?

Can an example be provided? (Just the term "y given a normal distribution" won't help me understand it - I am too stupid for that :) )

Answer 1

This notation has little to do with Bayesian statistics. The object $$p(y|\mu,\sigma^2)$$ is a density for the random variable $Y$ taking values $y$; the second part "|μ,σ²" means that this density (I suppose this is a probability density (pdf), but it could be a cdf as well) is indexed by two parameters $\mu$ and $\sigma^2$, as for instance in the Normal density $\text{N}(\mu,\sigma^2))$. Changing $(\mu,\sigma^2))$ modifies the density function. One could have used $$p(y;\mu,\sigma^2)\quad\text{or}\quad p_{\mu,\sigma^2}(y)$$instead. (Incidentally the bar "|" notation was introduced by Harold Jeffreys in the 30's, the same influential Bayesian Jeffreys as in Jeffreys' prior.)

When $\mu$ and $\sigma^2$ turn into random variables, as in Bayesian statistics, this becomes a conditional density of a random variable $Y$ with realisation $y$ given the random vector $(\mu,\sigma^2)$. If the concept of conditional density is new to you, you should first check an introductory probability book or just the first chapters of Casella and Berger for instance.

Answer 2

Since you do not write the context, but only something about normal distribution, you let the readers guess, what is exactly written in the text. Yous hould add the context.

I guess $p(y|\mu,\sigma^2)$ means a probability density of a normal distribution with mean $\mu$ and standard deviation $\sigma$. This is a function:

$$p(y|\mu,\sigma^2)= (2\pi)^{-1/2}(\sigma)^{-1} \exp(-(y-\mu)^2/(2\sigma^2))$$

Answer 3

Short answer:

It's really a shorthand way of writing things. If you have three distributions, $Y, \mu, \sigma^2$, $p(y|\mu, \sigma^2)$ is essentially just $p_Y(y|\mu = \mu_0, \sigma^2 = \sigma_0^2)$ (i.e. you're evaluating the density of $Y$, at the constant values of the distributions of $\mu, \sigma^2$)

---

Long answer:

Let's say that you've got some data $y_i$ - some i.i.d. samples from some distribution. Let's say (assume) that the distribution of these data points is a normal distribution.

The normal distribution has some parameters associated with it that change the location and scale of the normal "bell" curve. You would want to find out what those parameters are, using your data.

So, you've essentially got this model:

$$Y_i \sim N(\mu, \sigma^2)$$

In Bayesian statistics, you treat the parameters $\mu, \sigma^2$ as random variables and place some prior distribution on them. The role of data here is to narrow down your uncertainty regarding your parameters.

Here's where the posterior distribution comes in. As an example, let's pretend we know what $\sigma^2$ is, and just concentrate on $\mu$. If we let the prior distribution to be $p_{\mu}(\mu_0)$, i.e. the density of $\mu$ evaluated at constant $\mu_0$, then the posterior is usually expressed as:

$$p(\mu|y) \propto p(y|\mu) p(\mu)$$

... (as you've got it) is just a shorthand way of writing:

$$ p_{\mu}(\mu_0 | Y = y) \propto p_Y(y|\mu = \mu_0) * p_{\mu}(\mu_0) $$