What is the meaning of the notation $\theta \sim p(\theta)$ and $y \sim p(y|\theta)$?

Question

I've seen this notation pop up in some Bayesian statistics notes but despite my best efforts with Google I've not had any luck discovering the meaning. The only helpful context that I have is that $y$ is the observed data and $\theta$ is some sort of parameter. As for giving an example of this notation in use, I've seen De Finetti's theorem summarized as "$Y_1 , ... , Y_n | \theta$ are i.i.d. and $\theta \sim p(\theta)$ if and only if $Y_1 , ... , Y_n$ are exchangeable for all $n$".

EDIT: Note, I'm not asking how something like $Y \sim N(\mu, \sigma^2)$ is read, I know exactly what that means. It's the $p$ parts that are confusing me.

Answer 1

The notation is overloaded (or abused) so $p$ refers both to the probability density function as well as to the distribution. This seems a lot simpler than to have say capital letters indicate the distribution and lower case letters indicate the pdf/pmf.

Alternatively the $\sim$ notation can be thought of as overloaded so that it means the random variable on the left has the distribution on the right if the quantity on the right is a distribution. But if the quantity on the right is a pdf/pmf then $\sim$ means that the random variable on the left has the pdf/pmf of the quantity on the right.

Answer 2

The notation is very misleading with $p(\cdot)$ being heavily overloaded with multiple meanings and complete disregard for the difference between random variables and the values that they take on.

There is a parameter $\theta$ of unknown value which is modeled as a random variable $\Theta$, discrete or continuous depending on what $\theta$ is believed to be. Assuming that $\theta$ is believed to take on discrete values, $$\Theta \sim p_\Theta(\theta)\tag{1}$$ is saying that $\Theta$ is being modeled as a discrete random variable with probability mass function $p_\Theta(\theta)$, or to paint refined gold and gild the lily, the function $p_\Theta(\cdot)$ has the property that $$P\{\Theta = \theta_i\} = p_\Theta(\theta_i).$$ There are $n$ observations $x_1, x_2, \ldots, x_n$ which are also modeled as random variables $X_1$, $X_2$, $\ldots$, $X_n$. Assuming that everything is discrete, on a trial of the experiment, these random variables $\Theta, X_1, X_2, \ldots, X_n$ have values $\theta, x_1, x_2, \ldots, x_n$ of which we can observe only $x_1, x_2, \ldots, x_n$. Now, the conditional probability mass function of the $X_i$ given that $\Theta$ has value $\theta$ is $$p_{X_1, X_2, \ldots, Xn \mid \Theta = \theta}(x_1, x_2, \ldots, x_n \mid \Theta = \theta)\tag{2}$$ meaning that $$P\big\{X_1 = x_1, X_2 = x_2, \ldots, X_n = x_n \mid \Theta = \theta_i \big\} = p_{X_1, X_2, \ldots, Xn \mid \Theta = \theta_i}(x_1, x_2, \ldots, x_n \mid \Theta = \theta_i).$$ Notice that $p$ has different subscripts in $(1)$ and $(2)$ to emphasize that there are two different functions being used here; one with just one argument and another with $n$ arguments, a far better arrangement than $p$ meaning different things depending on where it occurs and also better than the "canonical"

Let $X \sim f(x)$ and $Y \sim f(y)$ be random variables $\ldots$

or the physicists'

$X \sim f(X), Y \sim f(Y), X,Y \sim f(X,Y)$

where the density of $X$ can be quite different from the density of $Y$ and one is supposed to figure out from the letter used as the argument of $f\cdot)$ as to whether the density function of $x$ is meant or the density function of $Y$.