6
$\begingroup$

I've seen this notation pop up in some Bayesian statistics notes but despite my best efforts with Google I've not had any luck discovering the meaning. The only helpful context that I have is that $y$ is the observed data and $\theta$ is some sort of parameter. As for giving an example of this notation in use, I've seen De Finetti's theorem summarized as "$Y_1 , ... , Y_n | \theta$ are i.i.d. and $\theta \sim p(\theta)$ if and only if $Y_1 , ... , Y_n$ are exchangeable for all $n$".

EDIT: Note, I'm not asking how something like $Y \sim N(\mu, \sigma^2)$ is read, I know exactly what that means. It's the $p$ parts that are confusing me.

$\endgroup$
7
  • $\begingroup$ I don't agree that the alleged duplicate question and its answers have anything to do with the question being asked here and vote to reopen this question. It is hard to believe that there are not already other questions on this site of which J.Min's question is a duplicate, but the one used by @Tim as a reason for closing this question is not a duplicate. $\endgroup$ Oct 21, 2017 at 16:10
  • $\begingroup$ Context, along with direct quotes, have already been given. As for what's confusing, I don't know how I could be more clear. As far as I can tell the statement "theta is distributed the probability density function of theta" simply doesn't make sense, nor does the $y$ example. $\endgroup$
    – J. Min
    Oct 21, 2017 at 16:12
  • $\begingroup$ "Probably" the answer you want is that in the context of Bayes theorem, $\endgroup$
    – meh
    Oct 21, 2017 at 16:12
  • $\begingroup$ @J.Min maybe you could link the paper where you've seen it? $\endgroup$
    – Tim
    Oct 21, 2017 at 16:16
  • 4
    $\begingroup$ @aginensky $\sim$ does not mean "proportional with unknown constant". It means is distributed as. The "is proportional" notation is $\propto$ and Bayesians do not use different notation then the rest of the world. $\endgroup$
    – Tim
    Oct 21, 2017 at 16:54

2 Answers 2

8
$\begingroup$

The notation is overloaded (or abused) so $p$ refers both to the probability density function as well as to the distribution. This seems a lot simpler than to have say capital letters indicate the distribution and lower case letters indicate the pdf/pmf.

Alternatively the $\sim$ notation can be thought of as overloaded so that it means the random variable on the left has the distribution on the right if the quantity on the right is a distribution. But if the quantity on the right is a pdf/pmf then $\sim$ means that the random variable on the left has the pdf/pmf of the quantity on the right.

$\endgroup$
3
  • 1
    $\begingroup$ Ah, i suspected as much. Do you have any source for this claim? $\endgroup$
    – J. Min
    Oct 21, 2017 at 17:00
  • 1
    $\begingroup$ Why would you need a source? Either this explanation will make sense of the notes you're reading or it won't. No number of sources will change that. $\endgroup$
    – whuber
    Oct 21, 2017 at 18:22
  • 2
    $\begingroup$ Point taken, although if nothing else a source would be reassuring. $\endgroup$
    – J. Min
    Oct 21, 2017 at 18:27
3
$\begingroup$

The notation is very misleading with $p(\cdot)$ being heavily overloaded with multiple meanings and complete disregard for the difference between random variables and the values that they take on.

There is a parameter $\theta$ of unknown value which is modeled as a random variable $\Theta$, discrete or continuous depending on what $\theta$ is believed to be. Assuming that $\theta$ is believed to take on discrete values, $$\Theta \sim p_\Theta(\theta)\tag{1}$$ is saying that $\Theta$ is being modeled as a discrete random variable with probability mass function $p_\Theta(\theta)$, or to paint refined gold and gild the lily, the function $p_\Theta(\cdot)$ has the property that $$P\{\Theta = \theta_i\} = p_\Theta(\theta_i).$$ There are $n$ observations $x_1, x_2, \ldots, x_n$ which are also modeled as random variables $X_1$, $X_2$, $\ldots$, $X_n$. Assuming that everything is discrete, on a trial of the experiment, these random variables $\Theta, X_1, X_2, \ldots, X_n$ have values $\theta, x_1, x_2, \ldots, x_n$ of which we can observe only $x_1, x_2, \ldots, x_n$. Now, the conditional probability mass function of the $X_i$ given that $\Theta$ has value $\theta$ is $$p_{X_1, X_2, \ldots, Xn \mid \Theta = \theta}(x_1, x_2, \ldots, x_n \mid \Theta = \theta)\tag{2}$$ meaning that $$P\big\{X_1 = x_1, X_2 = x_2, \ldots, X_n = x_n \mid \Theta = \theta_i \big\} = p_{X_1, X_2, \ldots, Xn \mid \Theta = \theta_i}(x_1, x_2, \ldots, x_n \mid \Theta = \theta_i).$$ Notice that $p$ has different subscripts in $(1)$ and $(2)$ to emphasize that there are two different functions being used here; one with just one argument and another with $n$ arguments, a far better arrangement than $p$ meaning different things depending on where it occurs and also better than the "canonical"

Let $X \sim f(x)$ and $Y \sim f(y)$ be random variables $\ldots$

or the physicists'

$X \sim f(X), Y \sim f(Y), X,Y \sim f(X,Y)$

where the density of $X$ can be quite different from the density of $Y$ and one is supposed to figure out from the letter used as the argument of $f\cdot)$ as to whether the density function of $x$ is meant or the density function of $Y$.

$\endgroup$
1
  • 2
    $\begingroup$ The "canonical" example, although frequently seen, is either mathematical nonsense or means something else altogether. "$f$" apparently is a function and "$x$" and "$y$" its arguments, whence--reading this in a conventional way--we would have to understand $f$ as being a function whose values are distributions! The "physicists" example makes no sense at all, because it's perfectly circular: it describes $X$ and $Y$ in terms of themselves! The example in the question--"$\theta\sim p(\theta)$"--is particularly egregious abuse. $\endgroup$
    – whuber
    Oct 21, 2017 at 19:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.