2
$\begingroup$

Bayes' rule is given by:

$$P(\theta|X) = \frac{P(X|\theta)P(\theta)}{P(X)}$$

Where $X$ are observations and $\theta$ is some model parameter. I would like to use an alternate notation to more strongly differentiate between the prior $P(\theta)$ and posterior $P(\theta|X)$ distributions. Is it appropriate to write:

$$P(\theta_\text{post}|X) = \frac{P(X|\theta_\text{prior})P(\theta_\text{prior})}{P(X)}$$

Can it be said that the posterior and prior describe the distributions of two different random variables namely $\theta_\text{post}$ and $\theta_\text{prior}$ respectively? Or are the prior and posterior different distributions of the same random variable $\theta$? So perhaps we should write:

$$P_\text{post}(\theta|X) = \frac{P(X|\theta)P_\text{prior}(\theta)}{P(X)}$$

Equally how should one denote the prior distribution?

$$\theta_\text{prior} \sim N(0,1)$$

Or:

$$P_\text{prior}(\theta) \sim N(0,1)$$

$\endgroup$
  • $\begingroup$ $\theta$ is the same random variable in $p(\theta|x)$ and $p(\theta)$. so giving $\theta$ two distinct names is totally counter-intuitive. $\endgroup$ – tgy Feb 17 '17 at 14:22
  • $\begingroup$ Your second notation seems correct, but we usually omit "prior" and "posterior" subscripts. You could write $P_{post}(\theta) = P(\theta|X)$. Also usually we write $\theta \sim N(0,1)$ $\endgroup$ – Łukasz Grad Feb 17 '17 at 14:34
4
$\begingroup$

If you want to distinguish them, you can use subscripts on the probability mass (or density) functions directly (as you have done in your second example). For simplicity, this is usually written using conditional notation, i.e. $$ P_{\theta|X}(\theta|X) = \frac{P_{X|\theta}(X|\theta)P_\theta(\theta)}{P_X(X)} $$ This has the advantage that you can distinguish $P_X(1)$ vs $P_\theta(2)$ and $P_{X|\theta}(1|2)$ vs $P_{\theta|X}(2|1)$. But if this distinction is unnecessary, then the notation just appears repetitive because the subscript is identical to the content between the parentheses. Thus, the subscripts are typically dropped.

To the second question $$ \theta \sim N(0,1)$$ is appropriate because the random variable $\theta$ has a standard normal distribution. Rather than the probability density function for the random variable $\theta$ having a standard normal distribution.

$\endgroup$
  • $\begingroup$ Could the posterior distribution be denoted: $$\theta|X \sim N(1,1)$$? $\endgroup$ – Fundamental Engineer Feb 17 '17 at 14:43
  • 1
    $\begingroup$ Yes, if that was the posterior. $\endgroup$ – jaradniemi Feb 17 '17 at 15:46
2
$\begingroup$

$\theta$ is the same random variable in both the posterior and the prior. The difference is that in the posterior you are conditioning on the data. It's your understanding of the values $\theta$ can take after you've considered the data whereas the prior is your understanding before considering the data. The $|X$ is already distinguishes the two so you don't need a subscript like $\theta_{post}$ and $\theta_{prior}$! It would be both redundant and confusing.

I also strongly suggest you do not drop the $|X$ part as in $P_{post}(\theta)$. In Bayesian statisics, it's very important to remember what information you're conditioning on. Leaving the $|X$ in there will emphasis this.

$\endgroup$
0
$\begingroup$

$P_r(\theta)$ is usually used to denote the prior distribution of $\theta$. Besides, as the posterior distribution of $\theta$ is a conditional distribution given $\theta$, no other notation is needed to distinguish it.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.