What is the difference in meaning between the notation $P(z;d,w)$ and $P(z|d,w)$ which are commonly used in many books and papers?

up vote 12 down vote accepted

I believe the origin of this is the likelihood paradigm (though I have not checked the actual historical correctness of the below, it is a reasonable way of understanding how iot came to be).

Let's say in a regression setting, you would have a distribution: p(Y | x, beta) Which means: the distribution of Y if you know (conditional on) the x and beta values.

If you want to estimate the betas, you want to maximize the likelihood: L(beta; y,x) = p(Y | x, beta) Essentially, you are now looking at the expression p(Y | x, beta) as a function of the beta's, but apart from that, there is no difference (for mathematical correct expressions that you can properly derive, this is a necessity --- although in practice noone bothers).

Then, in bayesian settings, the difference between parameters and other variables soon fades, so one started to you use both notations intermixedly.

So, in essence: there is no actual difference: they both indicate the conditional distribution of the thing on the left, conditional on the thing(s) on the right.

Although it hasn't always been this way, these days $P(z; d, w)$ is generally used when $d,w$ are not random variables (which isn't to say that they're known, necessarily). $P(z | d, w)$ indicates conditioning on values of $d,w$. Conditioning is an operation on random variables and as such using this notation when $d, w$ aren't random variables is confusing (and tragically common).

As @Nick Sabbe points out $p(y|X, \Theta)$ is a common notation for the sampling distribution of observed data $y$. Some frequentists will use this notation but insist that $\Theta$ isn't a random variable, which is an abuse IMO. But they have no monopoly there; I've seen Bayesians do it too, tacking fixed hyperparameters on at the end of the conditionals.

  • 2
    Re your 2nd paragraph, it's worth pointing out that in typical statistical situations (say, fitting a regression model), $X$ isn't considered a random variable either, but a set of known constants. – gung Aug 25 '13 at 13:49

Your Answer

 
discard

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.