What is the difference in meaning between the notation $P(z;d,w)$ and $P(z|d,w)$ which are commonly used in many books and papers?
$\begingroup$
$\endgroup$
4
-
14$\begingroup$ f(x;θ) is the same as f(x|θ), simply meaning that θ is a fixed parameter and the function f is a function of x. f(x,Θ), OTOH, is an element of a family (set) of functions, where the elements are indexed by Θ. A subtle distinction, perhaps, but an important one, esp. when it comes time to estimate an unknown parameter θ on the basis of known data x; at that time, θ varies and x is fixed, resulting in the "likelihood function". Usage of "|" is more common among statisticians, ";" among mathematicians. $\endgroup$– jbowmanCommented Jun 20, 2012 at 19:20
-
$\begingroup$ Yes jbowman is correct. We sometimes call it the density of X given Θ. $\endgroup$– Michael R. ChernickCommented Jun 20, 2012 at 19:51
-
$\begingroup$ @jbowman why not post that as an answer? My only question is - why would they use both, but I assume that it has something to do with the context (the "|" is used with "P" and the ";" with "$f$"). $\endgroup$– AbeCommented Jun 21, 2012 at 15:11
-
$\begingroup$ Good thinking, Abe; that's probably it. $f$ is more generic, I suppose. $\endgroup$– jbowmanCommented Jun 21, 2012 at 15:13
Add a comment
|
5 Answers
$\begingroup$
$\endgroup$
I believe the origin of this is the likelihood paradigm (though I have not checked the actual historical correctness of the below, it is a reasonable way of understanding how it came to be).
Let's say in a regression setting, you would have a distribution:
$$ p(Y | x, \beta) $$
Which means: the distribution of $Y$ if you know (conditional on) the $x$ and $\beta$ values.
If you want to estimate the betas, you want to maximize the likelihood:
$$ L(\beta; y,x) = p(Y | x, \beta) $$
Essentially, you are now looking at the expression $p(Y | x, \beta)$ as a function of the beta's, but apart from that, there is no difference (for mathematical correct expressions that you can properly derive, this is a necessity --- although in practice no one bothers).
Then, in bayesian settings, the difference between parameters and other variables soon fades, so one started to you use both notations intermixedly.
So, in essence: there is no actual difference: they both indicate the conditional distribution of the thing on the left, conditional on the thing(s) on the right.
answered May 2, 2011 at 10:45
$\begingroup$
$\endgroup$
9
$f(x;\theta)$ is the density of the random variable $X$ at the point $x$, with $\theta$ being the parameter of the distribution. $f(x,\theta)$ is the joint density of $X$ and $\Theta$ at the point $(x,\theta)$ and only makes sense if $\Theta$ is a random variable. $f(x|\theta)$ is the conditional distribution of $X$ given $\Theta$, and again, only makes sense if $\Theta$ is a random variable. This will become much clearer when you get further into the book and look at Bayesian analysis.
-
$\begingroup$ Uhhhh... $f(x|\theta)$ is the conditional distribution of $x$ given $\theta$ makes perfect sense even if $\theta$ is not a random variable. It's pretty much standard notation in classical statistics, where $\theta$ is not a random variable. $\endgroup$– jbowmanCommented Jun 21, 2012 at 17:10
-
$\begingroup$ Uhhhh....if you interpret that to mean that P[Θ=θ]=1 (left Θ is a random variable, right θ is a constant) then I agree. Otherwise I do not...for what then would P[Θ=θ] mean in the denominator of the definition of conditional distribution? $\endgroup$– PeterRCommented Jun 21, 2012 at 17:30
-
$\begingroup$ Denominator? I can write $x \sim f(x | \mu, \sigma)$ where $f$ is a Normal distribution without reference to Bayes' Rule. $\mu$ and $\sigma$ are fixed. Others do too, for example, ll.mit.edu/mission/communications/ist/publications/…. $\endgroup$– jbowmanCommented Jun 21, 2012 at 18:28
-
$\begingroup$ jbowman, so what is the definition of your f(x|μ,σ) as a conditional density when μ and σ are fixed numbers (i.e. not random variables)? $\endgroup$– PeterRCommented Jun 21, 2012 at 18:41
-
2$\begingroup$ The word "conditional", associated with the notation f(X|Y), is defined to be "conditional upon some random event occurring". If you are using it to mean something else, such as just "given", as in "f(x) given (specific values of) μ and σ", well then that is what the notation f(x;μ,σ) is for. Since the OP was asking about what the notation means, we should be precise about the notation in the answer. $\endgroup$– PeterRCommented Jun 21, 2012 at 19:49
$\begingroup$
$\endgroup$
11
$f(x;\theta)$ is the same as $f(x|\theta)$, simply meaning that $\theta$ is a fixed parameter and the function $f$ is a function of $x$. $f(x,\Theta)$, OTOH, is an element of a family (or set) of functions, where the elements are indexed by $\Theta$. A subtle distinction, perhaps, but an important one, esp. when it comes time to estimate an unknown parameter $\theta$ on the basis of known data $x$; at that time, $\theta$ varies and $x$ is fixed, resulting in the "likelihood function". Usage of $\mid$ is more common among statisticians, while $;$ among mathematicians.
-
1$\begingroup$ How is $f(x;θ)$ spoken verbally? Do you say " f of x given θ"? $\endgroup$ Commented Oct 23, 2014 at 22:09
-
$\begingroup$ @stackoverflowuser2010 - yes, exactly so. $\endgroup$– jbowmanCommented Oct 27, 2014 at 20:47
-
5$\begingroup$ I found in some Coursera videos that Stanford professor Andrew Ng verbalizes the semicolon as "parameterized by." See: class.coursera.org/ml-005/lecture/34 . So the example would be spoken as "f of x parameterized by theta". $\endgroup$ Commented Nov 10, 2014 at 2:50
-
7$\begingroup$ Saying "given" or "conditional" is very different (in general) from "parameterized." I'd hate if someone saw this and thought the two were equivalent. Saying "parameterized" is only appropriate when the quantity being conditioned on is a parameter indexing the pdf of the variable in the first term. For two variables (e.g., f(x;y)), using that term would be wrong. $\endgroup$– ATJCommented Jun 17, 2016 at 17:07
-
3$\begingroup$ @MikeWilliamson - Sure, pick a notation where you know what everything means and stick with it! That way when you go back to something you did earlier, like 4 hours earlier in my experience, you don't have to figure out what you meant when you used that "|". I agree, it is annoying, but after a while you just observe the first use of the notation and remember it for the rest of the paper / book; the distinctions are not usually what's important, anyway. $\endgroup$– jbowmanCommented Aug 18, 2018 at 18:42
$\begingroup$
$\endgroup$
1
Although it hasn't always been this way, these days $P(z; d, w)$ is generally used when $d,w$ are not random variables (which isn't to say that they're known, necessarily). $P(z | d, w)$ indicates conditioning on values of $d,w$. Conditioning is an operation on random variables and as such using this notation when $d, w$ aren't random variables is confusing (and tragically common).
As @Nick Sabbe points out $p(y|X, \Theta)$ is a common notation for the sampling distribution of observed data $y$. Some frequentists will use this notation but insist that $\Theta$ isn't a random variable, which is an abuse IMO. But they have no monopoly there; I've seen Bayesians do it too, tacking fixed hyperparameters on at the end of the conditionals.
-
2$\begingroup$ Re your 2nd paragraph, it's worth pointing out that in typical statistical situations (say, fitting a regression model), $X$ isn't considered a random variable either, but a set of known constants. $\endgroup$ Commented Aug 25, 2013 at 13:49
$\begingroup$
$\endgroup$
Depending on the context, the vertical bar in a function could have two different meanings:
- For example, $P(X=x|Y=y)$ could mean the probability of $X=x$ conditional upon $Y=y$, where $X$ and $Y$ are random variables and $x$ and $y$ are the corresponding realisation.
- On the other hand, $P(X=x| \theta)$ could mean the probability of $X=x$ given $\theta$, where $X$ is a random variable, $x$ is the corresponding realisation, and $\theta$ is a fixed parameter.
And semicolon in a function would mean fixing parameters. Consider the function $f: \mathbb{R}^2 \to \mathbb{R}$, say $(x,y) \mapsto f(x,y)$. If you would like to fix the variable $y$ in $f$ as a parameter, you are actually creating a new function $g: \mathbb{R} \to \mathbb{R}$, given by $g(x) := f(x,y)$. As a notation sugar, people would use $f(x;y)$ to represent $g(x)$.
You may notice that the meaning of semicolon is equivalent to the second meaning of vertical bar, i.e. $f(x;y)=f(x|y)$ and yes they are.
This kind of notation abuses happens in many areas. I also observed this when studying multivariable calculus.
