5
$\begingroup$

I think I'm confused about a very simple thing. When we say that some variable is distributed as a Poisson distribution and we write $y \sim \text{Pois}(\lambda)$, is this the same that saying $y|\lambda\sim \text{Pois}(\lambda)$?

I remember that at first, I thought the conditional was meant to describe things that we already know, so if I write $y|\lambda$, I assumed that $\lambda$ was a known constant. Then I encountered cases where $\lambda$ was a distribution, so I retroactively thought that $y\sim \text{Pois}(\lambda)$ was used with $\lambda$ as an implicit parameter because it is obvious that $y$ depends on $\lambda$, and many books routinely make some parameters implicit. Furthermore, it was unusual to think of a distribution $\lambda$ as a known value. In any case, it could be a possible realization of $\lambda$. However, using $y$ (without conditional), I think, is a bit a misleading since this implies a marginalized $y$.

So, what is the correct interpretation of the conditional in distributions such as $y \sim \text{Pois}(\lambda)$?

As a way to describe more clearly where both interpretations of conditional can be confusing, take this example: Suppose a Poisson distribution $y_{t}\sim \text{Pois}(\mu_{t})$ where $\log(\mu_{t}) = X_{t}\beta + b$. Then,

$$cov(y_{t},y_{s})=E[cov(y_{t},y_{s}|b)]+cov[E(y_{t}|b),E[y_{s}|b)]$$

(using the law of total covariance)

Therefore, $y_{t}$ should be understood as the Poisson distribution without conditional, because in the right hand side of this identity, we are using the conditional on $b$.

As you can see, I'm confused about what it really means $y|\lambda$ as opposed to a marginalized $y$ or a distribution $y$ with parameter $\lambda$.

$\endgroup$
7
  • 2
    $\begingroup$ This is not such an issue. In the Poisson example, the rv $Y$ cannot be defined without the knowledge of $\lambda$. In that sense, the distribution of $Y$ is conditional on $\lambda$, whether it is a constant (hence a rv with a Dirac mass) or a random variable. By the very nature of the conditional distribution, it does not make a difference. $\endgroup$ – Xi'an Dec 6 '14 at 19:46
  • 1
    $\begingroup$ That is exactly what I was thinking a minute ago. However, I was wondering if a conditional is always based on a single value: $p(y|\lambda=\lambda_{0})$. That is, when we write $p(y|\lambda)p(\lambda)$, this is actually $p(y|\lambda=\lambda_{0})p(\lambda=\lambda_{0})$ for every $\lambda_{0}$ in the distribution. $\endgroup$ – Robert Smith Dec 6 '14 at 20:22
  • $\begingroup$ No, no. no [good question!]! The definition of a conditional distribution is that, once the conditioning random variable $\lambda$ takes a realisation $\lambda_0$ say, then the distribution of $Y$ is $p(y|\lambda_0)$. But, when we write $p(y|\lambda)$, we mean that whatever the realisation of $\lambda$ is, this will be the corresponding density of $y$. So you can unreservedly write a density conditioned by a random variable. $\endgroup$ – Xi'an Dec 6 '14 at 20:29
  • 1
    $\begingroup$ Uhm, I'm not sure what is the difference between what you said and what I said. I don't mean to say that $p(y|\lambda)$ implies $p(y|\lambda=\lambda_{0})$. Since, $p(y|\lambda)$ (where $\lambda$ is a random variable) is a family of poisson distributions, each centered in a different $\lambda$. So, when we take a realization $\lambda_{0}$, there is a single poisson distribution $p(y|\lambda=\lambda_{0})$. Is this correct? $\endgroup$ – Robert Smith Dec 6 '14 at 20:55
  • $\begingroup$ (+1) this is indeed correct! $\endgroup$ – Xi'an Dec 6 '14 at 21:00
1
$\begingroup$

As a Bayesian, it does not make much of a difference to me to think of $\mathcal{P}(\lambda)$ as a given distribution, indexed by a parameter $\lambda$, or as a conditional distribution, conditional on the realisation of a random variable $\lambda$. Indeed, in either case, when I observe $$ y_1,\ldots,y_n\stackrel{\text{iid}}{\sim}\mathcal{P}(\lambda) $$ the data is indexed by a given if unknown value of $\lambda$. Whether or not this $\lambda$ is the realisation of a random variable does not change the behaviour of the data. Remember, there is only one realisation of $\lambda$ for a given dataset, no matter its size. So, even when making the assumption that $\lambda\sim\pi(\lambda)$ (a certain prior distribution chosen by me), I do not get observations from the marginal $$ m(y_i) = \int_0^\infty f(y_i|\lambda)\pi(\lambda)\,\text{d}\lambda $$ but observations from the conditional. Hence, for the likelihood function and related inference, conditioning or not does not make a difference as the parameter is assumed fixed for the data at hand.

Conditioning only makes a difference when running a Bayesian analysis, since the posterior $$ \pi(\lambda|y_1,\ldots,y_n) \propto f(y_1|\lambda)\cdots f(y_n|\lambda)\pi(\lambda) $$ only makes sense as a conditional distribution from the joint distribution $$ \pi(\lambda,y_1,\ldots,y_n) = f(y_1|\lambda)\cdots f(y_n|\lambda)\pi(\lambda) $$ (meaning that $\lambda$ has to be a random variable for this derivation to make sense).

At a probabilistic level, assuming the $\sigma$-algebra on $\mathcal{Y}\times\Theta$ is induced by the products of the measurable sets on $\mathcal{Y}$ and on $\Theta$, writing $p_\lambda(y)$ or $p(y|\lambda)$ does not make a difference either.

$\endgroup$
2
$\begingroup$

It seems ambiguous to me whether writing $y$ or $y \; | \; \lambda$ is correct. I'd say the answer depends on whether you're in a bayesian or a frequentist setting:

  • If you're being bayesian, you'll often model parameters as random. In that case, you'd write $y \;|\; \lambda \; \sim \; F(\cdot\,; \lambda)$, and you would have some prior over $\lambda$ (which you could use to calculate the marginal distribution of $y$, if desired)
  • If you're being frequentist, you think of $\lambda$ as something unknown but fixed, so you just write $y \; \sim \; F(\cdot\,; \lambda)$ -- it wouldn't make sense to condition on $\lambda$
$\endgroup$
5
  • 3
    $\begingroup$ Right, but one also writes $p(y \mid \lambda)$ in frequentist statistics to denote a distribution depending on $\lambda$. This has not a conditional meaning, this is just another notation for $p_\lambda(y)$ for example. $\endgroup$ – Stéphane Laurent Dec 6 '14 at 18:40
  • $\begingroup$ I would write $p(y \,;\, \lambda)$ in that case to avoid ambiguity, or $p_\lambda(y)$ as you pointed out. $\endgroup$ – Adrian Dec 6 '14 at 18:59
  • $\begingroup$ Thank you, Adrian. I was about to edit my question with Stéphane's observation. Sometimes, the conditional notation is used as a way to convey dependence. $\endgroup$ – Robert Smith Dec 6 '14 at 19:05
  • $\begingroup$ @Adrian Just to clarify. In the Bayesian case, we are using a conditional notation but not conditioning on $\lambda$, correct? Otherwise, a true conditional $y|\lambda$ would be a random variate with Dirac mass instead of $F$ as Xi'an pointed out. Then, how do you describe a true conditional in a Bayesian setting? Obviously, if a prior is explicit, there is a way to identify $y|\lambda$ as $y$ depending on $\lambda$ where $\lambda$ has some distribution. However, this is not clear in all cases. $\endgroup$ – Robert Smith Dec 6 '14 at 20:32
  • 1
    $\begingroup$ no, the Bayesian approach explicitly relies on all sampling distributions being conditional on the parameter(s) since the parameter(s) are then random. So $p(y|\lambda)$ is a true conditional then. And my remark about the Dirac mass was that the prior of $\lambda$ could formally be a Dirac mass, in which case $\lambda$ would a constant, e.g. $\lambda_0$. $\endgroup$ – Xi'an Dec 6 '14 at 21:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.