Let $X$ follow a Normal distribution, and let $Y = a+bX$ (not both constants zero). The normal distribution is closed under shifting and scaling, so $Y$ also follows a normal distribution.

If I recall correclty, there exists "Cramer's condition" for bivariate normality, where we get it if all linear combinations of two random variables have a univariate normal distribution. In our case it is easy to see that any $Z = \delta_1X + \delta_2 Y$ will follow a Normal distribution too (not both constants zero). So it appears that $(X,Y)$ are jointly normal, with correlation coefficient equal to unity, in absolute value.

The question is the following: the joint support of $(X,Y)$ is a line in $\mathbb R^2$, and as a $(n-1)-$dimensional object in a $n-$dimensional space, it has Lebesgue measure zero. This I think, implies that the joint probability density function does not exist.

Q1: Is the above correct?

Q2: What are the main practical consequences of not being allowed to define the joint density (like, indicatively, what kind of statistical work we cannot do, and what kind we still can do, what statistical methods are no longer applicable, and which are still applicable)?

I guess the situation generalizes to any distribution that is closed under affine transformations, and when we want to assume in addition that the bivariate vector follows the 2-D analogue of the marginal distributions.

  • 1
    $\begingroup$ In Cramer's condition, you don't need to state that $X$ and $Y$ are normal random variables: if all linear combinations $\delta_1X+\delta_2Y$ are normal (where the coefficients are allowed to be $0$) then $X$ and $Y$ are jointly normal. Many people take this as the definition of bivariate (or more generally, multivariate) normality even though it allows for $0$ (more generally, a constant) to be thought of as a (degenerate) normal random variable. See the extensive discussion in the comments following this answer $\endgroup$ – Dilip Sarwate Feb 6 '18 at 16:43
  • $\begingroup$ Also, if $Y=a+bX$ then the correlation coefficient could be $-1$ , not just $+1$ as implied by "correlation coefficient equal to unity". $\endgroup$ – Dilip Sarwate Feb 6 '18 at 16:46
  • 1
    $\begingroup$ Since the only potential problem in working with the random variable $(X,Y)$ with a singular distribution concerns how it is represented mathematically, any failure to perform "statistical work" would merely represent a failure on the analyst's part to describe the situation in an effective way. It could not possibly preclude the use of any statistical procedures. What would an "effective way" mean? It can be as simple as just ignoring $Y$, because its values are determined by $X$. Thus there seem to be no conceptual, theoretical, or practical difficulties engendered by this situation. $\endgroup$ – whuber Feb 6 '18 at 17:14
  • $\begingroup$ You still need to delete the requirement that the coefficients must be nonzero; otherwise, there is no way to show that $X$ and $Y$ are marginally normal. Unless, of course, you mean that $\delta_1,\delta_2)\neq (0,0)$ in which case you should say so explicitly. $\endgroup$ – Dilip Sarwate Feb 6 '18 at 18:13
  • $\begingroup$ @Xi'an As a stand-alone mere statement this is hardly useful. Could you perhaps elaborate by posting a related answer? Thank you. $\endgroup$ – Alecos Papadopoulos Feb 6 '18 at 20:44

Let $P_{XY}(A) := P\left((X , Y) \in A\right)$. There exists a set $A \in \mathfrak B^2$ (the Borel $\sigma$-algebra over $\mathbb R^2$) such that $\lambda^2(A) = 0$ yet $P_{XY}(A) > 0$, so $P_{XY}$ is not dominated by $\lambda^2$ and therefore it has no Lebesgue pdf (with respect to $\lambda^2$, the Lebesgue measure on $\mathbb R^2$), as you said. That's not up for debate. But, as @Dilip Sarwate says in the comments, we can consider $(X , Y)$ to have a degenerate bivariate Gaussian distribution. I think the way to reconcile this is that having a pdf proportional to $\exp\left(-\frac 12 (z - \mu)^T \Sigma^{-1} (z - \mu)\right)$ with $z \in \mathbb R^n$ is sufficient, but not necessary, for being a Gaussian over $\mathbb R^n$. That's one key advantage to the definition of a multivariate Gaussian being $Z \sim \mathcal N_n(\mu, \Sigma)$ iff $a^T Z \sim \mathcal N_1(a^T \mu, a^T \Sigma a)$ for all $a \in \mathbb R^n$ rather than specifying a pdf for $Z$.

Now let $h(x) = a + bx$ so $h$ is measurable. We have $$ P(X \in A_1, Y \in A_2) = P(X \in A_1, X \in h^{-1}(A_2)) = P(X \in A_1 \cap h^{-1}(A_2)) = P_X(A_1 \cap h^{-1}(A_2)). $$

We do have $P_X \ll \lambda$ (assuming $X$ is not also degenerate) so the 1-dimensional Lebesgue pdf $\frac{\text dP_X}{\text d\lambda}$ is defined. Thus we have $$ P_{XY}\left(A_1 \times A_2\right) = \int_{A_1 \cap h^{-1}(A_2)} \frac{\text dP_X}{\text d\lambda} \,\text d\lambda $$ so even though the 1-dimensional Gaussian pdf $\frac{\text dP_X}{\text d\lambda}$ isn't the pdf of $P_{XY}$, it is still all we need to perfectly describe $P_{XY}$.

To conclude, if $(X, Y)$ is degenerate it won't have a pdf with respect to $\lambda^2$ but it is still considered a bivariate Gaussian, and we can compute $P((X, Y) \in A_1 \times A_2)$ by integrating the pdf of $X$ with respect to to Lebesgue measure on $\mathbb R$. This agrees with @whuber's comment because there's no mystery about the behavior of $(X, Y)$, it just doesn't have a 2-D pdf.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.