1
$\begingroup$

Suppose we have two independent standard normal variables $X$ and $Y$ and lets construct $Z$ such as $Z = \rho X + \sqrt{1-\rho^2 Y}$. Therefore, $Z$ is dependent on $X$ and $Y$. I want to find the distribution of $Z|X, Y$. However, when I try to generate the Multivariate Normal distribution from these three Gaussians, I obtain a singular covariance matrix. I read the Wikipedia article on the Multivariate Normal distribution and it mentions a degenerate case when the covariance matrix is singular, giving the following expression:

$f(\mathbf{x})= \left(\det\nolimits^*(2\pi\boldsymbol\Sigma)\right)^{-\frac{1}{2}}\, e^{ -\frac{1}{2}(\mathbf{x}-\boldsymbol\mu)^{{{\!\mathsf{T}}}} \boldsymbol\Sigma^+(\mathbf{x}-\boldsymbol\mu) }$

where $\boldsymbol\Sigma^+$ is the generalized inverse and $\text{det}*$ is the pseudo-determinant.

So, my question is: is it correct to use this distribution to find $Z|X,Y$? And in this case, can I still say that $Z|X,Y$ has continuous pdf of $\frac{f(x,y,z)}{f(x,y)}$? I lack some knowledge on this topic, so I would also be grateful for some references.

$\endgroup$
7
  • $\begingroup$ Thank you for your answer. How can I know if n normally distributed variables have an n-variate joint density? And in my case, does $X$, $Y$ and $Z$ have a joint distribution, even if it is not trivarate normal? $\endgroup$
    – donut
    Nov 25, 2020 at 11:58
  • $\begingroup$ Thank you. Can you suggest me some bibliography on the matter? $\endgroup$
    – donut
    Nov 26, 2020 at 1:34
  • $\begingroup$ @Dilip I do not know of any distribution of joint Normality that would rule out $(X,Y,Z)$ as being jointly Normal. If it did, we would have to put all kinds of edge case restrictions on standard results about Normal distributions (such as, "linear combinations of joint normal variables are jointly Normal except when they are of reduced rank"). $\endgroup$
    – whuber
    Jul 20 at 20:49
  • $\begingroup$ @whuber I miswrote. $X, Y, Z$ indeed are jointly normal but they don't enjoy a trivariate density function. Since comments are not editable, should I just delete the incorrect comments and replace them by new ones? $\endgroup$ Jul 20 at 20:58
  • $\begingroup$ @Dilip Sure, that's fine. $\endgroup$
    – whuber
    Jul 20 at 21:31

1 Answer 1

1
$\begingroup$

Since $X$ and $Y$ are independent, $(X, Y)$ have a joint bivariate normal distribution. Then you define $Z$ as a function of $X$ and $Y$ (I assume $\rho$ is a known constant). So the conditional distribution of $Z$ given that $X=x, Y=y$ is simply the constant $z=\rho x + \sqrt{1-\rho^2} y$.

So the joint distribution of $X, Y, Z$ is singular (see Multivariate normal with singular covariance, Can a multivariate distribution with a singular covariance matrix have a density function?, )

In your case the function defining $Z$ is a linear function, so a full description of the joint distribution can be given by the bivariate normal distribution of $X, Y$, and that linear function. If you replaced the linear function by some non-linear function, for example $f(x,y) = x + y^3$, then the distribution would still be singular (the probability mass would be concentrated on a two-dimensional surface in 3-space), but the $3ţimes 3$ covariance matrix would not be singular (that is, it would have an inverse), so the invertibility of the covariance matrix does not guarantee a non-singular distribution.

A simple example in R:

n <- 10000
set.seed(7*11*13)
X <- rnorm(n)
Y <- rnorm(n)
f <- function(x, y) x + y^3
Z <- f(X, Y)
dat <- cbind(X, Y, Z)

C <- cov(dat)
C

X  0.998418216 -0.005227852  1.012374
Y -0.005227852  0.984617994  2.955338
Z  1.012373644  2.955338277 16.487973
> svd(C)
$d
[1] 17.0937095  1.0001744  0.3771255

$u
            [,1]          [,2]       [,3]
[1,] -0.06169015  0.9460834489 -0.3179944
[2,] -0.18008324 -0.3239229950 -0.9287863
[3,] -0.98171501 -0.0000315037  0.1903566

$v
            [,1]          [,2]       [,3]
[1,] -0.06169015  0.9460834489 -0.3179944
[2,] -0.18008324 -0.3239229950 -0.9287863
[3,] -0.98171501 -0.0000315037  0.1903566

As all the singular values are positive, the covariance matrix is not singular.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.