1
$\begingroup$

The pdf of a multivariate gaussian distribution with mean $\mu\in\mathbb{R}^n$ and variance $\Sigma\in\mathbb{S}^n_{++}$ ($\Sigma$ is positive definite) is given as $$ p(x) = \frac{1}{\sqrt{|\Sigma|(2\pi)^n}} \operatorname{exp}\left( (x-\mu)^\top \Sigma^{-1} (x-\mu) \right).$$

Now, consider the case of a degenerate multivariate gaussian such that $\Sigma \in \mathbb{S}^n_{+}$, i.e. one or more eigenvalues of $\Sigma$ are zero. For the univariate case ($\Sigma$ is a scalar), it has been established in this question if the variance is 0 then the pdf does not exist. I am interested in the implications for the multivariate case.

Suppose, for simplicity, $n=3$, $\mu = [1,2,3]^\top$, and $$ \Sigma = \begin{bmatrix} 1&0&0\\0&0&0\\0&0&1 \end{bmatrix}.$$ Furthermore, let $e_1,e_2,e_3$ be the canonical bases in $\mathbb{R}^3$. It is evident that the multivariate distribution $\mathcal{N}(\mu,\Sigma)$ for $x\in\operatorname{span}\{e_2\}$ stipulates a deterministic value of $x = [0,2,0]^\top$. However, for $x\in\operatorname{span}\{ e_1,e_3 \}$, $\mathcal{N}(\mu,\Sigma)$ specifies a non-degenrate distribution. My question is, is there any way to 'write down' the pdf for $x\in\operatorname{span}\{ e_1,e_3 \}$?

$\endgroup$

1 Answer 1

0
$\begingroup$

When $\Sigma$ is singular, you don't define a density over random vectors. Perfect correlation implies the $n$-vectors lie in a $n$-plane, which has volume 0, i.e. its Lebesgue measure $\mathcal{L}^n$ is 0, so it's not a density on $\mathbb{R}^n$. This is true of all densities, not only multivariate normal densities.

For simplicity, suppose we have a bivariate normal distribution and the covariance matrix has perfect correlation between the two vectors. All of the probability mass falls in a line on the plane. A line has 0 area. However, by the Kolmogorov axioms, all probability distributions have total measure 1. Due to this contradiction, this distribution is not a probability density.

Sometimes, software will cheat and ignore this fact. Some discussion: Why Multivariate Distribution with Covariance Matrix with Rank 1 Works?

$\endgroup$
2
  • $\begingroup$ Sure. But does this mean that the entity 'multivariate gaussian distribution $\mathcal{N}(\mu,\Sigma)$' is not well-defined if $\Sigma$ is singular? $\endgroup$ Jun 23, 2021 at 2:10
  • $\begingroup$ Yes. (fifteen characters) $\endgroup$
    – Sycorax
    Jun 23, 2021 at 2:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.