PDF of a degenerate multivariable gaussian distribution

The pdf of a multivariate gaussian distribution with mean $$\mu\in\mathbb{R}^n$$ and variance $$\Sigma\in\mathbb{S}^n_{++}$$ ($$\Sigma$$ is positive definite) is given as $$p(x) = \frac{1}{\sqrt{|\Sigma|(2\pi)^n}} \operatorname{exp}\left( (x-\mu)^\top \Sigma^{-1} (x-\mu) \right).$$

Now, consider the case of a degenerate multivariate gaussian such that $$\Sigma \in \mathbb{S}^n_{+}$$, i.e. one or more eigenvalues of $$\Sigma$$ are zero. For the univariate case ($$\Sigma$$ is a scalar), it has been established in this question if the variance is 0 then the pdf does not exist. I am interested in the implications for the multivariate case.

Suppose, for simplicity, $$n=3$$, $$\mu = [1,2,3]^\top$$, and $$\Sigma = \begin{bmatrix} 1&0&0\\0&0&0\\0&0&1 \end{bmatrix}.$$ Furthermore, let $$e_1,e_2,e_3$$ be the canonical bases in $$\mathbb{R}^3$$. It is evident that the multivariate distribution $$\mathcal{N}(\mu,\Sigma)$$ for $$x\in\operatorname{span}\{e_2\}$$ stipulates a deterministic value of $$x = [0,2,0]^\top$$. However, for $$x\in\operatorname{span}\{ e_1,e_3 \}$$, $$\mathcal{N}(\mu,\Sigma)$$ specifies a non-degenrate distribution. My question is, is there any way to 'write down' the pdf for $$x\in\operatorname{span}\{ e_1,e_3 \}$$?

When $$\Sigma$$ is singular, you don't define a density over random vectors. Perfect correlation implies the $$n$$-vectors lie in a $$n$$-plane, which has volume 0, i.e. its Lebesgue measure $$\mathcal{L}^n$$ is 0, so it's not a density on $$\mathbb{R}^n$$. This is true of all densities, not only multivariate normal densities.

For simplicity, suppose we have a bivariate normal distribution and the covariance matrix has perfect correlation between the two vectors. All of the probability mass falls in a line on the plane. A line has 0 area. However, by the Kolmogorov axioms, all probability distributions have total measure 1. Due to this contradiction, this distribution is not a probability density.

Sometimes, software will cheat and ignore this fact. Some discussion: Why Multivariate Distribution with Covariance Matrix with Rank 1 Works?

• Sure. But does this mean that the entity 'multivariate gaussian distribution $\mathcal{N}(\mu,\Sigma)$' is not well-defined if $\Sigma$ is singular? Jun 23, 2021 at 2:10
• Yes. (fifteen characters)
– Sycorax
Jun 23, 2021 at 2:25