Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
I came across the following statement
$cov(E(\mathbf{z}|y))$ is degenerate in all direction orthogonal to $Span(\mathbf{x}_1, ...,\mathbf{x}_K)$
Vector $\mathbf{z}$ is a centred random vector of size $p$, $y$ is a scalar random variable and the $\mathbf{x}_k$ are $K$ vectors of size $p$.
Does it mean that any vector orthogonal to $Span(\mathbf{x}_1, ...,\mathbf{x}_K)$ is in the kernel of the covariance matrix of $E(\mathbf{z}|y)$?
How would you interpret it?
Yes.
This figure shows the situation for $p=2$ where the span of $\{\mathbf{x}_1, \ldots, \mathbf{x}_K\}$ is one-dimensional, shown as a red line through the origin, and the orthogonal space--the kernel of the covariance matrix--also is one-dimensional, shown as a dashed gray line through the origin. Data are shown as points on the red line.
Evidently, the data exhibit no variation in directions parallel to the orthogonal space.
When $\mathbf{z}|y$ is a random variable, a similar picture and the same interpretation hold. Now, any realization of $\mathbf{z}|y$ must lie on the red line. No two realizations can differ by any nonzero element of the orthogonal space.
In this case the covariance matrix is singular, i.e. it is not of full rank. The directions associated with the zero eigenvalues are degenerate.
