For a set of $p$ datapoints in $m$ dimensional space, if the features are packed in a $p\times m$ matrix $X$, then $C = XX^T$ is the covariance matrix and $K = C^{-1}$ is the inverse covariance matrix. If $C$ is sparse, it means that certain datapoints are uncorrelated, and in particular, if the data is Gaussian, then certain pairs of datapoints are independent. If $K$ is sparse, it means that certain pairs of datapoints are independent conditioned on the entire dataset.

My question is, what is the practical application in which $K$ would be sparse but not $C$? One example might be if there is a latent variable that couples datapoints, and if that latent variable is conditioned then the other datapoints are independent. But again, that's not nearly as strong as saying it's independent conditioned on all other datapoints. Are there any real world scenarios where such a strong assumption is needed?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.