When modelling multiple variables with multivariate Gaussian, we often assume that the covariance matrix is diagonal to reduce the complexity and computation cost of the problem. In such cases, if we're trying to predict the distribution of the variables, even if the variables are not actually independent the relations between them might be observed by what predict the parameters of the distribution, e.g. a neural network.

However if we have partial prior (domain) knowledge on which and how variables could be dependent to each other, say, variable i and variable j might be related, what could we do to better model the variables? Note that we still could not model the entire N^2 sized covariance matrix due to computation costs, and our prior knowledge is partial and could be incomplete (i.e. some relations are unknown) and inaccurate.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.