1
$\begingroup$

I know that the conditional distribution of two gaussian is gaussian. But in the following statement how does the Θ captures the conditional distributions?

And what do they mean by the term "captures" here?

Let's suppose X is a random variable such that X = ( X1 , X2 , . . . , Xp ) has a multivariate Gaussian distribution with mean-vector 0 (for convenience ) and covariance Σ , then Θ =Σ−1 captures the conditional distributions of each Xj given the rest.

$\endgroup$
  • 1
    $\begingroup$ Zeroes in $\Theta$ are indicative of conditional independence. Given the rest. $\endgroup$ – Xi'an Jun 6 '18 at 11:41
  • $\begingroup$ Thanks, Sir, this fact I know but how they related the inverse covariance matrix with undirected graphical model. So that zeros in Θ represent independency between nodes in graph. $\endgroup$ – ironman Jun 6 '18 at 11:49
1
$\begingroup$

What is meant is simply that for Gaussian variables: dependency = linear dependency. In other words, all the information on the dependency between two Gaussians is in their covariance or correlation.

In turn, if you know the covariance matrix, then you have the variances and correlations between each element $X_i$ and the others: $\rho(X_j, X_j), j \neq i$.

Last, if you have centered gaussians and you know their variances and correlations, you can write them using independent Gaussians which will give you the expression of the conditional value of one given the others.

Let's take two standard guassians for example $(X_1, X_2)$ with a correlation $\rho$. If you want the distribution of $X_2 | X_1$ for example, you have to:

  1. write $X_2 = \rho X_1 + \sqrt{1 - \rho^2} X_0$ , with $X_0$ a standard gaussian independant from $X_1$.

  2. conclude that $X_2 | X_1$ is gaussian with mean $\rho X_1$ and variance $1 - \rho^2$.

This extends to an inversion of Cholesky's decomposition (a.k.a. square root matrix of the covariance matrix) in higher dimensions.

$\endgroup$
  • $\begingroup$ Thanks Sir for answering. But I have one more doubt. I understand that using covariance matrix we can write the conditional value. But what is the need of inverse covariance matrix here. I will be obliged if you explain your answer a little bit more. Would you explain me with a small example! $\endgroup$ – ironman Jun 6 '18 at 10:48
  • $\begingroup$ Completed my answer to clarify all this. I hope it's clear now. Would be good if you include the reference of the citation you used in the question. Thanks! The fact that the inverse of Cholesky's decomposition of a matrix $C$ is fully characterized by the inverse of the matrix $C^{-1}$ doesn't look straightforward to express explicitly unfortunately. $\endgroup$ – byouness Jun 6 '18 at 11:26
  • $\begingroup$ It is given in some undirected graphical model slides in google. Unable to find now. This bloody inverse covariance matrix is creating problem to me. I don't understand its purpose here. But thanks sir anyway. $\endgroup$ – ironman Jun 6 '18 at 11:38
  • $\begingroup$ You are welcome, always glad to help when I can. Maybe it's simply a typo and what they wanted to say was inverse of Cholesky decomposition. $\endgroup$ – byouness Jun 6 '18 at 14:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.