# Meaning of Square Root of Covariance / Precision Matrices

Say $X \in \mathbb{R}^n$ is a random variable with covariance $\Sigma \in \mathbb{R}^{n\times n}$. By definition, entries of the covariance matrix are covariances: $$\Sigma_{ij} = Cov( X_i,X_j).$$ Also, it is known that entries of the precision $\Sigma^{-1}$ satisfy: $$\Sigma^{-1}_{ij} = Cov(X_i,X_j| \{X_k\}_{k=1}^n \backslash X_i,X_j\}),$$ where the right hand side is the covariance of $X_i$ with $X_j$ conditioned on all other variables.

Is there a statistical interpretation to the entries of a square root of $\Sigma$ or $\Sigma^{-1}$? By square root of a square matrix $A$ I mean any matrix $M$ such that $M^tM = A$. An eigenvalue decomposition of said matrices does not give such entry-wise interpretation as far as I can see.

• @kjetilbhalvorsen added explanation. Swept a couple of details under the rugh, though. What is the value conditioned on? None given, so it has to be averaged over all values. – Yair Daon Jun 28 '17 at 17:00
• The account I gave of regression, correlation, and conditional distributions at stats.stackexchange.com/questions/71260/… provides explicit geometric constructions of two different square roots of the inverse covariance matrix. These geometric ideas generalize to higher dimensions, thereby supplying at least two distinct, well-known statistical interpretations (namely, PCA and multiple regression). – whuber Dec 21 '18 at 16:37

I will write matrix square roots of $$\Sigma$$ as $$\Sigma=A A^T$$, to be consistent with the Cholesky decomposition which is written as $$\Sigma = L L^T$$ where $$L$$ is lowtri (lower triangular). So let $$X$$ be a random vector with $$\DeclareMathOperator{\E}{\mathbb{E}} \DeclareMathOperator{\Cov}{\mathbb{Cov}}\DeclareMathOperator{\Var}{\mathbb{Var}} \E X=\mu$$ and $$\Var X =\Sigma$$. Let now $$Z$$ be a random vector with expectation zero and unit covariance matrix.

Note that there are (except for the scalar case) infinitely many matrix square roots. If we let $$A$$ be one of the, then we can find all the others as $$A \mathcal{O}$$ where $$\mathcal{O}$$ is any orthogonal matrix, that is, $$\mathcal{O} \mathcal{O}^T = \mathcal{O}^T \mathcal{O} =I$$. This is known as unitary freedom of square roots.

Let us look at some particular matrix square roots.

1. First a symmetric square root. Use the spectral decomposition to write $$\Sigma = U \Lambda U^T = U\Lambda^{1/2}(U\Lambda^{1/2})^T$$. Then $$\Sigma^{1/2}=U\Lambda^{1/2}$$ and this can be interpreted as the PCA (principal component analysis) of $$\Sigma$$.

2. The Cholesky decomposition $$\Sigma=L L^T$$ and $$L$$ is lowtri. We can represent $$X$$ as $$X=\mu + L Z$$. Multiplying out to get scalar equations, we get a triangular system in $$Z$$, which in the time series case can be interpreted as a MA (moving average) representation.

3. The general case $$A= L \mathcal{O}$$, using the above we can interpret this as a MA representation after rotating $$Z$$.

• +1 Re square root calculation through spectral decomposition, I've seen $\Sigma^{1/2}=U\Lambda^{1/2}U^{T}$ as well. But this would be different from the one you mentioned, yet both are valid, is that true? – NULL Nov 1 at 14:32

An nxn matrix can have many square roots as you mention. However a covariance matrix must be positive semi-definite and a positive semi-definite matrix has only one square root that is also positive semi-definite. Take a look at the wikipedia article titled "Square root of a matrix".

• I did not ask for a symmetric square root. For example, a Cholesky decomposition is enough for the purpose of this question. – Yair Daon Jan 16 '17 at 22:03
• I didn't say that a term in the covariance matrix couldn't be negative. I am talking about a different property that a covariance matrix is at least positive semi-definite. – Michael R. Chernick Feb 26 at 23:07

Sometimes people are interested in estimating the locations of zeros in the precision matrix for the same reason you describe above. If $$M$$ is your square root matrix, i.e. $$M'M = \Sigma^{-1}$$, then for nodes $$i \neq j$$ $$\Sigma^{-1}_{i,j} = 0 \iff M_i'M_j = 0$$ so I imagine that looking at the inner product between columns of your estimated square root matrix will give you some number akin to how close to conditionally independent two nodes are. Just an idea.

The square root of the covariance matrix, that's the scale. I imagine simulating a standard normal random vector, and then pre-multiplying by the square root matrix. If this matrix is lower-triangular, then I always imagine doing all the little multiplications and additions out.

There are also individual cases you can think of where certain elements of the square root matrix are just square roots of individual elements of the covariance matrix. This isn't that interesting, though, so I figure you have already thought of this.