# Tag Info

• 704

### Why does inversion of a covariance matrix yield partial correlations between random variables?

Here is a proof with just matrix calculations. I appreciate the answer by whuber. It is very insightful on the math behind the scene. However, it is still not so trivial how to use his answer to ...
• 350
Accepted

### A measure of "variance" from the covariance matrix?

(The answer below merely introduces and states the theorem proven in Eq. (0) The beauty in that paper is that most of the arguments are made in terms of basic linear algebra. To answer this question ...
• 22.8k
Accepted

### Why are symmetric positive definite (SPD) matrices so important?

A (real) symmetric matrix has a complete set of orthogonal eigenvectors for which the corresponding eigenvalues are are all real numbers. For non-symmetric matrices this can fail. For example, a ...
Accepted

### What to do with random effects correlation that equals 1 or -1?

Singular random-effect covariance matrices Obtaining a random effect correlation estimate of +1 or -1 means that the optimization algorithm hit "a boundary": correlations cannot be higher than +1 or ...
• 106k
Accepted

### Are a sum and a product of two covariance matrices also a covariance matrix?

Background A covariance matrix $\mathbb{A}$ for a vector of random variables $X=(X_1, X_2, \ldots, X_n)^\prime$ embodies a procedure to compute the variance of any linear combination of those random ...
• 328k
Accepted

### Do the Determinants of Covariance and Correlation Matrices and/or Their Inverses Have Useful Interpretations?

I was able to cobble together some general principles, use cases and properties of these matrices from a desultory set of sources; few of them address these topics directly, with most merely mentioned ...
• 1,299

### Determine the off - diagonal elements of covariance matrix, given the diagonal elements

An intuitive method to determine this answer quickly is to just remember that covariance matrices may be interpreted in the form A = \begin{pmatrix} \sigma_1^2 & \rho_{12}\sigma_1\...
• 343
Accepted

• 13.5k

### A measure of "variance" from the covariance matrix?

Although the trace of the covariance matrix, tr(C), gives you a measure of the total variance, it does not take into account the correlation between variables. If you need a measure of overall ...
• 141

### Determine the off - diagonal elements of covariance matrix, given the diagonal elements

$A$ is posdef so, by Sylvesters criterion, $det(A) = 121 \cdot 81 - c^2 \geq 0$. Thus, any $c \in [-99, 99]$ will produce a valid covariance matrix.
• 2,232

### Is it possible to have a multivariate random distribution with all its random variables (pair-wise) reverse correlated?

If $X \mathrel{:=} \left(X_1, \ldots, X_k\right)^\top \sim \mathop{\mathrm{Multinomial}}\left(n, \left(p_1, \ldots, p_k\right)^\top\right)$ with $n \in \mathbb N_{\geq 1}, k \in \mathbb N_{\geq 2},$ ...
• 6,235
Accepted

### What is the geometric relationship between the covariance matrix and the inverse of the covariance matrix?

Before I answer your questions, allow me to share how I think about covariance and precision matrices. Covariance matrices have a special structure: they are positive semi-definite (PSD), which means ...
• 605

### Is the sum of two singular covariance matrices also singular?

The Spectral Theorem informs us that any $n\times n$ covariance matrix $\Sigma$ can be diagonalized. That is, there exists an orthogonal matrix $Q$ for which Q\Sigma Q^\prime = \pmatrix{\sigma^2_1 &...
• 328k
Accepted

### Is it possible to have a multivariate random distribution with all its random variables (pair-wise) reverse correlated?

Mathematically, as I commented, you can use the equi-correlation matrix \begin{align*} \Sigma = \begin{bmatrix} 1 & \rho & \cdots & \rho \\ \rho & 1 & \cdots & \rho \\ \vdots ...
• 20.5k
Accepted

### Beyond AR(1) as a covariance structure for mixed models with repeated measures

When we encounter models with repeated measures in time, we naturally want to account for the correlations within subjects and also the possibility of heterogeneous variances across time points. When ...
• 63k

### Why are symmetric positive definite (SPD) matrices so important?

You'll find some intuition in the many elementary ways of showing the eigenvalues of a real symmetric matrix are all real: https://mathoverflow.net/questions/118626/real-symmetric-matrix-has-real-...
• 14k
Accepted

### Relation between Covariance matrix and Jacobian in Nonlinear Least Squares

This is based on the standard approximation to the Hessian of a nonlinear least squares problem used by Gauss-Newton and Levenberg-Marquardt algorithms. Consider the nonlinear least squares problem: ...
• 13.5k
Accepted

### Using shrinkage when estimating covariance matrix before doing PCA

The paper you cited (Donoho et al. 2013 Optimal Shrinkage of Eigenvalues in the Spiked Covariance Model) is an impressive piece of work which I confess I did not really study. Nevertheless, I believe ...
• 106k
One useful property of the standard deviation is that it has the same units as the mean, so the magnitudes of $\sigma_X$ and $\bar X$ are directly comparable. I've never seen anyone compute the co-...