1
$\begingroup$

Let $X$ be a random variable with an $n\times n$ covariance matrix $A$.

Then $A$ is symmetric and positive semi-definite, and it has real and non-negative eigenvalues.

Also, if none of the components of $X$ is a linear combination of the other components, then $A$ is positive definite. Equivalently, it has only strictly positive eigenvalues. Equivalently, it has a positive determinant and is invertible.

My question is: What does the number of eigenvalues/eigenvectors of $A$ tell us about the covariance matrix $A$? What does it tell us about the components of the random variable $X$ that has covariance matrix $A$?

$\endgroup$
3
  • $\begingroup$ (Symmetric and) "has a positive determinant and is invertible" is not equivalent to being positive-definite. The simplest counterexample is $$A=\pmatrix{-1&0\\0&-1},$$ with determinant $1\gt 0$ and inverse $A^{-1}=A$. This one happens to be negative-definite, but in three or more dimensions counterexamples can be indefinite. $\endgroup$
    – whuber
    Jul 16, 2017 at 17:29
  • 1
    $\begingroup$ But your example is not a covariance matrix. I believe the statement is true for such matrices (could be mistaken). $\endgroup$
    – arni
    Jul 16, 2017 at 19:37
  • $\begingroup$ That's right--but now your assertions have become logically circular. Covariance matrices are non-negative definite, but that does not mean that "having positive determinant and is invertible" is an "equivalent" property. Since we're talking about definiteness, we have to understand all matrices as representing quadratic forms, which means we are taking them to be real, square, and symmetric. Within that class, "has only strictly positive eigenvalues" is equivalent to being positive definite, but the other two characterizations you offer are not equivalent to that. $\endgroup$
    – whuber
    Jul 17, 2017 at 13:26

1 Answer 1

5
$\begingroup$

The number of eigenvalues itself tell us nothing that we did not know (probably) already, namely the dimensions of the original matrix $A$ (or the dimensions of system of equations represented by $A$ depends how we see things).

The number of unique eigenvalues can informative (eg. the number of unique solutions to the system of equation defined by the matrix $A$), the number of non-zero eigenvalues can be informative (eg. the rank of the matrix $A$), the number of positive or negative eigenvalues can be informative (eg. is $A$ non-negative, negative, etc.), their type being real or complex can be informative (eg. is $A$ symmetric - the eigenvalues of a symmetric matrix are always real), their magnitude compared to each other can be informative ( eg. the condition number of $A$, compactness of $A$'s spectrum, etc.) but actually their number.. not much really.

The only things I would draw extra attention on is that the number of eigenvalues is NOT the cardinality of the spectrum, which is the number of unique eigenvalues. In general, recall that the fundamental theorem of algebra guarantees that every polynomial of degree one or more has a possibly complex root and we are just solving the characteristic polynomial of $A$ such that: $p_A(\lambda)=|A- \lambda I|=0$ to get the eigenvalues. The fact got $n$ of them (with multiplicity) doesn't say much.

As whuber commented, please note that a single value might occur as a root of the characteristic equation $\nu$ times in which case we say that value has (algebraic) multiplicity $\nu$. To that extent, when we deal with a $n \times n$ matrix we have $n$-th degree polynomial and as such the sum of the multiplicities of distinct eigenvalues is $n$.

This thread on Eigenvalues are unique? in Math.SE is also informative on the matter.

$\endgroup$
2
  • 2
    $\begingroup$ Please be a little careful: the $n\times n$ identity matrix has just one eigenvalue, but $1$ is not necessarily equal to $n$. I believe that in most places where you refer to the "number" of eigenvalues you intend to count them with multiplicity, but you haven't explicitly made that distinction. $\endgroup$
    – whuber
    Jul 16, 2017 at 17:33
  • $\begingroup$ @whuber: Yes, thank you for drawing attention to this (+1), of course you are correct but realistically this not what the OP is asking about it. More generally we could say that if $\lambda$ is an eigenvalue of $A$ of multiplicity $\nu$, then we can include as columns of the diagonalizing matrix $Q$ any $\nu$ linearly independent eigenvectors that correspond to $\lambda$ (such that $Q^TAQ = D$ where $D$ a diagonal matrix holding the eigenvalues of $A$ - which may not be unique). I will edit my answer to mention the multiplicity issue explicitly. $\endgroup$
    – usεr11852
    Jul 16, 2017 at 19:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.