0
$\begingroup$

I have a set of input data X consisting of S&P 500 returns, that provides me with a covariance matrix C that is non positive semi-definite. The reason for the non-semi definite nature of the covariance matrix is that S&P returns data are asynchronous or incomplete. Is it possible to 'correct' this covariance matrix by kicking out the negative eigenvalues of C (or setting them equal to zero)? Could the Karhunen Loeve Transform be used, as it provides the smallest mean squared error approximation?

I used this standard procedure for the covariance matrix which is performed in Matlab using cov(): https://www.statlect.com/images/covariance-matrix__97.png

Example set: C = \begin{bmatrix}0.99&0.78&0.59&0.44\\0.78&0.92&0.28&0.81\\0.59&0.28&1.12&0.23\\0.44&0.81&0.23&0.99\end{bmatrix}

The eigenvalues in this case are -0.0004, 0.4214, 0.9988, 2.6001. In other cases the negative value is more significant than -0.0004.

Improve this question
$\endgroup$
9
  • $\begingroup$ Since it is mathematically impossible for the covariance matrix of data to have a negative eigenvalue, please explain in more detail what your situation is. How did you compute this matrix? Just how small is that negative eigenvalue? $\endgroup$
    – whuber ♦
    Commented May 18, 2017 at 18:35
  • $\begingroup$ @whuber - Thank you for your response. The data consists of S&P 500 security returns. The negative eigenvalues are not close to 0. $\endgroup$
    – Chris B
    Commented May 18, 2017 at 18:48
  • 1
    $\begingroup$ Then you have made a computational mistake. Perhaps the commonest way in which this arises is when data are partially missing and covariances are computed pairwise, which effectively uses different subsets of the data to estimate different entries. If that's not the case, then look for numerical errors. (A final possibility is that the matrix actually is positive semi-definite and the error lies in your eigenvalue calculation.) $\endgroup$
    – whuber ♦
    Commented May 18, 2017 at 18:50
  • 1
    $\begingroup$ That little detail, as well as the procedure you used to compute the matrix, are essential parts of your problem statement. Consider including them in the question. $\endgroup$
    – whuber ♦
    Commented May 18, 2017 at 18:52
  • 1
    $\begingroup$ Thank you, but they're not sufficiently clear. The image you link to is a theoretical calculation--it does not refer to data and it does not recognize the special circumstances you are describing in your comments. The reason for the problem isn't that the data are incomplete: it lies in the specific procedures you are using to compute the entries in your matrix. Normally, one would use only the data that have no missing entries. Evidently you are not, but what we don't yet know is whether that recourse is available to you or exactly how you are doing your calculation. $\endgroup$
    – whuber ♦
    Commented May 18, 2017 at 19:06

1 Answer 1

Reset to default
1
$\begingroup$

Yes: if you view your observation $\hat C$ as a noisy estimate of the true covariance matrix, you could find the closest positive semidefinite matrix to your estimate $\hat C$ by discarding the negative eigenvalues. That is, letting the eigendecomposition of $\hat C$ be $Q \Lambda Q^T$, we have that $$ \operatorname*{argmin}_{\tilde C : \tilde C \succeq 0} \lVert \tilde C - \hat C \rVert_F = Q \max(\Lambda, 0) Q^T ,$$ where the max is elementwise.

For a proof, as well as an alternating algorithm for the same problem with correlation matrices, see

Nicholas J. Higham (2002). Computing the nearest correlation matrix — a problem from finance. IMA J Numer Anal (2002) 22 (3): 329-343. (doi, free version)

This corresponds to, for example, an observation model where $\hat C$ is the true correlation matrix $C$ plus isotropic Gaussian noise. Whether that's an appropriate assumption in your case is unclear to me, not totally understanding your setting; it seems that your discussion with whuber should help sort that out.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.