Can I fix a non-positive definite correlation matrix to study partial correlations?

Question

Simple question. I have many correlation matrices $\mathbb C$. I want to build a partial correlation matrices using the inverse $\mathbb C^{-1}$, e.g.:

$$\mathbf {P}_{i,j} = \frac{(\mathbf {C}^{-1})_{i,j}}{\sqrt{\mathbf {(\mathbf {C}^{-1})}_{i,i} \cdot \mathbf {(\mathbf {C}^{-1})}_{j,j}}}$$

The thing is in my particular situation every matrix $\mathbb C$ is not positive definite: many eigenvalues are arbitrarily smaller than 0.

I know I can make it positive definite (at least in my case) by shrinking all non-diagonal elements towards zero, e.g. $C_{i,j} := \tanh{(0.999 \cdot \tanh^{-1}(C_{i,j}))} $, but this doesn't appear correct to me nor I have seen it applied anywhere else.

What should I do?

Answer 1

Note: Major edits vs. original version.

Per your comments, $C$ is a correlation matrix, so it has diagonal elements all equal to 1.

If you can compute the correlation coefficients more accurately, then do so. Use of higher precision might be a good idea. If that solves the problem, fine.

Otherwise, you need to make C a positive definite correlation matrix in order that its inverse will have a positive diagonal. To do so, you can try to find a positive definite correlation matrix which is as close as possible to the original matrix in the Frobenius norm sense (square root of sum of squared differences of all elements). Proceed per my solution method B at Generate normally distributed random numbers with non positive-definite covariance matrix , with the imposition of the extra constraint that all diagonal elements must be 1.

So pick a minimum eigenvalue value, say mineig = 1e-10, and solve the convex Semidefinite Programming (SDP) problem as follows:

Let C = original n by n "correlation" matrix. Solve for D, which will be the closest (per the Frobenius norm) correlation matrix to C having minimum eigenvalue of mineig.

In CVX, this would look like:

cvx_begin
variable D(n,n)
minimize(norm(D-C,'fro'))
lambda_min(D) >= mineig
diag(D) == 1
cvx_end

Here is an example run: C is the original matrix, and has minimum eigenvalue = -1.0e-8. D is the correlation matrix found as above, using mineig = 1e-10, and has minimum eigenvalue 1.0e-10.

C =

   1.000000000000000   0.775392316842042   0.775915347375462  -0.775460000000000
   0.775392316842042   1.000000000000000   0.587465374971244  -1.000000000000000
   0.775915347375462   0.587465374971244   1.000000000000000  -0.587459134116206
  -0.775460000000000  -1.000000000000000  -0.587459134116206   1.000000000000000

D =

   1.000000000000000   0.775392316905840   0.775915347450035  -0.775460000057614
   0.775392316905840   1.000000000000000   0.587465375009171  -0.999999989530503
   0.775915347450035   0.587465375009171   1.000000000000000  -0.587459134157367
  -0.775460000057614  -0.999999989530503  -0.587459134157367   1.000000000000000

D - X =

   1.0e-07 *
                   0   0.000637978558871   0.000745732364749  -0.000576140246622
   0.000637978558871                   0   0.000379269948780   0.104694971581054
   0.000745732364749   0.000379269948780                   0  -0.000411605194373
  -0.000576140246622   0.104694971581054  -0.000411605194373                   0

No element of D is more than 1.05e-8 different than the corresponding element of C.

eig(C) =

  -0.000000010368721
   0.171424190788319
   0.566827871126914
   3.261747948453490

eig(D) =

   0.000000000100778
   0.171424189980178
   0.566827867004568
   3.261747942914477

No eigenvalue of D is more than 1.05e-8 different than the corresponding element of C.

Here is the inverse of C

   1.0e+07 *
   0.000000001185191  -0.001409651498643   0.000000012853632  -0.001409643028590
  -0.001409651498643  -4.821534275445975   0.000735529929189  -4.822195310021794
   0.000000012853632   0.000735529929189   0.000000139399859   0.000735621788386
  -0.001409643028590  -4.822195310021794   0.000735621788386  -4.822856184065902

Note that 2 of the diagonal elements are negative: Ouch!!

Here is the inverse of D

   1.0e+09 *
   0.000000428103881   0.001450242932478  -0.000000223256533   0.001450443740640
   0.001450242932478   4.960731035829519  -0.000756759768677   4.961411023839190
  -0.000000223256533  -0.000756759768677   0.000000117959798  -0.000756863590705
   0.001450443740640   4.961411023839190  -0.000756863590705   4.962091107569242

All the diagonal elements are positive. Yippee.

Wait a minute. Not so fast. What if we chose a different minimum eigenvalue for D, say 1e-6, instead of 1e-10.

Here is the partial correlation matrix using D (mineig = 1e-10)

   1.000000000000000   0.995160970407181  -0.993487907487549   0.995162354269721
   0.995160970407181   1.000000000000000  -0.989277623628771   0.999999999746921
  -0.993487907487549  -0.989277623628771   1.000000000000000  -0.989277740783460
   0.995162354269721   0.999999999746921  -0.989277740783460   1.000000000000000

Here is the partial correlation matrix using mineig = 1e-6

   1.000000000000000   0.100512651870996  -0.629191171229188   0.101907771942626
   0.100512651870996   1.000000000000000  -0.067837697192938   0.999997491360146
  -0.629191171229188  -0.067837697192938   1.000000000000000  -0.067917378792838
   0.101907771942626   0.999997491360146  -0.067917378792838   1.000000000000000

Oh no, changing the minimum eigenvalue from 1e-10 to 1e-6 totally changed the partial correlation matrix. And the D corresponding to mineig = 1e-6 has no element differing by more than 1e-6 from the D using mineig = 1e-10. If you have a minimum eigenvalue close to 0, things are very sensitive, and I wouldn't put much stock in any of the results.