Skip to main content
Cross Validated

Return to Answer

replaced http://stats.stackexchange.com/ with https://stats.stackexchange.com/
Source Link
Community Bot
Community Bot
  • 1

Otherwise, you need to make C a positive definite correlation matrix in order that its inverse will have a positive diagonal. To do so, you can try to find a positive definite correlation matrix which is as close as possible to the original matrix in the Frobenius norm sense (square root of sum of squared differences of all elements). Proceed per my solution method B at Generate normally distributed random numbers with non positive-definite covariance matrixGenerate normally distributed random numbers with non positive-definite covariance matrix , with the imposition of the extra constraint that all diagonal elements must be 1.

Otherwise, you need to make C a positive definite correlation matrix in order that its inverse will have a positive diagonal. To do so, you can try to find a positive definite correlation matrix which is as close as possible to the original matrix in the Frobenius norm sense (square root of sum of squared differences of all elements). Proceed per my solution method B at Generate normally distributed random numbers with non positive-definite covariance matrix , with the imposition of the extra constraint that all diagonal elements must be 1.

Otherwise, you need to make C a positive definite correlation matrix in order that its inverse will have a positive diagonal. To do so, you can try to find a positive definite correlation matrix which is as close as possible to the original matrix in the Frobenius norm sense (square root of sum of squared differences of all elements). Proceed per my solution method B at Generate normally distributed random numbers with non positive-definite covariance matrix , with the imposition of the extra constraint that all diagonal elements must be 1.

Major edits
Source Link
Mark L. Stone
Mark L. Stone
  • 13.5k
  • 1
  • 38
  • 58

IfNote: Major edits vs. original version.

Per your comments, $C$ is a correlation matrix, then ${C}_{i,i}^{-1}$ and ${C}_{j,j}^{-1}$ both $= 1$. So, it's just downso it has diagonal elements all equal to ${C}_{i,j}^{-1}$1.

If you can compute the correlation coefficients more accurately, then do so. Use of higher precision might be a good idea. If that solves the problem, fine.

Otherwise, it's probably not a great harmyou need to usemake C as is, given smallest eigenvalue ofa positive definite correlation matrix in order of -1e-8that its inverse will have a positive diagonal.

However To do so, if you want to clean things upcan try to make Cfind a positive definite correlation matrix which is as close as possible to the original matrix in the Frobenius norm sense (square root of sum of squared differences of all elements), then proceed. Proceed per my solution method B at Generate normally distributed random numbers with non positive-definite covariance matrix , with the imposition of the extra constraint that all diagonal elements must be 1.

Let C = original n by n "correlation" matrix. Solve for D, which will be the closest (per the in the Frobenius norm) correlation matrix to C having minimum eigenvalue of mineig.

So inHere is the grand schemeinverse of things, for this usage, I don't thinkC

   1.0e+07 *
   0.000000001185191  -0.001409651498643   0.000000012853632  -0.001409643028590
  -0.001409651498643  -4.821534275445975   0.000735529929189  -4.822195310021794
   0.000000012853632   0.000735529929189   0.000000139399859   0.000735621788386
  -0.001409643028590  -4.822195310021794   0.000735621788386  -4.822856184065902

Note that 2 of the diagonal elements are negative: Ouch!!

Here is the inverse of D

   1.0e+09 *
   0.000000428103881   0.001450242932478  -0.000000223256533   0.001450443740640
   0.001450242932478   4.960731035829519  -0.000756759768677   4.961411023839190
  -0.000000223256533  -0.000756759768677   0.000000117959798  -0.000756863590705
   0.001450443740640   4.961411023839190  -0.000756863590705   4.962091107569242

All the diagonal elements are positive. Yippee.

Wait a minute. Not so fast. What if we chose a different minimum eigenvalue for D, say 1e-6, instead of 1e-10.

Here is the partial correlation matrix using D (mineig = 1e-1e10)

   1.000000000000000   0.995160970407181  -0.993487907487549   0.995162354269721
   0.995160970407181   1.000000000000000  -0.989277623628771   0.999999999746921
  -0.993487907487549  -0.989277623628771   1.000000000000000  -0.989277740783460
   0.995162354269721   0.999999999746921  -0.989277740783460   1.000000000000000

Here is the partial correlation matrix using mineig = 1e-8 amo8unts6

   1.000000000000000   0.100512651870996  -0.629191171229188   0.101907771942626
   0.100512651870996   1.000000000000000  -0.067837697192938   0.999997491360146
  -0.629191171229188  -0.067837697192938   1.000000000000000  -0.067917378792838
   0.101907771942626   0.999997491360146  -0.067917378792838   1.000000000000000

Oh no, changing the minimum eigenvalue from 1e-10 to a hill of beans1e-6 totally changed the partial correlation matrix. But you can clean it up if And the D corresponding to mineig = 1e-6 has no element differing by more than 1e-6 from the D using mineig = 1e-10. If you wanthave a minimum eigenvalue close to 0, things are very sensitive, and I wouldn't put much stock in any of the results.

If $C$ is a correlation matrix, then ${C}_{i,i}^{-1}$ and ${C}_{j,j}^{-1}$ both $= 1$. So, it's just down to ${C}_{i,j}^{-1}$.

If you can compute the correlation coefficients more accurately, then do so. Use of higher precision might be a good idea.

Otherwise, it's probably not a great harm to use C as is, given smallest eigenvalue of order of -1e-8.

However, if you want to clean things up to make C a positive definite correlation matrix as close as possible to the original matrix in the Frobenius norm sense (square root of sum of squared differences of all elements), then proceed per my solution method B at Generate normally distributed random numbers with non positive-definite covariance matrix , with the imposition of the extra constraint that all diagonal elements must be 1.

Let C = original n by n "correlation" matrix. Solve for D, which will be the closest (per the in the Frobenius norm) correlation matrix to C having minimum eigenvalue of mineig.

So in the grand scheme of things, for this usage, I don't think minimum eigenvalue of -1e-8 amo8unts to a hill of beans. But you can clean it up if you want to.

Note: Major edits vs. original version.

Per your comments, $C$ is a correlation matrix, so it has diagonal elements all equal to 1.

If you can compute the correlation coefficients more accurately, then do so. Use of higher precision might be a good idea. If that solves the problem, fine.

Otherwise, you need to make C a positive definite correlation matrix in order that its inverse will have a positive diagonal. To do so, you can try to find a positive definite correlation matrix which is as close as possible to the original matrix in the Frobenius norm sense (square root of sum of squared differences of all elements). Proceed per my solution method B at Generate normally distributed random numbers with non positive-definite covariance matrix , with the imposition of the extra constraint that all diagonal elements must be 1.

Let C = original n by n "correlation" matrix. Solve for D, which will be the closest (per the Frobenius norm) correlation matrix to C having minimum eigenvalue of mineig.

Here is the inverse of C

   1.0e+07 *
   0.000000001185191  -0.001409651498643   0.000000012853632  -0.001409643028590
  -0.001409651498643  -4.821534275445975   0.000735529929189  -4.822195310021794
   0.000000012853632   0.000735529929189   0.000000139399859   0.000735621788386
  -0.001409643028590  -4.822195310021794   0.000735621788386  -4.822856184065902

Note that 2 of the diagonal elements are negative: Ouch!!

Here is the inverse of D

   1.0e+09 *
   0.000000428103881   0.001450242932478  -0.000000223256533   0.001450443740640
   0.001450242932478   4.960731035829519  -0.000756759768677   4.961411023839190
  -0.000000223256533  -0.000756759768677   0.000000117959798  -0.000756863590705
   0.001450443740640   4.961411023839190  -0.000756863590705   4.962091107569242

All the diagonal elements are positive. Yippee.

Wait a minute. Not so fast. What if we chose a different minimum eigenvalue for D, say 1e-6, instead of 1e-10.

Here is the partial correlation matrix using D (mineig = 1e-10)

   1.000000000000000   0.995160970407181  -0.993487907487549   0.995162354269721
   0.995160970407181   1.000000000000000  -0.989277623628771   0.999999999746921
  -0.993487907487549  -0.989277623628771   1.000000000000000  -0.989277740783460
   0.995162354269721   0.999999999746921  -0.989277740783460   1.000000000000000

Here is the partial correlation matrix using mineig = 1e-6

   1.000000000000000   0.100512651870996  -0.629191171229188   0.101907771942626
   0.100512651870996   1.000000000000000  -0.067837697192938   0.999997491360146
  -0.629191171229188  -0.067837697192938   1.000000000000000  -0.067917378792838
   0.101907771942626   0.999997491360146  -0.067917378792838   1.000000000000000

Oh no, changing the minimum eigenvalue from 1e-10 to 1e-6 totally changed the partial correlation matrix. And the D corresponding to mineig = 1e-6 has no element differing by more than 1e-6 from the D using mineig = 1e-10. If you have a minimum eigenvalue close to 0, things are very sensitive, and I wouldn't put much stock in any of the results.

Source Link
Mark L. Stone
Mark L. Stone
  • 13.5k
  • 1
  • 38
  • 58

If $C$ is a correlation matrix, then ${C}_{i,i}^{-1}$ and ${C}_{j,j}^{-1}$ both $= 1$. So, it's just down to ${C}_{i,j}^{-1}$.

If you can compute the correlation coefficients more accurately, then do so. Use of higher precision might be a good idea.

Otherwise, it's probably not a great harm to use C as is, given smallest eigenvalue of order of -1e-8.

However, if you want to clean things up to make C a positive definite correlation matrix as close as possible to the original matrix in the Frobenius norm sense (square root of sum of squared differences of all elements), then proceed per my solution method B at Generate normally distributed random numbers with non positive-definite covariance matrix , with the imposition of the extra constraint that all diagonal elements must be 1.

So pick a minimum eigenvalue value, say mineig = 1e-10, and solve the convex Semidefinite Programming (SDP) problem as follows:

Let C = original n by n "correlation" matrix. Solve for D, which will be the closest (per the in the Frobenius norm) correlation matrix to C having minimum eigenvalue of mineig.

In CVX, this would look like:

cvx_begin
variable D(n,n)
minimize(norm(D-C,'fro'))
lambda_min(D) >= mineig
diag(D) == 1
cvx_end

Here is an example run: C is the original matrix, and has minimum eigenvalue = -1.0e-8. D is the correlation matrix found as above, using mineig = 1e-10, and has minimum eigenvalue 1.0e-10.

C =

   1.000000000000000   0.775392316842042   0.775915347375462  -0.775460000000000
   0.775392316842042   1.000000000000000   0.587465374971244  -1.000000000000000
   0.775915347375462   0.587465374971244   1.000000000000000  -0.587459134116206
  -0.775460000000000  -1.000000000000000  -0.587459134116206   1.000000000000000

D =

   1.000000000000000   0.775392316905840   0.775915347450035  -0.775460000057614
   0.775392316905840   1.000000000000000   0.587465375009171  -0.999999989530503
   0.775915347450035   0.587465375009171   1.000000000000000  -0.587459134157367
  -0.775460000057614  -0.999999989530503  -0.587459134157367   1.000000000000000

D - X =

   1.0e-07 *
                   0   0.000637978558871   0.000745732364749  -0.000576140246622
   0.000637978558871                   0   0.000379269948780   0.104694971581054
   0.000745732364749   0.000379269948780                   0  -0.000411605194373
  -0.000576140246622   0.104694971581054  -0.000411605194373                   0

No element of D is more than 1.05e-8 different than the corresponding element of C.

eig(C) =

  -0.000000010368721
   0.171424190788319
   0.566827871126914
   3.261747948453490

eig(D) =

   0.000000000100778
   0.171424189980178
   0.566827867004568
   3.261747942914477

No eigenvalue of D is more than 1.05e-8 different than the corresponding element of C.

So in the grand scheme of things, for this usage, I don't think minimum eigenvalue of -1e-8 amo8unts to a hill of beans. But you can clean it up if you want to.