I would like to know the mathematical concepts behind singular matrices. Matrices that do not have inverses in R throw one of two errors. I have provided some examples of both errors below:

  1.  Error in solve.default(W) : 
    system is computationally singular: reciprocal condition number = 1.58603e-17
    
  2.  Error in solve.default(Z) : 
    Lapack routine dgesv: system is exactly singular: U[2,2] = 0
    

Can anyone explain the difference between the two? Why is the first "nearly singular" whereas the second is "exactly singular"?

  • I think it mean 1.58603e-17 is close to zero such that it is hard to perform next calculation, and 0 is zero. – a_statistician Oct 11 at 0:19
  • Exactly, but I’m wondering why exactly that happens. Why does the first matrix return practically zero and the second return pure zero? – sebelly Oct 11 at 3:38
  • Sorry, I know very little about R, so I do not know what solve.default() does. My previous comment was based on my guess that solve.default() involves the inverse of square matrix. – a_statistician Oct 11 at 5:41
  • 1
    Usually a result of highly correlated variables, you may need to drop some variables. – user2974951 Oct 11 at 8:05
  • Your computer distinguishes small floating point values, like 1.5e-17, from true zeros. One of the differences is that it is capable of dividing numbers by the former but not the latter. – whuber Oct 11 at 16:51

The condition number that is typically used in linear algebra to describe a matrix $A$ indicates, very roughly, how many significant digits of accuracy you can expect to lose when solving a linear system $Ax=b$ using infinite-point accurate arithmetic. In the first case (condition number of of $1.58603e-17$), the matrix is so close to singular that you are losing about 17 digits of accuracy. Double precision floats, according to the IEEE 754 standard, have about 16 digits of accuracy, so losing 17 digits means there's no accuracy left, basically. The matrix does have an inverse, but, at some point, a responsible algorithm will tell you that you are losing more accuracy than your data has rather than return a result that is almost certainly meaningless and letting subsequent calculations (including your analysis of the results) continue on unaware of this problem. That is what "computationally singular" means in this context.

The condition number is a function of the matrix $A$, not of the algorithms operating on $A$, so, for example, changing to a different linear solver won't change the condition number.

A system that is exactly singular is one for which some of the columns are linear combinations of each other; in this case, the matrix does not have an inverse at all.

Your Answer

 
discard

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.