I would like to know the mathematical concepts behind singular matrices. Matrices that do not have inverses in R throw one of two errors. I have provided some examples of both errors below:

  1.  Error in solve.default(W) : 
    system is computationally singular: reciprocal condition number = 1.58603e-17
    
  2.  Error in solve.default(Z) : 
    Lapack routine dgesv: system is exactly singular: U[2,2] = 0
    

Can anyone explain the difference between the two? Why is the first "nearly singular" whereas the second is "exactly singular"?

closed as off-topic by Robert Long, kjetil b halvorsen, Peter Flom Oct 26 at 12:42

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "This question appears to be off-topic because EITHER it is not about statistics, machine learning, data analysis, data mining, or data visualization, OR it focuses on programming, debugging, or performing routine operations within a statistical computing platform. If the latter, you could try the support links we maintain." – Robert Long, kjetil b halvorsen, Peter Flom
If this question can be reworded to fit the rules in the help center, please edit the question.

  • I think it mean 1.58603e-17 is close to zero such that it is hard to perform next calculation, and 0 is zero. – user158565 Oct 11 at 0:19
  • 1
    Usually a result of highly correlated variables, you may need to drop some variables. – user2974951 Oct 11 at 8:05
  • 1
    Your computer distinguishes small floating point values, like 1.5e-17, from true zeros. One of the differences is that it is capable of dividing numbers by the former but not the latter. – whuber Oct 11 at 16:51
  • 1
    R is using double-precision arithmetic. It's not an estimation method, it's a calculation. If you want to find the inverse by hand, you can do so (without a calculator) to as many digits of accuracy as you please, at least from a computational standpoint. However, your accuracy is still limited by the accuracy of the values of your matrix; it will do no good to use 32 digit accuracy calculations and leave yourself with 15 digits of accuracy after you've lost 17 digits if the inputs themselves are only accurate to, say, 10 digits, as you will have lost all 10 digits due to ill-conditioning. – jbowman Oct 13 at 23:47
  • 1
    Consider an analogous operation: $y = x - z$. If $x=1.0000000123$ and $z = 1.0$, you'll lose 8 digits of accuracy due to subtraction. If $x$ is only accurate to 7 digits, i.e., that "$123$" is just noise, the result is noise, even though the subtraction algorithm is completely accurate. – jbowman Oct 13 at 23:51

The condition number that is typically used in linear algebra to describe a matrix $A$ indicates, very roughly, how many significant digits of accuracy you can expect to lose when solving a linear system $Ax=b$ using infinite-point accurate arithmetic. In the first case (condition number of of $1.58603e-17$), the matrix is so close to singular that you are losing about 17 digits of accuracy. Double precision floats, according to the IEEE 754 standard, have about 16 digits of accuracy, so losing 17 digits means there's no accuracy left, basically. The matrix does have an inverse, but, at some point, a responsible algorithm will tell you that you are losing more accuracy than your data has rather than return a result that is almost certainly meaningless and letting subsequent calculations (including your analysis of the results) continue on unaware of this problem. That is what "computationally singular" means in this context.

The condition number is a function of the matrix $A$, not of the algorithms operating on $A$, so, for example, changing to a different linear solver won't change the condition number.

A system that is exactly singular is one for which some of the columns are linear combinations of each other; in this case, the matrix does not have an inverse at all.

Not the answer you're looking for? Browse other questions tagged or ask your own question.