3
$\begingroup$

I would like to know the mathematical concepts behind singular matrices. Matrices that do not have inverses in R throw one of two errors. I have provided some examples of both errors below:

  1.  Error in solve.default(W) : 
system is computationally singular: reciprocal condition number = 1.58603e-17

  2.  Error in solve.default(Z) : 
Lapack routine dgesv: system is exactly singular: U[2,2] = 0

Can anyone explain the difference between the two? Why is the first "nearly singular" whereas the second is "exactly singular"?

Improve this question
$\endgroup$
8
  • $\begingroup$ I think it mean 1.58603e-17 is close to zero such that it is hard to perform next calculation, and 0 is zero. $\endgroup$
    – user158565
    Commented Oct 11, 2018 at 0:19
  • 1
    $\begingroup$ Usually a result of highly correlated variables, you may need to drop some variables. $\endgroup$
    – user2974951
    Commented Oct 11, 2018 at 8:05
  • 1
    $\begingroup$ Your computer distinguishes small floating point values, like 1.5e-17, from true zeros. One of the differences is that it is capable of dividing numbers by the former but not the latter. $\endgroup$
    – whuber ♦
    Commented Oct 11, 2018 at 16:51
  • 1
    $\begingroup$ R is using double-precision arithmetic. It's not an estimation method, it's a calculation. If you want to find the inverse by hand, you can do so (without a calculator) to as many digits of accuracy as you please, at least from a computational standpoint. However, your accuracy is still limited by the accuracy of the values of your matrix; it will do no good to use 32 digit accuracy calculations and leave yourself with 15 digits of accuracy after you've lost 17 digits if the inputs themselves are only accurate to, say, 10 digits, as you will have lost all 10 digits due to ill-conditioning. $\endgroup$
    – jbowman
    Commented Oct 13, 2018 at 23:47
  • 1
    $\begingroup$ Consider an analogous operation: $y = x - z$. If $x=1.0000000123$ and $z = 1.0$, you'll lose 8 digits of accuracy due to subtraction. If $x$ is only accurate to 7 digits, i.e., that "$123$" is just noise, the result is noise, even though the subtraction algorithm is completely accurate. $\endgroup$
    – jbowman
    Commented Oct 13, 2018 at 23:51

1 Answer 1

Reset to default
9
$\begingroup$

The condition number that is typically used in linear algebra to describe a matrix $A$ indicates, very roughly, how many significant digits of accuracy you can expect to lose when solving a linear system $Ax=b$ using infinite-point accurate arithmetic. In the first case (condition number of of $1.58603e-17$), the matrix is so close to singular that you are losing about 17 digits of accuracy. Double precision floats, according to the IEEE 754 standard, have about 16 digits of accuracy, so losing 17 digits means there's no accuracy left, basically. The matrix does have an inverse, but, at some point, a responsible algorithm will tell you that you are losing more accuracy than your data has rather than return a result that is almost certainly meaningless and letting subsequent calculations (including your analysis of the results) continue on unaware of this problem. That is what "computationally singular" means in this context.

The condition number is a function of the matrix $A$, not of the algorithms operating on $A$, so, for example, changing to a different linear solver won't change the condition number.

A system that is exactly singular is one for which some of the columns are linear combinations of each other; in this case, the matrix does not have an inverse at all.

Improve this answer
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.