When is it appropriate to override the default reciprocal condition number tolerance for solve() in R?

I am estimating a GMM IV model, where I'm creating a weighting matrix by taking the inverse of Z'Z, where Z is a matrix of instruments. For certain combinations of instruments, when I try to compute this in R with the solve() function, I get an error along the lines of:

system is computationally singular: reciprocal condition number = 4.96805e-21


Sometimes there's an obvious reason for the singularity (i.e., I've included something stupid in the instrument set), but sometimes there's no obvious reason for it. Now, I know I can override the error by setting the tol option to something smaller. What I want to know is: under what circumstances is this a reasonable thing to do (for instance because R's default tolerance is too high for my type of problem), and under what circumstances is this going to get me in trouble?

• It's best to spell out your abbreviations, like "GMM" and "IV", the first time you mention them, so it's clearer what you're referring to. For instance, I suspect in your case "IV" means "instrumental variable" but "IV" is also regularly used to refer to an "independent variable". Another advantage of doing this is that someone searching for "instrumental variable" can find your thread. (It would also help if you add a few more relevant tags - while it's true this is a problem about matrix inverses, that's not the only thing going on here.) – Silverfish Sep 16 '16 at 14:11

Let $A$ be the matrix that R claims is computationally singular. For the purpose of this answer lets assume that it is really mathematically non-singular but close to singular (otherwise lowering the tolerance will do nothing). Let $\epsilon$ be the tolerance in use. For condition numbers see: https://en.wikipedia.org/wiki/Condition_number. The condition number is a (maximal bound on) the quotient between relative error in result and relative error in the data. That is, it is a kind of derivative! But since a matrix can be changed in many different directions, this derivative could (will) depend on in which direction you perturb the matrix, and the effect on the relative error in solution could be large. If you are lucky, the dependence for the directions that are relevant to you could be small! The condition number is then the maximum of all this directional derivatives. The condition number given in your R output is about $\kappa =.5\cdot 10^{21}$.
In R all calculations are done in double precision, so machine epsilon eps is about $10^{-16}$. Suppose you change the input data with about an eps, then the output could change with as much as $\text{eps}\cdot \kappa= .5\cdot 10^{5}$. So you could ask yourself if you are comfortable with that!
For the case of linear models, there is an R package on CRAN, perturb, that helps with this.