I have a matrix, $X'X$, which is singular meaning that I cannot invert it. I need the inverse of this matrix to perform two independent things. I need it for the design of experiments, in R using optFederov() algorithm. Also I need it invertible for OLS.

I believe the two possible sources of singularity are:

1.- Many points lying in the same plane

2.- There are linear combinations between columns

I think it is the first problem. To solve the problem I have added very small random noise to each value of x, $x_{i}$, from a Normal Distribution such that $x_{i,noise} \sim N(0,0.001)$.

Apparently this trick makes the matrix invertible. However, I am not sure if this is a good approach. My thought is that it is good for OLS given that the esitmators will change veeery little, but not sure on the design of experiments, for example using a D-Efficiency design.

I would appreciate some suggestion regarding whether this is a good way to proceed or not!

  • $\begingroup$ Presumably $X$ is the design matrix. $\endgroup$ Mar 30 '17 at 8:30
  • $\begingroup$ @Scortchi yes, X is the full factorial design, but in order to obtain a D-optimal fractional factorial design I have to minimize $|(X'X)^{-1}|$ which indeed requires the inverse of $X'X$ $\endgroup$
    – adrian1121
    Mar 30 '17 at 8:36
  • $\begingroup$ en.wikipedia.org/wiki/Moore%E2%80%93Penrose_pseudoinverse ? $\endgroup$ Mar 30 '17 at 11:23
  • 2
    $\begingroup$ If it is singular certainly it cannot be Doptimal, unless tjere are so few points that all designs are singular. You shpuld splve that problem rather than just perturb $\endgroup$ Mar 30 '17 at 13:49

Rather than random noise, adding a small multiple of the identity matrix is the standard way to handle this in regression problems. It's equivalent to ridge regression. I.e $X^\prime X+\epsilon I$ is used in place of $X^\prime X$, for $\epsilon$ small, say 0.001 or 0.01. I'm not sure about the experimental design aspects.

  • $\begingroup$ Yes, but it wouldbebetter tofind out what whent wrong $\endgroup$ Mar 30 '17 at 13:51
  • 1
    $\begingroup$ This is a good point, but it would be well to explain that "$\epsilon$ small" is meaningful only when the columns of the design matrix have been standardized. $\endgroup$
    – whuber
    Mar 30 '17 at 14:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.