3
$\begingroup$

I have a matrix, $X'X$, which is singular meaning that I cannot invert it. I need the inverse of this matrix to perform two independent things. I need it for the design of experiments, in R using optFederov() algorithm. Also I need it invertible for OLS.

I believe the two possible sources of singularity are:

1.- Many points lying in the same plane

2.- There are linear combinations between columns

I think it is the first problem. To solve the problem I have added very small random noise to each value of x, $x_{i}$, from a Normal Distribution such that $x_{i,noise} \sim N(0,0.001)$.

Apparently this trick makes the matrix invertible. However, I am not sure if this is a good approach. My thought is that it is good for OLS given that the esitmators will change veeery little, but not sure on the design of experiments, for example using a D-Efficiency design.

I would appreciate some suggestion regarding whether this is a good way to proceed or not!

Improve this question
$\endgroup$
4
  • $\begingroup$ Presumably $X$ is the design matrix. $\endgroup$
    – Scortchi ♦
    Commented Mar 30, 2017 at 8:30
  • $\begingroup$ @Scortchi yes, X is the full factorial design, but in order to obtain a D-optimal fractional factorial design I have to minimize $|(X'X)^{-1}|$ which indeed requires the inverse of $X'X$ $\endgroup$
    – adrian1121
    Commented Mar 30, 2017 at 8:36
  • $\begingroup$ en.wikipedia.org/wiki/Moore%E2%80%93Penrose_pseudoinverse ? $\endgroup$
    – Dikran Marsupial
    Commented Mar 30, 2017 at 11:23
  • 2
    $\begingroup$ If it is singular certainly it cannot be Doptimal, unless tjere are so few points that all designs are singular. You shpuld splve that problem rather than just perturb $\endgroup$
    – kjetil b halvorsen ♦
    Commented Mar 30, 2017 at 13:49

1 Answer 1

Reset to default
2
$\begingroup$

Rather than random noise, adding a small multiple of the identity matrix is the standard way to handle this in regression problems. It's equivalent to ridge regression. I.e $X^\prime X+\epsilon I$ is used in place of $X^\prime X$, for $\epsilon$ small, say 0.001 or 0.01. I'm not sure about the experimental design aspects.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Yes, but it wouldbebetter tofind out what whent wrong $\endgroup$
    – kjetil b halvorsen ♦
    Commented Mar 30, 2017 at 13:51
  • 1
    $\begingroup$ This is a good point, but it would be well to explain that "$\epsilon$ small" is meaningful only when the columns of the design matrix have been standardized. $\endgroup$
    – whuber ♦
    Commented Mar 30, 2017 at 14:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.