5
$\begingroup$

Structural equation models sometimes cannot be fit due to a singular sample covariance matrix. Now some authors suggest to apply a "gentle ridging" to the diagonal which helps (this is for example done in Mplus).

  1. How does ridging exactly work? and
  2. If your preferred SEM software does not support ridging, how can you do it "by hand"?
$\endgroup$
4
$\begingroup$

I can kill two birds with one stone by explaining how you get an ols software to fit ridge regression for a fixed ridging parameter. You simply augment the data set you have with one observation for each beta you intend to ridge but set the covariate pattern to be all zero except for the ridging variable which gets set to the square root of the ridging parameter.

This includes the intercept. So you need to define your own intercept variable as it will have zeros in the data augmented version.

Also you set the data augmented "response" variable to whatever you want the betas to be shrunk towards. Typically these are all zero.

If you do the maths on the augmented data set, you'll get the ridged beta estimate.

You will also find that the augmented x matrix is now full column rank. I can add the maths if you want but I don't think you need it.

Note that this only helps with understanding for fixed value of the ridging parameter. Choosing a good value is another issue.

$\endgroup$
  • 1
    $\begingroup$ The paucity of capital letters leads me to suspect you're posting from a mobile device, again. :-) In Hastie, et al., there is a problem which leads the reader through this data-augmentation interpretation. $\endgroup$ – cardinal Jul 30 '12 at 13:51
1
$\begingroup$

If you have a singular sample covariance matrix, it means that your variables are perfectly collinear, or one of the variables does not vary. I would say you have to reconsider your model, rather than to try to tweak something in the black box called Mplus to make it kinda work.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.