Ridging a singular covariance matrix (SEM)

Question

Structural equation models sometimes cannot be fit due to a singular sample covariance matrix. Now some authors suggest to apply a "gentle ridging" to the diagonal which helps (this is for example done in Mplus).

1. How does ridging exactly work? and
2. If your preferred SEM software does not support ridging, how can you do it "by hand"?

Answer 1

I can kill two birds with one stone by explaining how you get an ols software to fit ridge regression for a fixed ridging parameter. You simply augment the data set you have with one observation for each beta you intend to ridge but set the covariate pattern to be all zero except for the ridging variable which gets set to the square root of the ridging parameter.

This includes the intercept. So you need to define your own intercept variable as it will have zeros in the data augmented version.

Also you set the data augmented "response" variable to whatever you want the betas to be shrunk towards. Typically these are all zero.

If you do the maths on the augmented data set, you'll get the ridged beta estimate.

You will also find that the augmented x matrix is now full column rank. I can add the maths if you want but I don't think you need it.

Note that this only helps with understanding for fixed value of the ridging parameter. Choosing a good value is another issue.

Answer 2

If you have a singular sample covariance matrix, it means that your variables are perfectly collinear, or one of the variables does not vary. I would say you have to reconsider your model, rather than to try to tweak something in the black box called Mplus to make it kinda work.