How to compare parameters between different types of generalized linear model?

Question

I am doing some work on the effects of collinearity on different types of model (OLS, binomial logistic, ordinal logistic, multinomial logistic and maybe others). I have found the perturb package in R, which will be very useful for this. This package shows how small changes in the data affect the parameter estimates of a model. The perturb package perturbs the data that is used to build the model, and then re-runs the model - a little like a bootstrap. In this particular case, I chose to perturb the independent variables, but I could also perturb the dependent variable.

However, I am struggling to figure out the best way to compare these effects. For example, I ran perturb on an OLS model and got

----------


 Impact of perturbations on coefficients:

                 mean  s.d.    min   max

    x0          0.014 0.264 -0.743 0.604

    x1          1.451 0.347  0.771 3.772

    x2          0.847 0.321  0.308 2.817

    x3          2.222 0.243  1.129 2.746
    ----------

Then I dichotomized the DV and ran perturb on a logistic model and got

----------


 Impact of perturbations on coefficients:

                 mean  s.d.    min    max

    x0          0.119 0.234 -0.477  0.732

    x1          0.918 0.819 -6.612  2.215

    x2          0.247 1.008 -9.413  1.175

    x3          1.533 2.292  0.574 24.083
----------

The latter certainly looks like it was affected much more, but I don't know of any more formal ways to compare these parameter estimates and, especially, the size of the sd of the different estimates.

Here is a link to the perturb package and to the R version

Thanks

Answer 1

I'm not familiar with the perturb package. I think what you're looking for is a sensitivity analysis to determine the extent of collinearity on the estimated model parameters or their predicted outcomes.

If you're interested in prediction, I would look at ridge regression and show cross validated MSE over a range of tuning parameters for the L2-penalty. It should have a quadratic shape whose minimum represents an estimated optimal L2-penalty.

If you're interested in inference, well then the model is part of a prespeficied analysis plan, so your adjustment variables are necessary. You wouldn't want to change your inference approach based on the data, otherwise your statistics become exploratory instead of confirmatory. A reasonable sensitivity analysis would be to calculate simple correlations between predictors and show conditioning plots with range highlighting indicating the spread of your main effect (x axis) over a range of the adjustment variable (w).