I am trying to do an ordinal logistic regression (ordinal outcome variables with more than 2 categories) with nominal (more than 2 categories for some) predictor variables, as well as nominal (more than 2 categories for some) covariates/moderators. First of all, I would like to test for effect interaction to see which of my covariates/moderators are indeed moderators so that I can treat them as such. Second of all, I would like to know what's the best way to come up with the optimal model? Do I need to do dummy variables for the covariates? Can I do a stepwise for ordinal regression?
$\begingroup$
$\endgroup$
1
-
$\begingroup$ I advise against stepwise model selection (you can see my answer here: algorithms-for-automatic-model-selection). In general, I think finding the "optimal model" is overrated. If you want to build a predictive model, you should look into cross validation. $\endgroup$– gung - Reinstate MonicaOct 30, 2012 at 21:12
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
5
You shouldn't use stepwise for any kind of model building. Stepwise results have parameters biased away from 0, standard errors and p-values that are too small, models that are too complex - all in ways that are difficult if not impossible to control.
In ordinal regression, as in any other type, the best way to build a model is to use substantive knowledge. Barring that, for the main effects and the interactions, you should look at effect sizes (you can do this effectively with categorical IVs by outputting the predicted value for each combination of IVs).
In some cases, model averaging can be a good solution (especially if your main goal is prediction rather than explanation). If you must use an automatic procedure, use one that penalizes for the complexity of the model (e.g Lasso, LAR).
answered Oct 30, 2012 at 21:14
-
$\begingroup$ thank you for your reply M Peter Flom, I wasn't referring to automatic stepwise, but manual stepwise by taking out manually the least significant ones in turn. I am not familiar with LASSO/LAR? Is that in SPSS? I am only working with SPSS. Also, since I am not an expert in SPSS would you mind explaining a bit better what do I look for at effect sizes when I decide on my optimal model? Thank you! $\endgroup$– AndreeaOct 30, 2012 at 21:29
-
$\begingroup$ I don't know
SPSSat all. But the parameter estimates are effect sizes. I am sureSPSSlets you output predicted values too, but I don't know how. Manually doing what the computer could do faster doesn't make it better. $\endgroup$ Oct 30, 2012 at 21:37 -
3$\begingroup$ "You shouldn't use stepwise for any kind of model building." - this is an incredibly sweeping statement and it just isn't true. If your goal is prediction then a stepwise selection algorithm may be quite sensible. Yes, stepwise selection destroys the interpretation of $p$-values but inquiries where you care about $p$-values does not subsume all model building. Dogma and unconditional statements like this do not spread good statistical practice. $\endgroup$– MacroOct 31, 2012 at 2:20
-
$\begingroup$ The parameter estimates are also wrong in stepwise. The only case where it might be OK is where you do it on a training set and then apply the model to a test set. $\endgroup$ Oct 31, 2012 at 10:42
-
$\begingroup$ @Marco, it is not just that the $p$-values aren't correctly interpreted anymore. Let's assume, as the user asking the question suggests, that we eliminate those predictors that aren't significant. A problem with this is that we are eliminating those predictors for which we've underestimated their significance, and keeping those for which we've overestimated their significance. $\endgroup$ Dec 30, 2012 at 15:16