0
$\begingroup$

Simplified, I have 17 items that are barriers to helping. Each item is rated on a 5-point Likert scale. I have four outcomes that are different types of helping (3 binary outcomes, 1 continuous). I want to determine the relative importance of these barriers in explaining each of these helping behaviors. I would prefer to compare the barrier items themselves and not factor analyze the items. I have been reading about different variable selection and importance methods (here, here, or here).

I am learning toward a relative importance method (relaimpo/relimp) or one using a random forest method (randomForest, Boruta). Are either of these (or any other variable selection or importance methods) better/more robust given I have 5 point Likert items as predictors?

$\endgroup$
1
$\begingroup$

I don't think either is more or less relevant or robust for five-point Likert-type scales. The key consideration is more focused on whether any model selection has occurred.

Overall, the two methods are fairly different in terms of how they approach importance but can result in similar results as is discussed by Groemping here.

The Shapley/Dominance/LMG method as discussed by Groemping assumes model selection has been completed. Thus, LMG method assumes the helping behaviors used as features/predictors are valid predictors and will assign non-zero importance to all features.

The PMVD and RF methods discussed by Groemping do not assume such selection has taken place and will assign zero importance to truly non-relevant features. This latter approach might fit your case better as it sounds like no other modeling has been completed based on your description.

Reference:

Grömping, U. (2009). Variable importance assessment in regression: linear regression versus random forest. The American Statistician, 63(4), 308-319.

$\endgroup$
  • $\begingroup$ Thank you for the link and information. I agree after reading the RF methods seem the most appropriate, with conditional permutation importance given that my predictors are highly correlated in some cases. $\endgroup$ – Mark Aug 5 at 14:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.