2
$\begingroup$

In package relaimpo in R, we measure the relative importance with different methods. However, the $R^2$ for all the methods is similar.

Is it supposed to be similar or a bug?

$\endgroup$
2
  • $\begingroup$ What do you measure? For what methods? What is your data, what models do you compare? $\endgroup$
    – Tim
    Sep 13 '16 at 7:48
  • $\begingroup$ I measure the relative importance of some variables. Relaimpo package has a range of methods which are c("lmg", "pmvd", "last", "first", "betasq", "pratt", "genizi", "car"). I hoped that using different methods give me different coefficient of determination but it is not true. Wondering if I'm missing something or it is a bug $\endgroup$
    – MFR
    Sep 13 '16 at 7:53
2
$\begingroup$

$R^2$ measures fit of the model, not of particular variables.

From the documentation (bold added by myself):

Five of the metrics in calc.relimp (lmg, pmvd, pratt, genizi and car), decompose the model $R^2$. calc.relimp (lmg, pmvd, pratt, genizi and car) sum to the $R^2$ that is to be decomposed [...]

You can find out more about the methods from the references provided in the documentation.

Notice however that $R^2$ can be misleading and does not really tell you about "variance explained" unless you are using linear regression.

$\endgroup$
4
  • $\begingroup$ Thanks for the quick reply, do you have any idea if there is a way to compare theses methods? In my case, there are different weights in different methods? How do I know which one is more accurate? $\endgroup$
    – MFR
    Sep 13 '16 at 8:07
  • $\begingroup$ What models? You use "methods", "models" and "variables" interchangeably what makes it unclear what do you mean. Do you compare different models? What models? $\endgroup$
    – Tim
    Sep 13 '16 at 8:10
  • $\begingroup$ @Eddie: The different importance metrics each purport to replace our fuzzy, qualitative notion of "importance" with something exact & quantitative. The question of their accuracy doesn't arise. $\endgroup$ Sep 13 '16 at 8:16
  • $\begingroup$ Thanks Scortchi, It's making more sense to me now. How people know which metrics they need to use in practice? Is it anyway to make these metrics actionable? $\endgroup$
    – MFR
    Sep 13 '16 at 22:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.