1
$\begingroup$

Given a set of features, how do I find what all features are contributing how much to the accuracy / prediction ?

$\endgroup$
2
$\begingroup$

When using one of the gold standards, the maximum likelihood, we have an ideal optimality criterion and associated proper accuracy scoring rule (deviance/log likelihood/measures derived from these such as $R^2$). This means that you have available

  1. The combined influence of all predictors, from for example the model likelihood ratio $\chi^2$ statistic
  2. Partial contributions of predictors such as partial likelihood ratio $\chi^2$ test statistics or approximations of them such as partial Wald tests (in the simplest case the square of a single regression coefficient divided by the square of its standard error). Partial $R^2$ can be derived from these.

One nice and fast way to present things while allowing for combination of multiple terms per predictor is the proportion of $\chi^2$ explained by each predictor. See Relative importance of variables in Cox regression and the comment about bootstrapping ranks of predictors, which exposes the true difficulty of this task.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.