Given a set of features, how do I find what all features are contributing how much to the accuracy / prediction ?
When using one of the gold standards, the maximum likelihood, we have an ideal optimality criterion and associated proper accuracy scoring rule (deviance/log likelihood/measures derived from these such as $R^2$). This means that you have available
- The combined influence of all predictors, from for example the model likelihood ratio $\chi^2$ statistic
- Partial contributions of predictors such as partial likelihood ratio $\chi^2$ test statistics or approximations of them such as partial Wald tests (in the simplest case the square of a single regression coefficient divided by the square of its standard error). Partial $R^2$ can be derived from these.
One nice and fast way to present things while allowing for combination of multiple terms per predictor is the proportion of $\chi^2$ explained by each predictor. See Relative importance of variables in Cox regression and the comment about bootstrapping ranks of predictors, which exposes the true difficulty of this task.