How to measure the "Impact/influence" of a feature y on Logistic regression model based on the coefficients?

Question

I have a dataset X which contains probabilities returned for classification 4 different classification models, say M1, M2, M3, M4 those probabilities are use to feed a fourth model M4 and that model also returns probabilities between (0, 1)

The 4 models above try to measure the probability of a customer to fall in default, and each of them measure different characteristics of customers.

So mi table X looks like:

M1 | M2| M3| M4 | y
.5 | .7|.3 | .6 | 1
.3 | .6|.2 | .7 | 0
.7 | .1|.2 | .1 | 1
.1 | .2|.4 | .1 | 0
.6 | .7|.1 | .6 | 1
.2 | .9| 0 | .5 | 0

I have another binary variable y for which I want to measure the impact of every model on y so I want to find a measure of impact /influence of each model on this variable y

What I tried so far was to run a logistic regresion on X to explain y say I obtaine the coeffients:

Intercept : -3.74587192 b1 : 4.647923 b2 : -0.354599 b3 : 2.984094 b4 : 8.983295

Q1: Haw can I find a measure of influence on y based on the Logistic regression coefficients?

Q2: Is it possible to establish a relationship of relative importance of each feature based on the coefficients?

Answer 1

If you scale each feature to have the same standard deviation (i.e. divide each column of the model matrix by its standard deviation), then the magnitude (absolute value) of the coefficients is a direct measure of the importance of the corresponding features. See Schielzeth 2010 Methods in Ecology and Evolution "Simple means to improve the interpretability of regression coefficients" https://doi.org/10.1111/j.2041-210X.2010.00012.x.

Answer 2

To add to the other answer, we have $P(Y=1|x_1,..,x_p)=sigmoid(b_0+b_1x_1+...+beta_px_p).$ the inverse of sigmoid is log odds (aka logit).

Note that

$$log(odds(Y=1))=b_0+b_1x_1+...+b_px_p$$

and hence for coefficient $b_i$ you have that the log odds changes by $b_i$ With an increase in $x_i$ of one unit holding all other variables constant (which is only possible if $x_1,...,x_p$ are independent).

So a coefficient’s absolute magnitude dictates the effect of its corresponding feature on $Y$. Therefore the absolute magnitude of the coefficient is related to its importance. To some extent this answer depends on your definition of influence and of importance.