0
$\begingroup$

I've trained a logistic regression over my data. I checked feature importance:

from matplotlib import pyplot

features = X_train.columns
importance = Model.best_estimator_.coef_[0]

plt.bar(features, importance)
plt.title("Feature Importance according to logistic regression")
plt.ylabel("Improtance")
plt.show()

enter image description here

and the importance could be seen also here

Model.best_estimator_.coef_[0]

array([1.09365005e+00, 7.50386093e-01, 4.29639078e-04, 5.99721148e-01])

My question is what is the meaning of these results? what are those numbers? I thought it was percentages before but 1.09365005e+00 means more than 100% importance, so it can't be percentages

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ Aren't these the estimated coefficients? In what sense do you conceive of them as being "feature importance"? $\endgroup$
    – whuber ♦
    Jan 2 at 23:23
  • 1
    $\begingroup$ These are estimated coefficients. $\endgroup$
    – gunes
    Jan 2 at 23:24
  • 1
    $\begingroup$ Because these are coefficients, their interpretation as importance is at best nuanced and at worst dubious, depending on who you ask & what your data represent. One source of caution is that the coefficients are expressed as units of the target per units of the independent variable, so comparing coefficients means comparing incomparable units of measurement. Re-expressing an IV in different units (eg. length in cm vs km) will likewise change the coefficient value. For more detail, see the "Race of Variables" section of Gary King, "How Not to Lie with Statistics" (1985). $\endgroup$
    – Sycorax ♦
    Jan 2 at 23:31
  • $\begingroup$ More information in this search: stats.stackexchange.com/… and stats.stackexchange.com/… $\endgroup$
    – Sycorax ♦
    Jan 2 at 23:33

0

Reset to default