I have a binary classification problem from several features. Do the coefficients of a (regularized) logistic regression have an interpretable meaning?
I thought they could indicate the size of influence, given the features are normalized beforehand. However, in my problem the coefficients seem to depend sensitively on the features I select. Even the sign of the coefficients changes with different feature sets chosen as input.
Does it make sense to examine the value of the coefficients and what is the correct way to find the most meaningful coefficients and state their meaning in words? Are some fitted models and their sign of the coefficients wrong - even if when they sort-of fit the data?
(The highest correlation that I have between features is only 0.25, but that certainly plays a role?)