Consider the following MWE example. A dataset with only one feature (categorical feature with 4 different categories ['cat', 'dog', 'hamster', 'frog']) + target (overall 10% positive class). After ohe and dropping one of the 4 resulting columns, e.g.,
is_cat is_dog is_hamster Target
1 0 0 1
0 1 0 0
0 0 1 1
0 1 0 0
1 0 0 0
.
.
.
1 0 0 0
Let's suppose also that the average target for each category is 'cat':15%, 'dog':10%, 'hamster':8%. Now, consider a Logistic Regression $log(\frac{p}{1-p}) = \beta_0 + \sum_i \beta_i x_i$ fitted with the data above. The obtained estimates are:
$\beta_0=-1.53$, $\beta_{dog}=-0.65$, $\beta_{cat}=-0.2$, $\beta_{hamster}=-0.9$
However, I would expect the estimates on the log scale to be
negative for hamster (decrease the odds)
zero for dogs (no difference with the baseline)
positive for cats (increase the odds)
However, the 3 coefficients for the fitted curve show a negative sign, so either the approach or the interpretation is not correct. Is that the right way to interpret the coefficients?
is_dog
from the model? $\endgroup$