I would recommend you to read this post (if not duplicate question)
Why there are two different logistic loss formulation / notations?
The highlight is that
Using 0, 1 representation is more natural and can be easily related to probability.
Using -1, +1 is more concise for some cases (such as hinge loss or zero one loss). The reason it is concise is ...
If you have a binary variable coded as -1/1, this has the benefit of making the linear combination of coefficients for that category a contrast. That may be nice for statistical models, but for machine learning my opinion is that 0/1 is better.
Not only does 0/1 make interpretation of means and variances easier, but it also makes interpreting conditions of ...
Let us to use $X$ to represent the feature and $Y$ to represent the label. Essentially, if $P(Y|X)=P(Y)$ or $X$ and $Y$ are independent, we can drop $X$.
What you described
feature has zero variance only for cases that belong to one class but not in the other
Just tells this feature is an important feature that can differentiate different classes, i.e....
Even if that feature have zero variance for one of the classes (or even if it has zero variance for both classes!), it could still have very different values between the classes, so be a good discriminator. You should probably keep it!
There is nothing specifically about one-hot encoding that makes ANOVA inappropriate.
Most software (certainly R and SAS) does some version of coding of categorical variables for you. With R and SAS (and maybe others) you can choose among various parameterizations for categorical variables (effect coding, dummy coding, Helmert contrasts etc.)
The z test of proportions assumes your samples are independent (your single index $i$ for both $X$ and $Y$ and your indication of a regression context implies that these are paired, not independent observations).
Therefore, McNemar's test (1947) is an appropriate test for association in paired binary data:
The positivist null hypothesis is that $X$ and $Y$ ...
The fact that your algorithm is giving very low probabilities when asked to predict whether an unlabelled example belongs to a given class, should not necessarily be seen as a bad thing. Over- and under-sampling will mean the algorithm assigns higher probabilities..but these are incorrect.
Consider the following, you have a data set of medical records, and ...
Tan et al, in Information Systems 29 (2004) 293-313, consider 21 different measures for association patterns between 2 binary variables. Each of these measures has its strengths and weaknesses. As the authors state in the Abstract:
Objective measures such as support, confidence, interest factor, correlation, and entropy are often used to evaluate the ...