# Difference between logistic regression models for classification problems

In various papers, I had often seen the logistic regression model for classification problems written in two forms.

$$p(y =\pm1|\mathbf{x},\mathbf{w}) = \sigma(y\mathbf{w}^{T}\mathbf{x}) = \frac{1}{1+\exp(-y\mathbf{w}^T\mathbf{x})} \tag 1$$

and

$$p(y =1|\mathbf{x},\mathbf{w}) = \sigma(\mathbf{w}^{T}\mathbf{x}) = \frac{1}{1+\exp(-\mathbf{w}^T\mathbf{x})} \tag 2$$

The first form has the label being multiplied to the dot product of $\mathbf{w}$ and $\mathbf{x}$ and the second doesn't. Are both the forms are equivalent? If not, how do they differ?

References:

1. Minka, T.P. (2003). Algorithms for maximum likelihood logistic regression (pdf)
2. Abu-El-Haija, S. Derivation of logistic regression (pdf)
• I don't think the first form is correct. After all, this would mean that to determine the probability of whether y is is +/- 1, one would have to know y.
– meh
Dec 22, 2017 at 14:41
• @aginensky i have added some examples where they are used. Dec 22, 2017 at 14:44
• @morpheus - You tricked me :) . Your second formula only considers the case $y = 1$. I hadn't noticed that. Of course this means that $P( y = -1) = 1- P(y = 1))$ and as Lii observed, this gives the correct answer.
– meh
Dec 22, 2017 at 19:09

They are equivalent, so both are correct.

It can be proved simply by checking the probabilities for $Y= +/- 1$

In the first model:

$$P(Y=1 |x,w) = \frac{1}{1+\exp(-w^Tx)} = \frac{\exp(w^Tx)}{1+\exp(w^Tx)}$$

$$P(Y=-1 |x,w) = \frac{1}{1+\exp(w^Tx)}$$

In the second:

$$P(Y=1 |x,w) = \frac{1}{1+\exp(-w^Tx)} = \frac{\exp(w^Tx)}{1+\exp(w^Tx)}$$ $$P(Y=-1 |x,w) = 1 - P(Y=1 |x,w) = \frac{1}{1+\exp(w^Tx)}$$

Equivalent.

Note:

The first model requires $y$ to be labeled with $+/- 1$.

In the second, $y$ can be any label, as long as it's binary, but usually a 1/0 coding is used.