# Confused about logistic regression equality

Problem: Prove that: \begin{align} \Delta E(in) &= -\frac{1}{N} \sum_{n=1}^N \frac{y_n x_n}{1 + e^{(y_n w^t x_n)}} \\[10pt] &=\frac{1}{N} * \sum_{n=1}^N - y_n x_n \theta (-y_n w^T x_n) \end{align}

Problem Description: I'm trying to prove this equality but I'm finding that my answer is not making any sense to me. Does anyone have a clue about this equality?

My doubt is about $\theta$, if theta is the log regression, then how the $...+e^{-...}$ is supposed to become positive? $...+e^{+...}$

Also, $e$ from the logistic regression, has only -x, and in the formula there are a lot of variables!

On The Equality

The denominator is being 'powered' by $y_n w^T x_n$. However, on the logistic regression, $e ^{-x}$. So how is logistic regression supposed to have more variables than X? Unless I consider that X can be anything. Which is obvious to me, but I'm not sure. And about the sign "-/+"

Edit: The questions goes further, inferring that this equality would make "misclassified" examples contribute more to the gradient. However, logistic regression just calculates p(x|y), so how is that supposed to change anything? Also the range is just from $0\to 1$, so negative values wouldn't change either, since they aren't supposed to be inserted anyway.

So what about that? Unless I could add a variable to penalize misclassified values, I fail to see how that equality changes anything.

Conclusion This was my first question here, and I did the tour and learning the rules and applying them! Sorry for anything!

References

Wikipedia - Logistic Regression

• Please add the [self-study] tag & read its wiki. Then tell us what you understand thus far, what you've tried & where you're stuck. We'll provide hints to help you get unstuck. Please make these changes as just posting your homework & hoping someone will do it for you is grounds for closing. – Silverfish Jul 1 '16 at 1:46
• It would also help to include a little more of the context for this question - particularly to define your variables. – Silverfish Jul 1 '16 at 1:46
• Will do @Silverfish ! I just need a bit more time! – KenobiShan Jul 1 '16 at 21:00
• How can i approve the edits? Do i need to aprove anything? Thanks for editing! I really appreciate! – KenobiShan Jul 1 '16 at 21:01

I suppose $θ$ is the logistic function $θ(x) = \frac{1}{1+e^{-x}}$. Now, let's rewrite your initial formula like this (notice I also moved the minus into the summation): $$ΔE(in) = \frac{1}{N}*\sum\limits_{i=1}^N(-y_nx_n * \frac{1}{1+e^{-(-y_nw^Tx_n)}})$$ It is clear that $\frac{1}{1+e^{-(-y_nw^Tx_n)}} = θ(-y_nw^Tx_n)$. Substituting this into the first equation we get the needed equality: $$ΔE(in) = \frac{1}{N}*\sum\limits_{i=1}^N(-y_nx_n * θ(-y_nw^Tx_n))$$ Please ask if you need further help.

• I was stuck with the "-" sign ! , i was going through the wrong way! Now i see everything makes sense and its so simple! Thanks so much! Does this change changes how the gradient will be calculated? – KenobiShan Jul 1 '16 at 17:29