The original derivation

  • How does the y term vanish/get cancelled ?

Shouldn't it be this instead, h here is the Sigmoid function.

The proposed equation

  • $\begingroup$ I am trying to understand how the y term vanished in the derivation. Help much appreciated. $\endgroup$ – Sud K Dec 23 '16 at 20:10

The expression is correct but only for logistic regression where the outcome is +1 or -1 [ie y(i) = 1 or -1].

If y(i) = 1 or -1, y(i)*y(i) is always one.

You can expand and simplify the h(theta) expressions to show: H(theta)[-y(i)x(i)]{1-H(theta)[-y(i)x(i)]} = 1/{1+exp[-y(i)x(i)]} * 1/{1+exp[y(i)x(i)]} = 1/{1+exp[-x(i)]} * 1/{1+exp[x(i)]} if y(i) is 1 or -1.

1/{1+exp[x(i)]} * 1/{1+exp[x(i)]} is equal to the last the h(theta) expressions in the original photo, and given that y(i)*y(i) is always one, this proves your second expression is equal to the first in the special case when y(i) is 1 or -1.

Hope this helps.

  • $\begingroup$ Hey, Thanks. I already found the answer here. It concurs with what you've said. $\endgroup$ – Sud K Dec 30 '16 at 19:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.