# Maximize log-likelihood of logistic regression

I'm trying to understand the derivation of the equations for the logistic regression. I'm following the cs229 notes:

http://cs229.stanford.edu/notes/cs229-notes1.pdf

At some point in the derivation, in the notes we need to maximize the log-likelihood of the parameters:

ℓ(θ) = log L(θ) = y log h(x) + (1 - y) log(1-h(x))

In the notes, after the derivation we get:

$$\frac{\partial}{\partial \theta_j}$$ = (y $$\frac 1 {h(x)}$$ - (1 - y) $$\frac 1 {1 - h(x)}$$) $$\frac{\partial}{\partial \theta_j}$$h(x)

Shouldn't this be:

$$\frac{\partial}{\partial \theta_j}$$ = (y $$\frac 1 {h(x)}$$ + (1 - y) $$\frac 1 {1 - h(x)}$$) $$\frac{\partial}{\partial \theta_j}$$h(x)

I've been looking at this for a while and I can't see where this minus sign is coming from...

• $\frac{\partial \log (1-h(x,\theta))}{\partial\theta}=\frac{1}{1-h(x,\theta)}\frac{\partial (1-h(x, \theta) ) }{\partial\theta}=\frac{1}{1-h(x,\theta)}(-1)\frac{\partial h}{\partial\theta}$. For more details I refer to math.stackexchange.com/questions/477207/… Dec 23, 2014 at 11:44
• There is a mistake in the different equations in that there is not $\theta$. Do you mean $h(x^\text{T}\theta)$? Dec 23, 2014 at 14:30
There is little to explain: \begin{align*} \frac{\partial}{\partial \theta_j} \ell(\theta) &= \frac{\partial}{\partial \theta_j} \left\{ y \log h(x) + (1 - y) \log(1-h(x)) \right\}\\ &= y \frac{\partial}{\partial \theta_j} \log h(x) + (1 - y) \frac{\partial}{\partial \theta_j} \log(1-h(x))\\ &= y \frac{1}{h(x)} \frac{\partial}{\partial \theta_j} h(x) + (1 - y) \frac{-1}{1-h(x)}\frac{\partial}{\partial \theta_j} h(x)\\ &= \left( \frac{y}{h(x)}-\frac{1-y}{1-h(x)} \right) \frac{\partial h(x)}{\partial \theta_j} \end{align*} So it is because $$\frac{\text{d}}{\text{d}x}\log(1-x) = \frac{-1}{1-x}$$ that you get this expression.