# Logistic regression iterative process [duplicate]

I don't understand something very fundamental about Logistic Regression.

For Linear Regression and for Naive Bayes there is a closed form for calculating parameters.

For some reason, logit regression doesn't have such a form. Do you know why?

## marked as duplicate by Nick Cox, Glen_b♦, Scortchi♦, Andy, kjetil b halvorsenMar 5 '15 at 11:20

• I edited your "LR" to logit regression. Although it is evident what you mean, "LR" could also mean Linear Regression, especially given your own capitalisation. – Nick Cox Mar 5 '15 at 9:37

In the case of OLS, $f(\beta)=\frac12\|y - X\beta\|_2^2$, the gradient is $\nabla f(\beta)=X^T(X\beta - y)$. The first order condition is $\nabla f(\beta)=0$. Thus, at the optimum, $X^T(X\beta - y)=X^TX\beta - X^Ty = 0$. Solve for $\beta$ as $\beta=(X^TX)^{-1}X^Ty$. Thus, we are able to analytically solve for the optimum.
Update: Thanks to the comment of Nick, I will be more clear: The partial derivatives of the log-likelihood function for logistic regression are: $$\frac{\partial}{\partial\beta_j}=\sum_{i=1}^n\left(y_i - \frac{1}{1+e^{-x_i^T\beta}}\right)x_{ij},$$ where thus each partial derivative depends on $\beta$ in a highly non-linear way (as a sum of non-linear functions). The first-order condition of these set of partial derivatives, i.e. that $\frac{\partial}{\partial\beta_i}=0$ for all $i$, does not have an analytical solution. I do not have a reference to a proof, perhaps someone can add this is the comments. But it should be clear that this is a difficult problem.