Why does logistic regression's likelihood function have no closed form? The derivative of Log likelihood function of logistic regression with respect to theta is 



*

*Why can't we equate it to zero and solve for theta so that we can obtain a 'closed form solution' for theta?
Is it because of the non linearity of theta in the equation?

*Can OLS method in linear regression which is a closed form solution, be categorized as a 'linear optimization' technique? (unlike gradient descent which is a 'non-linear optimization' method?)
 A: *

*Indeed, when using a binomial logistic model, the estimate is the solution of the equation : 
$$\sum_{i}[y_i - \sigma(\theta^T \boldsymbol{x}_i)] \boldsymbol{x}_i = 0$$
Unfortunately (you can try), this equation is not solvable, that's why it's said that there is no closed form solution for $\theta$. One must use optimisation technique to numerically approximate a solution (for logistic regression, Newton-Raphson algorithm works fine since likelihood is concave). Non linearity does complicate the equation, but there are some estimators which are closed form and solution of non linear equations, for example the median solves the non linear equation : $\sum_i [\mathbb{1}_{x_i > \theta} - \mathbb{1}_{x_i < \theta}] = 0$. Linear estimating equations are solvable but there are not the only ones.

*I am not sure about the question. Indeed the estimating equation of logistic regression is non linear in $\theta$ and the estimating equation of an OLS is linear (and thus easily solvable). But if you refer to linear programming (LP) by "linear optimization", then OLS is not a LP, since the solution of OLS equation is a minimizer of a quadratic equation. OLS is a quadratic programming (QP). Be carefull not to confuse "linear estimating equation" with linear programming.
