What is the difference between Maximum Likelihood Estimation & Gradient Descent? What are the pro & cons of both the methods?
 A: Maximum likelihood estimation is a general approach to estimating parameters in statistical models by maximizing the likelihood function defined as
$$ L(\theta|X) = f(X|\theta) $$
that is, the probability of obtaining data $X$ given some value of parameter $\theta$. Knowing the likelihood function for a given problem you can look for such $\theta$ that maximizes the probability of obtaining the data you have. Sometimes we have known estimators, e.g. arithmetic mean is an MLE estimator for $\mu$ parameter for normal distribution, but in other cases you can use different methods that include using optimization algorithms. ML approach does not tell you how to find the optimal value of $\theta$ -- you can simply take guesses and use the likelihood to compare which guess was better -- it just tells you how you can compare if one value of $\theta$ is "more likely" than the other.
Gradient descent is an optimization algorithm. You can use this algorithm to find minimum (or maximum, then it is called gradient ascent) of many different functions. The algorithm does not really care what is the function that it minimizes, it just does what it was asked for. So with using optimization algorithm you have to know somehow how could you tell if one value of the parameter of interest is "better" than the other. You have to provide your algorithm some function to minimize and the algorithm will deal with finding its minimum.
You can obtain maximum likelihood estimates using different methods and using an optimization algorithm is one of them. On another hand, gradient descent can be also used to maximize functions other than likelihood function.
A: Usually, when we get likelihood function $$f = l(\theta)$$, then we solve equation $$\frac{ df }{ d\theta } = 0$$.
we can get the value of $$\theta$$ that can give max or min value of f, done!
But logistic regression's likelihood function no closed-form solution by this way. So we have to use other method, such as gradient descent.
