1
$\begingroup$

In Andrew Ng's Coursera Machine Learning class or Logistic Regression.he cost function for logistic regression is defined as:

$$ J({\theta}) =\frac{1}{m} \sum_{i=1}^m Cost(h_{\theta}(x^{(i)},y^{(i)}) $$

$$ \begin{equation} Cost(h_{\theta}(x),y)=\begin{cases} -\log(h_{\theta}(x)), & \text{if $y=1$}.\\ -\log(1-h_{\theta}(x)), & \text{if $y=0$}. \end{cases} \end{equation} $$ Then It is said that this simplifies to:

$$ J(\theta) = - \frac{1}{m} \displaystyle \sum_{i=1}^m \left[y^{(i)}\log \big(h_\theta (x^{(i)})\big) + (1 - y^{(i)})\log \big(1 - h_\theta(x^{(i)})\big)\right] \\[6pt] $$

In this blog under why this cost function? she had given the Maximum Likelihood Explanation which I did not understand. even this, the accepted answer has the last line as Two loss functions can be derived by maximizing likelihood function.

Can anyone give me proper mathematical(Maximum Likelihood Explanation) proof of how we arrive at the following cost function?

$$ \begin{equation} Cost(h_{\theta}(x),y)=\begin{cases} -\log(h_{\theta}(x)), & \text{if $y=1$}.\\ -\log(1-h_{\theta}(x)), & \text{if $y=0$}. \end{cases} \end{equation} $$

$\endgroup$
1
  • $\begingroup$ This is my blog on Logistic Regression here you will find the answer in 10.Logistic Regression from Statistical point of view. $\endgroup$ – anil May 1 '20 at 21:24