Simple explanation of maximum likelihood estimation

Question

I was watching a video from code emporium, here he describes Maximum Likelihood Estimation (MLE) as to finding weights which maximizes the probability of seeing the training data (D = [X,Y])

$\theta^{MLE} =argmax \{ P(Y|X;\theta) \}$

Let's consider our training data is XOR data

Input   Output
A   B
0   0   0
0   1   1
1   0   1
1   1   0

and say this data's real probability distribuiton is the blue one $P(Y|X)$ and our current model's (randomly initialized weights) probability disbribution is the red one $P(Y|X;\theta_{Random})$

Then the aim of Maximum Likelihood Estimation is to push the weights from randomly initialitzed weights to the real weights which approximately "describes" our real data distribution.

MLE = $ \{ P(Y|X;\theta_{Random})\rightarrow P(Y|X;\theta_{Real})\approx P(Y|X)\}$

Question 1: Is my thinking right; Is this the correct approach to describe MLE?

In gradient descent, to train a model we have to minimize a cost function but in MLE we are maximizing it. To solve this in here he talks about Negative Log Likelihood which is a negative added to the log converted MLE?

Question 2: How is adding a negative to log converted MLE maximises MLE?

My intiution tells me instead of learning the real distribution $P(Y|X;\theta_{Real})$ this should have pushed away the weights to some random distribution?

Answer 1

Maximum likelihood is a method for estimating parameters by maximizing the probability of the observed data. The main ingredients are:

• The data: $D=(X,Y)$
• The model parameters: $\theta$
• The model that relates data to the parameters: $P(D|\theta)$ (which can be written differently, depending on the situation: $P(Y|X, \theta)$ is one of the possibilities.)

One would typically name the probability of observing data given parameters likelihood and write it as $L(\theta|D)=P(D|\theta)$. This is just a change of notation. One then maximizes the likelihood in respect to the values of the parameters: \begin{equation} \hat{\theta}=\mathrm{argmax}_\theta L(\theta|D). \end{equation}

Quite often the likelihood is a product of many identical functions for different data points. In this case it is mathematically more convenient to maximize its logarithm, i.e. the log-likelihood: $LL(\theta|D) = \log L(\theta|D)$. The likelihood and its logarithm has the same maximum, since logarithm is a monotonous function. Finally, maximizing a function is the same as minimizing its negative, therefore minimizing a negative log-likelihood is the same as maximizing the log-likelihood, which is the same as maximizing the likelihood. This is important since the optimization algorithms are often spelled explicitly only for minimization (or only for maximization).