Im developing some machine learning code, and I'm using the softmax function in the output layer.
My loss function is trying to minimize the Negative Log Likelihood (NLL) of the network's output.
However I'm trying to understand why NLL is the way it is, but I seem to be missing a piece of the puzzle.
From what I've googled, the NNL is equivalent to the Cross-Entropy, the only difference is in how people interpret both.
The former comes from the need to maximize some likelihood (maximum likelihood estimation - MLE), and the latter from information theory
However when I go on wikipedia on the Cross-Entropy page, what I find is:
Question 1: Why are they raising the estimated outcome to the power of the (N * training outcome).
Question 2: Why are they dividing the whole formula by N? Is is just just for convinience like adding the log to the likelihood?
This is what I've got so far:
Thank you for your time, and excuse me if the question is too easy, but I just can't wrap my mind around it.
Math is not my forte, but I'm working on it :)