Let's consider I have a multi layer neural network that is doing multi class classification. So each input sample belongs to one on N classes.
Now, lets say the last layer has Sigmoid activation units. This is followed by a Softmax normalization and then a cross entropy loss.
What is the neural network intuitively trying to do with training? Is is trying to fit the distribution of each of the classes around the input training samples that are labelled with the corresponding class?
As in, if we draw a set of points on a line and then we try to fit a gaussian distribution around the points such that the likelihood of all the points is maximum - is this something the neural network is doing for each of the class?
If so, and if neural networks are basically trying to estimate the parameters of the distribution of each of the classes around the training samples of each of the classes, then why don't we just use a Sigmoid activation and a cross entropy loss? Why do we need Softmax and what is the real intuition behind this please?
I have encountered a lot of web pages that lists a lot of equations but unfortunately none of them have managed to explain this clearly.
Or is my understanding that the neural network trying to fit the distribution of each of the classes around the training examples wrong in the first place?
I would appreciate if you throw some light on the intuition behind the role of logistic regression here.
The answer will be highly appreciated!