I recently presented my M.D work and one question got me into trouble.
Context:
In my neural network classification, I use 1 single output sigmoid neuron.
My problem requires that i classify the input into two groups : 0 or 1.
For the loss function, I use lasagne.objectives.binary_crossentropy :
Computes the binary cross-entropy between predictions and targets.
L= $−tlog(p)− (1−t) log(1−p)$
Question made in my presentation by a teacher
"You cant be using binary cross-entropy because entropy is just used when your data either has noises or some probability, why and how do you use that loss function? You cant use it because for each input the model already classify it to either 1 group, so 'entropy' makes no sense".
Im confused, especially by the last part, because my model has a sigmoid output that ranges from 0 to 1. I know that the output from the sigmoid neuron isnt exactly a probability, However his question doesnt make sense to me.
Would someone help me come up with an answer for that?
Sources:
http://lasagne.readthedocs.io/en/latest/modules/objectives.html#loss-functions