I am building a model whose outputs are between 0-1 and the goal is to minimize a cost function over the predicted values and labels. So far everything seems to be easy but my labels are real-valued and therefore, I cannot use the ordinary cross entropy loss function. For instance, suppose the predicted value is
0.2 and the label value is
0.2. Then simply applying the cross entropy will not give the output of
0 (as the desired output) and still generates gradients unnecessarily. Another problem is when the prediction value is greater than the corresponding label the cost ordinary cross entropy does not output sensible outputs. So I wonder, is there an equivalent version of the cross entropy function which deals with continuous [0, 1] labels?
I may note that I have used loss functions such as the L2, L1 and squared loss and so far the ordinary cross entropy loss is getting me the best results! So that's why I think a cross entropy loss which is suitable for continuous labels will even work better.