# Cross entropy-equivalent loss suitable for real-valued labels

I am building a model whose outputs are between 0-1 and the goal is to minimize a cost function over the predicted values and labels. So far everything seems to be easy but my labels are real-valued and therefore, I cannot use the ordinary cross entropy loss function. For instance, suppose the predicted value is 0.2 and the label value is 0.2. Then simply applying the cross entropy will not give the output of 0 (as the desired output) and still generates gradients unnecessarily. Another problem is when the prediction value is greater than the corresponding label the cost ordinary cross entropy does not output sensible outputs. So I wonder, is there an equivalent version of the cross entropy function which deals with continuous [0, 1] labels?

I may note that I have used loss functions such as the L2, L1 and squared loss and so far the ordinary cross entropy loss is getting me the best results! So that's why I think a cross entropy loss which is suitable for continuous labels will even work better.

For continuous distributions $p$ and $q$, the cross entropy is defined as: $$H(p, q) = -\int_{Y} p(y) \log q(y) dy$$
Just considering a single observed input/output pair $(x, y)$, $p$ would be the empirical conditional distribution (a delta function over the observed output value), and $q$ would be the modeled conditional distribution (probability of output given input). In this case, the cross entropy reduces to $-\log q(y \mid x)$. Summing over data points, this is just the negative log likelihood!