For hard integer labels {0,1}, the cross entropy simplifies to the log loss. In this case, it is easy to show that minimizing the cross entropy is equivalent to maximizing the log likelihood, see e.g. https://stats.stackexchange.com/a/364237/179312
Can we also show this for soft float labels [0,1]? This thread states that the cross entropy function is also appropriate here. But how does the log likelihood function look like in this case?
Soft labels define a 'true' target distribution over class labels for each data point. As I described previously, a probabilistic classifier can be fit by minimizing the cross entropy between the target distribution and the predicted distribution. In this context, minimizing the cross entropy is equivalent to minimizing the KL divergence. So, what we're doing is finding a good approximation to the target distribution (as measured by KL divergence). However, as described below, the problem can equivalently be cast as a weighted maximum likelihood problem, where the soft labels determine the weights. I'll show this for binary classification, but the same reasoning also applies to multiclass problems.
Let $X = \{x_1, \dots, x_n\}$ be a set of data points with binary class labels $\mathbf{y} \in \{0, 1\}^n$. Assume the class labels are conditionally independent, given $X$. The class labels are unknown, but we have soft labels $\mathbf{\ell} \in [0,1]^n$, where $\ell_i$ gives the probability that $y_i=1$. The soft labels define a Bernoulli target distribution over class labels for each data point:
$$p(y \mid \ell_i) = \left\{ \begin{array}{cl} \ell_i & y = 1 \\ 1 - \ell_i & y = 0 \\ \end{array} \right.$$
The goal is to learn a conditional distribution $q(y \mid x, \theta)$ (a.k.a. probabilistic classifier, parameterized by $\theta$), such that the predicted class probabilities approximate those given by the soft labels. We do this by minimizing the cross entropy between the target and predicted distributions over class labels, summed over data points:
$$\min_\theta \ \sum_{i=1}^n H \Big( p(y \mid \ell_i), q(y \mid x_i, \theta) \Big) \tag{1}$$
Writing out the expression for the cross entropy, the problem is:
$$\min_\theta \ -\sum_{i=1}^n \ell_i \log q(y=1 \mid x_i, \theta) - \sum_{i=1}^n (1-\ell_i) \log q(y=0 \mid x_i, \theta) \tag{3}$$
Suppose we define a new dataset $(\tilde{X}, \tilde{\mathbf{y}})$ by duplicating each data point. We assign hard class label $1$ to the first duplicate, and $0$ to the second duplicate. Furthermore, we assign a weight to each new data point. The first duplicates are weighted by the soft labels, and the second duplicates are weighted by one minus the soft labels. That is:
$$\begin{array}{ccl} \tilde{X} & = & \{x_1, \dots, x_n, x_1, \dots, x_n\} \\ \tilde{y} & = & [1, \dots, 1, 0, \dots, 0]^T \\ \tilde{w} & = & [\ell_1, \dots, \ell_n, 1-\ell_1, \dots, 1-\ell_n]^T \end{array} \tag{4}$$
Intuitively, you can think of the weights as a continuous analog of 'how many times' we've seen each case. We've constructed the new dataset in a way that translates soft labels into 'replications'. For example, if a point has soft label $0.75$, this is like seeing the same point three times with hard label $1$ and once with hard label $0$ (giving weights .75 and .25, respectively).
As above, we want to learn a conditional distribution $q(y \mid x, \theta)$, but this time using the new dataset with hard labels and weights. We do this by maximizing the weighted likelihood:
$$L_{\tilde{w}}(\theta; \tilde{X}, \tilde{\mathbf{y}}) = \prod_{i=1}^{2 n} q(\tilde{y}_i \mid \tilde{x}_i, \theta)^{\tilde{w}_i} \tag{5}$$
This is equivalent to minimizing the weighted negative log likelihood:
$$-\log L_{\tilde{w}}(\theta; \tilde{X}, \tilde{\mathbf{y}}) = -\sum_{i=1}^{2 n} \tilde{w}_i \log q(\tilde{y}_i \mid \tilde{x}_i, \theta) \tag{6}$$
Substitute in our expressions for $\tilde{X}, \tilde{\mathbf{y}}, \tilde{w}$:
$$\begin{matrix} -\log L_{\tilde{w}}(\theta; \tilde{X}, \tilde{\mathbf{y}}) = \\ -\sum_{i=1}^n \ell_i \log q(y=1 \mid x_i, \theta) - \sum_{i=1}^n (1-\ell_i) \log q(y=0 \mid x_i, \theta) \end{matrix}\tag{7}$$
The weighted negative log likelihood in $(7)$ is the same as the cross entropy loss in $(3)$. So, the weighted maximum likelihood problem here is equivalent to the cross entropy minimization problem above.
If we consider a continuous relaxation of Bernoulli that allows the true probability to be between 0 and 1, a recent paper argues [1] that, no, cross-entropy is not adequate for $y \in [0,1]$, because it is not a Bernoulli distributed variable. While their work is concerned with Variational Autoencoders, the argument can be extended to other uses of the Bernoulli likelihood. The continuous $y$ can be regarded as a soft-label.
A Beta distribution could be used instead, but they also propose a new distribution that augments the Bernoulli, which entails a simple correction to cross-entropy.
The Continuous Bernoulli distribution is given by, with $\lambda \in (0,1)$, $x \in [0,1]$:
$$p_{\mathcal{CB}}(x|\lambda) = C(\lambda)\lambda^x(1-\lambda)^{1-x}$$
Contrast it with the original Bernoulli, with $p \in (0,1)$, $ k \in \{0,1\} $:
$$p_{\mathcal{B}}(k|p) = p^k(1-p)^{1-k}$$
The Continuous Bernoulli is proportional to the Bernoulli, but with continuous $k$, and the correction term is introduced to make it a valid distribution.
The new cross-entropy then is:
$$\mathcal L(\hat y, y) = y\log(\hat y) + (1 - y) \log(1-\hat y) + \color{red}{\log C(\hat y)}$$
This last term, the normalizing correction, is given by:
$$C(x) = \begin{cases} \begin{align} &\frac{2\tanh^{-1}(1-2x)}{1-2x} \quad &\text{if} \quad x \neq 0.5\\ &2 \quad &\text{if} \quad x = 0.5 \end{align} \end{cases}$$
