# Analytical expression of the minimizer of cross entropy loss when the predicted function is a constant fucntion?

Let $$\{y_1...y_n\} \in \{0,1\}$$, and let $$c \in [0,1]$$. Define the cross-entropy of loss of $$c$$ by:

$$C(c): = \sum_{j=1}^{n}- y_j ln c - (1- y_j) ln (1-c)$$.

Define $$c*= arg min _{c} C(c)$$

Is there an analytical expression of this $$c*$$, in terms of $$\{y_1...y_n\}$$? I'm asking this question motivated by other two loss functions: $$L^1, L^2$$ loss, i.e. quadratic and absolute losses for $$c$$, where the median and the mean of $$\{y_1...y_n\}$$ are the corresponding minimizers of $$C(c)$$.

You're effectively interested in the Maximum Likelihood Estimate of a Bernoulli random variable with parameter $$p=c$$.

Taking the derivative in $$c$$ gives $$-\sum_{j=1}^n y_j/c +(1-y_j)/(1-c) = 0.$$

Define $$Y_n:= \sum_{j=1}^n y_j$$. Then

$$-\frac{1}{c}Y_n+(n-Y_n)\frac{1}{1-c}=0,$$

giving:

$$c=\frac{Y_n}{n}.$$

Taking a step back, this should be obvious, as the numerator counts the number of successes $$(y_i=1)$$ and the denominator counts the number of trials.