While the Bias-Variance decomposition of the squared loss is part of any introductory ML class, I am curious to know if similar decompositions can be done for other loss functions, e.g., cross entropy?
These two papers might be relevant:
- Shivaswamy, et al, Bias-Variance Decomposition for Ranking, WSDM 2021
- Yang, et al, Rethinking Bias-Variance Trade-off for Generalization of Neural Networks, PMLR 2020
The latter states some result for binary cross entropy.