While the Bias-Variance decomposition of the squared loss is part of any introductory ML class, I am curious to know if similar decompositions can be done for other loss functions, e.g., cross entropy?
$\begingroup$
$\endgroup$
2
-
$\begingroup$ A near-duplicate: stats.stackexchange.com/questions/192286/… See through the list stats.stackexchange.com/search?q=bias+varian*+decompo*+answers%3A1 $\endgroup$– kjetil b halvorsen ♦Feb 23, 2019 at 10:17
-
$\begingroup$ Do the following search in google scholar: "bias-variance tradeoff for general loss function" (without the quotes). Many very relevant hits $\endgroup$– kjetil b halvorsen ♦Mar 5, 2019 at 21:18
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
These two papers might be relevant:
- Shivaswamy, et al, Bias-Variance Decomposition for Ranking, WSDM 2021
- Yang, et al, Rethinking Bias-Variance Trade-off for Generalization of Neural Networks, PMLR 2020
The latter states some result for binary cross entropy.
answered Aug 19, 2021 at 7:55