Calculation of the Generalized Dice Loss Gradient

Question

I am trying to understand the gradient of the Generalized Dice Loss (GDL) shown here Link. It says that the GDL for two classes is: $$ GDL = 1 - 2 \frac{\sum_{l=1}^2w_l \sum_{n=1}^{N} r_{ln}p_{ln}}{\sum_{l=1}^2w_l \sum_{n=1}^{N} r_{ln} + p_{ln}} $$ Also, it says that the gradient with respect to $p_i$ is: $$ \frac{\partial GDL}{\partial p_i} = - 2 \frac{ (w_1^2-w_2^2) \left[ \sum_{n=1}^N p_nr_n - r_i \sum_{n=1}^N (p_n + r_n) \right] + Nw_2(w_1+w_2)(1-2r_i)}{ \left[ (w_1-w_2) \sum_{n=1}^N(p_n+r_n) + 2Nw_2 \right]^2 } $$

My two questions are:

1. How this gradient was calculated? Shouldn't it be something like: $$ \frac{\partial GDL}{\partial p_i}= -2\frac{\left[ \sum_{l=1}^2 w_l \sum_{n=1}^{N} r_{ln} + p_{ln} \right] \left( \sum_{l=1}^2 w_l \sum_{n=1}^{N} r_{ln} \right) + \left[ \sum_{l=1}^2 w_l \sum_{n=1}^{N} r_{ln}p_{ln} \right] \left( N \sum_{l=1}^2 w_l \right)}{\left[ \sum_{l=1}^2 w_l \sum_{n=1}^{N} r_{ln} + p_{ln} \right]^2} $$
2. I assume that to extend it to more than two classes would be to make $l$ vary from $1$ to $L$, where $L$ is the number of classes. But how is this gradient extended?

Thanks

Answer 1

How this gradient was calculated?

The referenced article doesn't use the same subscripts as in your expression. If you substitute $p_{i1}=p_i$ and $p_{i2}=1−p_i$ then you should be able to obtain the same expression.

I haven't being able to understand what happens when the number of classes is increased.

In this case you will have to come up with similar expressions that relate to the relationship $\sum p_{ik} = 1$ such that a change in one of these $p_{ik}$ must be accompanied by a change in the other parameters. It is a bit arbitrary how you do this.

If you are using these gradients for a minimisation problem, then an alternative approach could be to ignore the constraints in the computation of the derivatives. That is, compute the derivatives $\frac{\partial GDL}{\partial p_{ik}}$ instead of some $\frac{\partial GDL}{\partial p_{i}}$. Then make the updating step constrained (change the $p_{ik}$ such that their sum remains equal). A simple way would be to decrease the $p_{ik}$ with the highest derivative at the cost of an increase of the $p_{ik}$ with the lowest derivative. Another way would be to have some weighted step where all $p_{ik}$ are changed in some way. One possibility for that would be to maximize

$$\sum d_{ik} \frac{\partial GDL}{\partial p_{ik}} \qquad \text{subject to the constraints $\sum d_{ik} = 0$ and $\sqrt{\sum d_{ik}^2} = 1$}$$

(ie. we choose the vector with $\sum d_{ik} = 0$ along which the directional derivative of $GDL$ is highest)