I'm trying to train a neural network for classification, but the labels I have are rather noisy (around 30% of the labels are wrong).
The cross-entropy loss indeed works, but I was wondering are there any alternatives more effective in this case? or is cross-entropy loss the optimal?
I'm not sure but I'm thinking of somewhat "clipping" the cross-entropy loss, such that the loss for one data point will be no greater than some upper bound, will that work?
Thanks!
Update
According to Lucas's answer, I got the following for the derivatives for the prediction output $y$ and input of the softmax function $z$. So I guess essentially it is giving an upper bound and lower bound to the refined output $p$ (and therefore the loss as well). Please let me know if I'm wrong. Thanks!
$$p_i=0.3/N+0.7y_i$$
$$l=-\sum t_i\log(p_i)$$
$$\frac{\partial l}{\partial y_i}=-t_i\frac{\partial\log(p_i)}{\partial p_i}\frac{\partial p_i}{\partial y_i}=-0.7\frac{t_i}{p_i}$$
$$\frac{\partial l}{\partial z_i}=0.7\sum_j\frac{t_j}{p_j}\frac{\partial y_j}{\partial z_i}=0.7(y_i\sum_jt_j\frac{y_j}{p_j}-t_i\frac{y_i}{p_i})$$