L1 loss giving a better result than L2 loss for optimizing PSNR in an image super resolution problem

Question

in an Image Super Resolution kind of problem, I want to get the highest PSNR values for the super resolution images from the low resolution images obtained after training a model. I experimented with L1, L2 and PSNR loss.

What I observed was the final output in terms of PSNR was better for the model trained on L1 than L2 and L2 than PSNR. So L1 > L2 > PSNR

Isn't this counter-intuitive? At each epoch, the PSNR on the validation set is calculated and based on that value the back-propagation happens. So shouldn't the model performance be PSNR > L2 > L1? L2 should be greater than L1 since it is a squared error something like PSNR denominator term.

Why is this happening? Am I missing something here?

P.S: PSNR loss is simply the negative of the PSNR value between the super resolution image from the model and the high resolution ground truth.

Answer 1

It is surely unexpected. This phenomenon was also mentioned in the paper Loss Functions for Image Restoration with Neural Networks. Briefly, they have discussed that although $\ell_2$ loss is widely used in the image restoration tasks, it does not bear appealing results, especially for the human visual system. And they have tried some other functions as a loss and analyzed. Moreover, they came across the thing you asked. Here is how they interpreted that:

We hypothesize that this result may be related to the smoothness and the local convexity properties of the two measures: $\ell_2$ gets stuck more easily in a local minimum, while for $\ell_1$ it may be easier to reach better minimum, both in terms of $\ell_1$ and $\ell_2$ — the “good” minima of the two should be related, after all. To test this hypothesis, we ran an experiment in which we take two networks trained with $\ell_1$ and $\ell_2$ respectively, and train them again until they converge using the other loss. Figure 7 shows the $\ell_2$ loss computed on the testing set at different training iterations for either network. The network trained with $\ell_1$ only (before epoch 1200 in the plot) achieves a better $\ell_2$ loss than the one trained with $\ell_2$. However, after switching the training loss functions, both networks yield a lower $\ell_2$ loss, confirming that the $\ell_2$ network was previously stuck in a local minimum. While the two networks achieve a similar $\ell_2$ loss, they converge to different regions of the space of parameters. At visual inspection the network trained with $\ell_2$ first and $\ell_1$ after produces results similar to those of $\ell_1$ alone; the output of the network trained with $\ell_1$ first and $\ell_2$ after is still affected by splotchy artifacts in flat areas, though it is better than $\ell_2$ alone, see Figure 8.
[Here is figure 7:]
[Here is figure 8:]

Sorry for the long quote, I hoped it helps.