Regression loss function to yield high correlation

Question

I am using a neural network to predict a target. Currently, my loss is the mean squared error. I am not interested in the absolute values of my predictions, thus I evaluate the predictions using the Pearson correlation coefficient instead of the MSE.

I have the intuition that I can improve the Pearson by optimizing it more directly. Additionally, I found in this answer that "Maximizing correlation is useful when the output is highly noisy", which is definitely the case for my data.

Is there something similar to the siamese neural network/Triplett loss approach which is as far as I know only used for classification/distinction?

Is it reasonable to just use the Pearson correlation as a loss function? I could not find much about this in the literature, apart from some comments on this site.

Answer 1

Pearson correlation as a loss function presents some problems. In particular, $\text{corr}(y, \hat y)=\text{corr}(y,a+b\hat y)$ for any real $a$ and positive $b$, so $y=(1,2,3)$ and $\hat y=(105, 205, 305)$ have perfect correlation, yet the predictions are terrible.

If you are in a situation where this is acceptable, perhaps Pearson correlation could work for you, but you should be aware of this issue.

The comment in one of your links that $R^2$, for the definition given below, is not correct. Such an equation absolutely catches the poor quality of the above predictions $\hat y$ of $y$. Please see my simulations here, but it makes sense. The below formula is just a monotonic transformation of the MSE.

$$ R^2=1-\left(\dfrac{ \overset{N}{\underset{i=1}{\sum}}\left( y_i-\hat y_i \right)^2 }{ \overset{N}{\underset{i=1}{\sum}}\left( y_i-\bar y \right)^2 }\right) $$