TL;DR (too long, didn't read):
I'm working on a time-series prediction problem, which I formulate as a Regression problem using Deep Learning (keras). I want to optimize for the Pearson correlation between my prediction and the true labels. I'm confused by the fact that using MSE as a proxy actually leads to better results (in terms of the correlation) than using Pearson as a loss function directly. Is it considered bad practice to use correlation metrics as loss functions for deep learning? If so, why?
I have a time-series prediction task: I observe values for $T$ consecutive time-steps and need to predict the value at time-step $T+1$. Since the values are usually in $[-200,200]$, I'm treating this as a regression problem, which I'm solving using Deep Learning (keras).
My question is regarding the choice of loss & metrics.
My data has true labels mostly around $[-10,10]$ with some extreme values. Many of the extreme values are erroneous and I don't want to shift my learning to focus on getting them right. In other words, I want to be able to catch the general trend (correctly classify period of positive versus negative values), and I can "live with" with predicting 100 instead of 200, for example.
For this reason, I think my evaluation metric should be the Pearson correlation between the predicted and true values.
Now, for the loss function: Ideally, if I want to optimize for high Pearson correlation, it would make sense to use that as the loss function, right? I've tested a simple architecture that is my "baseline model" twice: Once with using Pearson (as calculated on a mini-batch) directly as my loss function, and once with using the common MSE as a proxy. In both cases I track both MSE and Pearson for different epochs and I do "early stopping" based on a validation set.
- MSE as a loss: MSE 160, Pearson 0.7
- Pearson as a loss: MSE 250, Pearson 0.6
I understand the higher MSE for the Pearson loss being the result of the fact that optimizing for correlation has no scale, so all the prediction can be "off" by a factor in a way that increases the MSE. But how come using MSE as a proxy actually does better in terms of the Pearson correlation itself? Is there any optimization-related reason as to why Pearson correlation shouldn't be used as a loss function? Indeed, I see that in practice it's hardly used, but I would like to understand the reason behind this.