# Why don't we train neural networks to maximize linear correlation instead of error?

Recently a project I've been a part of has involved training neural networks so that we maximize the Pearson correlation between actual and predicted values. So this came to my mind: why don't we change the mathematical workings of, say, gradient descent so that instead of minimizing RMSE, we maximize $r$? If we can make the network predict with a high correlation, all we have to do is chain a linear function to the predictions and we have good prediction.

Because that would be a completely different objective altogether. Note that unlike MSE, Pearson correlation is maximal iff there is a linear relationship between both variables. This means that

• The network would "think" it has correctly learned its inputs if its output is roughly proportional to the dependent variable samples, rather than equal (or similar). Therefore predicting $Y$ or $2Y$ or $-Y$ (etc.) would be equivalent. This is generally undesirable, since we would like our network to give prediction similar to its inputs, rather than proportionally to said inputs.

• There would not be a global minimum to the optimisation problem thus posed. Any proportional constant as set above would give an optimal solution. This is undesirable from a numerical point of view and would lead to instability.

• It's worth noting that we don't even maximize $R^2$ for linear regression, where it might actually make sense to do so. – shadowtalker Mar 10 '15 at 16:03
• But if we let the network converge to $r=1-\epsilon$, couldn't we just normalize the output linearly and have great predictions? – Simon Kuang Mar 10 '15 at 18:03
• 1) Assuming the network would converge and 2) You'll find (write it down) that if you constrain everything to have mean 0 and variance 1 (which is equivalent to what you call normalizing linearly), maximizing correlation and minimizing mse is strictly equivalent. – Youloush Mar 11 '15 at 10:17

It's more like a comment, though I can't comment yet.

I'd suspect that minimizing RMSE respective to a normalized input is (roughly?) equivalent to maximizing the Pearson correlation coefficient, but the latter is more computationally expensive.