I'm using Pearson and Spearman correlation between predicted value and ground truth to evaluate the performance of my model (a deep neural network). On my first dataset the longer I train my model the better Pearson and Spearman correlation are, but on my second dataset meanwhile Pearson increase, Spearman decrease. How is that possible? If the linear correlation (Pearson) increase, the non-linear correlation (Spearman) should be increasing too? I don't know how to interpret those results.
EDIT:
I have a vector with real scores and a vector with predicted scores, I just a calculate the correlation between both of them at each step of the training. For the first test set both Pearson and Spearman increase during the training, while for the second test set Pearson increase and Spearman decrease over the training.
EDIT 2:
A lot of people don't understand what a PCA is or just don't even read what I wrote, so I deleted this graph to avoid confusion.
EDIT 3:
I'm calculating is the one between the orange (predicted semantic similarity score) and blue curves (semantic similarity based on human review) below :
First testSet
Second testSet
mains differences between both test sets are the number of samples (4 times higher in second) and the score distribution.
For both testSets Pearson correlation increase during training:
But for the second test set Spearman correlation decrease:
I should have posted those graphs earlier. By the way, the difference in performance between both test set isn't the problem, it's really the fact that Spearman correlation decrease while Pearson correlation increase for the second test set that bothers me.
PS:
I asked this question here in the first place, but I didn't get any answer