ML newbie here. If whatever information I have provided is not sufficient feel free to let me know what more I need to add. Now, the question:

I am working with multi-task Gaussian processes. I have 3 dimensional real vectors as inputs and 3 dimensional real vectors as outputs. The training data has been cleaned of Nans and anomalies. The training data is normalized by subtracting the mean and standard deviation along each dimension of the data.

For testing, I am feeding the model individual test samples (and NOT batches). But before the test sample is sent in, I subtract the mean of the training data from this test sample and divide it by the training data's standard deviation.

I compute the error of the output. When I collect these errors I find that their mean is [-0.12, -0.08, -0.14] and that their standard deviation is [0.62, 0.20, 0.34].

What surprises me is that the mean of the errors is not zero and the standard deviation along any dimension is pretty far from being 1.

Am I expecting something completely unjustified? If yes, why? If no, what do you think I can try doing to better understand the issue?

Some more information:

As a next step, I whitened each error vector using the predictive covariance matrix for it and then found that the resulting set of whitened errors does not form a spherical Gaussian - again, something that I was not expecting.

I am using GPyTorch for the multi-task Gaussian process implementation.