I have a classification model which has been trained on a simulated training set, with a given mean and std deviation which of course I apply also on the (simulated) testing data.
Now I received new (real) data, whose mean and std dev are however very far from the ones computed on the simulated set (the mean is almost 10 times bigger). The problem is even worse since one of the classes has a higher mean on the simulated data, so that my new data is always classified like that if I just perform inference on the new data with the model I have.
I would like to perform fine-tuning with the new data, but my problem is how should I standardize it? With the new mean and std dev or with the old ones? Note that the network will be used to perform inference on real data that will have roughly the same distribution of the real data I just received.