I have a classification model which has been trained on a simulated training set, with a given mean and std deviation which of course I apply also on the (simulated) testing data.

Now I received new (real) data, whose mean and std dev are however very far from the ones computed on the simulated set (the mean is almost 10 times bigger). The problem is even worse since one of the classes has a higher mean on the simulated data, so that my new data is always classified like that if I just perform inference on the new data with the model I have.

I would like to perform fine-tuning with the new data, but my problem is how should I standardize it? With the new mean and std dev or with the old ones? Note that the network will be used to perform inference on real data that will have roughly the same distribution of the real data I just received.

  • $\begingroup$ Just checking, by inference do you mean prediction? Unfortunately, that word now has two meanings: statistical inference, which goes back to fisher, and now part of the community is using it to also mean prediction. I take you mean prediction here? $\endgroup$ – Matthew Drury Mar 8 at 17:08
  • $\begingroup$ It seems like your simulated training data is missing something very important about your real world data. Do you have a compelling reason for not training on real data from the population the model will actually be applied to? $\endgroup$ – Matthew Drury Mar 8 at 17:17
  • $\begingroup$ Yes, by inference I mean prediction. Regarding the reason about using simulated data is that unfortunately I don’t have a lot of real data. Nevertheless the simulated images are visually good, the only problem is the fact that their distribution is not the same as the real one (and I didn’t know it before). $\endgroup$ – a_silve Mar 8 at 21:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.