We have built a machine learning classifier for some experimental data. During this process, we performed discretization on the continuous target variable using its median as a threshold. We would like to test our classifier on a similar new study. The problem we are facing is that the median of the new input data is significantly different than that of our original data. This would mean that the classifier will produce erroneous predictions. How should we address this issue? Do we need to normalize the new input data? If so, then what method would be best?
The correct way to do it would be use the median for the train data as the threshold as the transformations that are applied on the training set are to be replicated as it is on the test set.
The same is true whenever you are performing normalization, also check out this link on scikit-learn. Notice that the fit transform method is called on the train set and only transform is called on the test set meaning that the transformations are repeated.
You need to perform normalization on the test data (your new input data) only if you have done so on the train set(your experimental set) ,and in that case just call transform and not fit transform on your test set. Whether to perform normalization or not depends upon the algorithm used for machine learning for example Random forests can work without normalization also but logistic regression would require normalizing.
Instead of median of the continuous valued feature, why not just divide them into bins and use one hot encoding for these bins. Whatever new values that you will get will definitely conform to a particular bin.