Data normalization and classification

Question

I hope this clearly states the problem I have in hand. Here goes:

I've trained a neural network with one initial data set that was normalized in order to guarantee an equal participation of each variable in the learning process. Once the the neural network is trained I'll have new sets of data coming to be classified.

Q1: How should I proceed in terms of normalizing a new coming data set? Is there any chance that a different mean and standard deviation will make this new data set significantly different from the one I originally used to train my classifier? Can this generate "wrong" results? (of course I'm considering the new income samples to come from the same source as the original training set).

Q2: Am I supposed to follow the same normalization procedure if instead of just classifying I'm first clustering the training data set and them classifying new income samples (into the clusters/groups found by my clustering algorithm)?

Answer 1

The catch with learning systems that is not usually mentioned in the books is that your variables should represent data coming from some stationary process or at least a wide sense stationary process (there are some systems trying to overcome this but I will not get into that now). In such cases the mean and variance of the data is assumed to be stable. For that reason mean and variance of the training and test sets should be equivalent. Therefore the initial transformation should work for both sets. If the data is not coming from such a process then you need to find a transformation that does the trick (but most of the time this is not as easy as it sounds).

Answer 2

Q1

I find the distinction in preprocessing such as normalization and modeling misleading and rather recommend to treat the preprocessing steps as part of the model.

This point of view makes it clear that you center with the centering vector and standardize with the factors calculated for your training data.

Predictions should be possible for each single new case - independent of possible other new cases. It is not possible to derive meaningful center or standardization factors of one single case. Also, a single case is assumed to come from exactly one class, therefore it can never be representative for the whole problem. So yes, mean and standard deviation for new cases can vary significantly from the values you used in your model, and this is perfectly OK: there is no guarantee for new data to come in according to the prior probabilities of the classes.

The situation is different for operations that are performed "within" each case independently of other cases: those you may apply within each of the new cases. Examples from optical spectroscopy would be subtracting a baseline fit to each spectrum, or normalizing the total intensity of the spectrum to correct for differences in the optical path length etc.

Q2

I recommed that the primary concern is to choose a normalization procedure that makes sense with your type of data and classification problem. The particular needs of different modeling algorithms are a secondary concern for me: the numerical needs can usually be satisfied by a number of normalization strategies that can differ widely in how appropriate they are for your data and your problem.

You may want to have a look at this question: Variables are often adjusted (e.g. standardised) before making a model - when is this a good idea, and when is it a bad one?