0
$\begingroup$

I am training a model to classify the traffic signs with CNN.
1. Training data: 32x32 photo of traffic signs
2. preprocess:
Calculate the train_mean and train_std with all the training data. Then nor_x = (x - train_mean)/train_std
3. Training CNN model with nor_x.
4. Validating the model. Here is a strange thing I found. I preprocess the validation data with train_mean and train_std, but the result is slightly worse than using validation data's valid_mean and valid_std. I believe I should get a better result with train_mean and train_std since it is used to train a model.
Or it is a normal phenomenon and there are more other factors effecting this experiment.

Any comment or resource is welcomed. Thanks.
This question is probably duplicated of this question. But what I concerned is the result is not consistent with the correct way(the result of using training data's mean and std is worse than using validation data's mean and std).

$\endgroup$
1
$\begingroup$

Imagine the validation data has considerably more spread than the training data. You learn a parameter to scale from the training data that easily puts it In the interval [-1,1]. This parameter is not severe enough for the validation data, however. If you instead peek at the optimal scaling parameter for the validation data, you can put it easily into the desired interval. Your final learning algorithm, say, only works when the data is in said interval. You get better results with the validation parameters because they truly scale that data appropriately.

They are false better results, however.

You are correct in referencing the other question and in that you should not use the validation parameters if you want an honest estimate of your model generalization error. Your model, incidentally, includes the parameters used for scaling.

$\endgroup$
  • $\begingroup$ I kind of understand what you mean. According to your point, I think I am using the wrong normalized method. Or in this task, there is no need to normalize data(since they are images, every element ranges from 0 to 255), Or if I insist normalize them, I should use nor_x = (x - 128) / 128, in this way, it guarantees that training and validation data use the same approach and the result will both land in the same interval. Am I correct? $\endgroup$ – Lion Lai Aug 2 '17 at 9:09
  • 1
    $\begingroup$ You should use the parameter from the training data (in the case of the original Q train mean and train std) and the fact that you are getting worse results is completely normal. Whether there is some better way to transform the data is a different question. If you decide to use a predefined rule like 128, it must be derived solely from the training data and not based on its effect on the validation data. $\endgroup$ – user0 Aug 2 '17 at 9:18
  • $\begingroup$ I reread your answer. "Your model, incidentally, includes the parameters used for scaling". Just want to be clear, are you infer/imply that for a model, it's possible to not include the parameters used for scaling. Since a model uses training data after scaling, it's inevitable for a model to include the parameters used for scaling? $\endgroup$ – Lion Lai Aug 2 '17 at 10:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.