0
$\begingroup$

I use min-max normalization for my training data, so each variable should have range, and then I use de-normalization for my test output value to compare the actual output value. My question is: 1. Does my output value should locate between minimum and maximum? 2. If there is an extra variable out of range, do I need to redo the model?

$\endgroup$
0
$\begingroup$

You should get the minimum and maximum from the training data alone, otherwise you're fooling yourself by letting the test data leak into the training data.

If you do that, then the range of the de-normalized estimates will not necessarily cover the range of the test points. You can have 'stranded' test points outside the normalization range. Paraphrasing your question 1: "Does my output value fall within min-max?" The answer is yes, and that may not be enough range to cover all the test points.

For question 2 it depends on what you want. You could choose a different normalization method which doesn't strand any of your test points. On the other hand, if you're getting a good estimate using min-max normalization, you could accept the stranded test data point errors just as we have to accept errors in estimating the other test points.

$\endgroup$
  • $\begingroup$ Another question, do I have to normalize data before training? Could I set the log-sigmoid transfer function in hidden layer and linear transfer function in output layer to train the model without normalize data? $\endgroup$ – Jeffrey May 9 '17 at 23:56
  • $\begingroup$ @Jeffrey : Yes you could, but it may not perform very well. Normalizing the input data such that the mean is zero and the standard deviation is one centers it in the sigmoid such that the neurons are in the active portion of their transfer function. If your un-normalized data is out on the flat tails of the sigmoid the neurons will have a low-response to changes in the input. $\endgroup$ – Keith Brodie May 10 '17 at 18:59
  • $\begingroup$ I have tried to search other normalization methods to see if there is any method can not be limited by maximum value, but I didn't find anything. No matter Z-score, min-max normalization or log(x)/log(max.x), the test points also are limited by mean and maximum value. Do you think I can use Decimal Scaling to increase the range for my forecasting model? Because my variables are silt, moisture, weight of truck, etc., so they shouldn't out of that range under normal condition. Is that make sense? $\endgroup$ – Jeffrey May 11 '17 at 4:44
  • $\begingroup$ @Jeffrey : I'm not sure this helps - but that's nearly always the case when predicting an analog value. No matter what range we allow the de-normalization to cover, there may be at some future time a data-set applied to the network with a result we cannot estimate exactly because of the restricted range of our output. So we have to make a judgement call. If we choose a normalization method that covers 99.99% of all of the output values we've ever seen - is that good enough for this estimator to do what you want it to do? $\endgroup$ – Keith Brodie May 12 '17 at 19:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.