# Why does $[0,1]$ scaling dramatically increase training time for feed forward ANN (1 hidden layer)?

I am using an ANN for classification. My covariates are the relative lagged returns for gold, SPX and Oil.

When i do not scale my inputs between 0 and 1 I achieve a fast training time. however after scaling my training time increases dramatically, sometimes 40 fold.

My accuracy is also better when inputs are not scaled. I cant seem to find any reason for this online. Has anyone got any ideas?

• Is this also true for $z$-score scaling (0-mean, unit sd)? – Sycorax Aug 30 '18 at 17:36
• No, this only appears to happen when scaling the relative returns between 0 and 1. I realize the relative returns are scaled and the model preforms well when using these. But when i scale all covariates between 1 and 0 my accuracy for the training set decreases and the time taken to train my model increases alot. – tonyf Aug 30 '18 at 17:56
• How are you initializing the parameters? – user20160 Aug 30 '18 at 17:57
• In terms of my layers? I have 21 neurons in my input layer, one in my output. i vary the number of perceptions in my hidden layer between 1 and 20. The training time for my unscaled data remains under 0.5 seconds throughout. When i used the scaled data, beyond 4 hidden layer perceptrons it will take anything between 5 and 20 seconds to train. Is there any explanation for this? – tonyf Aug 30 '18 at 18:04
• Do you scale your learning rate appropriately? The magnitude of your gradient may be wildly different. – kbrose Aug 30 '18 at 20:35

We can find a reasonable explanation for this behavior in the Neural Network FAQ. TL;DR - try rescaling your data to lie in $[-1,1]$.
The key detail that makes me think that this is the answer is because you do not observe that it takes a long time to train the network when you use $z$-scores, which have negative and positive input values due to the mean-centering.